The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
How to avoid out of memory when running FSSs?
IngoRM
Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 
RM Founder
Original message from SourceForge forum at http://sourceforge.net/forum/forum.php?thread_id=2031544&;forum_id=390413
Hi: I have a microarray dataset with 7079 attributes (a lot of them). So, when I'm trying to execute any FSSs operator to reduce the dimensionality the RapidMiner is not able to execute the algorithm, a out-of-memory error message appears. I have 2 MB of RAM and set the virtual memory to 4 MB. Is it possible to execute RapidMinner's search algorithms using a such high dimensional problem? Any technical suggestions to solve this situations are welcome, please? Gladys
Edit by Gladys:
Sorry, I mean 2 GB of RAM, not MB. I use Windows XP
Answer by Ingo Mierswa:
Hi Gladys,
first try for memory problems in feature selection should be to reduce the number of individuals, e.g. using a 1+1 genetic algorithm using only one individual. This should always work when the data fits into memory. Other approaches like Forward Selection or Backward elimination will hardly work on data sets with this amount of features. You could also try to apply a feature weighting first and (moderately) filter out features by means of the AttributeWeightSelection operator. Search in the forum for this, there were several discussions on selecting features for high-dimensional data sets.
Cheers,
Ingo
Hi: I have a microarray dataset with 7079 attributes (a lot of them). So, when I'm trying to execute any FSSs operator to reduce the dimensionality the RapidMiner is not able to execute the algorithm, a out-of-memory error message appears. I have 2 MB of RAM and set the virtual memory to 4 MB. Is it possible to execute RapidMinner's search algorithms using a such high dimensional problem? Any technical suggestions to solve this situations are welcome, please? Gladys
Edit by Gladys:
Sorry, I mean 2 GB of RAM, not MB. I use Windows XP
Answer by Ingo Mierswa:
Hi Gladys,
first try for memory problems in feature selection should be to reduce the number of individuals, e.g. using a 1+1 genetic algorithm using only one individual. This should always work when the data fits into memory. Other approaches like Forward Selection or Backward elimination will hardly work on data sets with this amount of features. You could also try to apply a feature weighting first and (moderately) filter out features by means of the AttributeWeightSelection operator. Search in the forum for this, there were several discussions on selecting features for high-dimensional data sets.
Cheers,
Ingo
0
