Memory Operator function

seshadotcom

Hello Rapidminer team,

I have a question regarding the memory operator. I am using it to Free Memory in my Strategy/Process after each and every block, I want to really see if it does free some memory and if it does how many KB or MB is freed.. Is there a way to look at this from the logs?? I could not find any GetMemory functions in the rapidminer to see what is the available memory.

Also it would be nice to have a prediction of how much memory is needed for the data I am going to pass to for example to the CreateAssociationRule Operator. Right now this is kind of a black box for me , The createAssociationRule runs for ever and after a long time I get a memory error, so would be nice to have a real time status of memory. This is how my process looks like now and I am not trying to fetch and create rules for more than 10000 rows.. I did try changing JAVA HEAP SPACE in the environment variable but unfortunately it did not help me

        subprocess 'Main Process'
            +- Transaction_Join1[1] (Read Database)
            +- order_header[1] (Read Database)
            +- Join_T_OH[1] (Join)
            +- Free Memory[1] (Free Memory)
            +- order_line[1] (Read Database)
            +- Join[1] (Join)
            +- Free Memory (2)[1] (Free Memory)
            +- Read State[1] (Read Database)
            +- Join (2)[1] (Join)
            +- Write CSV[1] (Write CSV)
            +- Set Role ID[1] (Set Role)
            +- Set Role Label[1] (Set Role)
            +- Select Attributes[1] (Select Attributes)
            +- Numerical to Binominal[1] (Numerical to Binominal)
            +- FP-Growth[1] (FP-Growth)
            +- Free Memory (3)[1] (Free Memory)
      ==>   +- Create Association Rules[1] (Create Association Rules)



Regards
Sesha

Marco_Boeck

Hi,

you can have a look at the System Monitor. To enable it, go to "View" -> "Show View" -> "System Monitor", it displays the past and current memory usage.

Regards,
Marco

seshadotcom

Hi Marco I did try to do this but it only gives me the real time consumption of resources, but what I need is I would like to know what kind of memory would be required at maximum for a resource for processing say x number of tuples with y attributes so based on that I could limit my query.- I know it is realistically very complex but still a wish from a modest student

Now since I do not have enough memory everytime it crashes so my idea was I use the allowable resources that rapidminer could use and I have a contented life - may be could be an enhancement.