Using Macro value as Loop parameter iterations

colo

Hello everybody,

today I needed the functionality of a simple FOR-loop in RapidMiner for the first time. So I took a look at the Loop operators and picked the simple "Loop". The number of iterations depends on some information extraction and preprocessing, so I prepared a nice attribute holding a number of type Integer for my loop. I just extract this cute litte Integer via "Extract Macro" and feed it into the iterations parameter of my "Loop" operator. But unfortunately the macro value seems to be of type Real and causes an error (extracted from value 5 of an integer attribute):

 Message: A value for the parameter 'iterations' must be specified! Expected integer but found '5.0'.

I couldn't find a way to manipulate the data type of the macro, so I'm a bit perplexed. I can hardly believe that it's not possible to loop with a variable amount of iterations. I want to do something simple like for(i=1; i <= variableValue; i++). Is this currently not possible with "Loop" or am I completely off the track?

Kind regards,
Matthias

land

Hi Matthias,
this is of course ugly. But you can avoid this trap, if you use a generate macro operator before and put the iterations macro into an floor() function. Assign this to a new macro called for example intIterations and put this into the loop operator. Should work this way

Greetings,
  Sebastian

colo

Hi Sebastian,

thank you - this workaround helped out. After leaving this problem behind a new question arose.
The previously mentioned "Loop" operator runs inside a "Loop examples" and does some things according to a special attribute value from the current example. This is why I needed to extract this into a macro. My next problem is that I need to do some special things in the first run of the loop. I usually would check against my count variable or set some boolean flag. A simple if-statement would distinguish between the first and all following loop iterations. But I was a little confused that it's not possible to check a macro value inside the "Branch" operator. But checking a variable value with "if" is exactly what i would usually do. "Branch" allows only to check the existence of a macro, this would work well for a simple flag. But since there seems to be no way to unset a macro once it is defined, this would just work for my first example. I guess there is a (simple) way to differentiate the loop iterations or to have a special first run but I can't figure it out. Sometimes it's still hard to find the right things to do in RapidMiner analog to the things I would directly code in Java. But experience is slowly rising... not least due to your hints and help

Thanks again and best regards,
Matthias

land

Hi,
unfortunately not. Please do me the favor of adding an macro unset operator as well as an condition on a macro's value to the feature request list on our bug tracker.

A not very nice way out: Use the remember / recall operators to store a dummy object in memory. This can be retrieved and deleted. If retrieved and not available an exception will be thrown. If wrapped inside a handle Exception operator, it can avoid the execution of the following operators inside the handle execution operator.

Wow. That was very much from the dark side.


Greetings,
  Sebastian

land

Hi,
by the way: We stumbled in one of our projects over the same issue. In the current SVN version it is already added.

Greetings,
  Sebastian

cherokee

Hi colo,

a short workaround: Before each loop initialize a macro "run" to 1; in your loop set this macro to 2; now you can use a select subprocess operator to do something else in the first run.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" expanded="true" name="Process">
    <process expanded="true" height="358" width="539">
      <operator activated="true" class="generate_data" expanded="true" height="60" name="Generate Data" width="90" x="30" y="79"/>
      <operator activated="true" class="loop_examples" expanded="true" height="76" name="Loop Examples" width="90" x="169" y="83">
        <process expanded="true" height="376" width="557">
          <operator activated="true" class="set_macro" expanded="true" height="76" name="reset runs" width="90" x="38" y="29">
            <parameter key="macro" value="run"/>
            <parameter key="value" value="1"/>
          </operator>
          <operator activated="true" class="loop" expanded="true" height="76" name="just loop 10 times" width="90" x="199" y="31">
            <parameter key="iterations" value="10"/>
            <process expanded="true" height="376" width="557">
              <operator activated="true" class="select_subprocess" expanded="true" height="76" name="do something or not" width="90" x="47" y="34">
                <parameter key="select_which" value="%{run}"/>
                <process expanded="true" height="376" width="314">
                  <operator activated="true" class="extract_log_value" expanded="true" height="60" name="Extract Log Value" width="90" x="42" y="30">
                    <parameter key="attribute_name" value="att1"/>
                    <parameter key="example_index" value="%{example}"/>
                  </operator>
                  <operator activated="true" class="log" expanded="true" height="76" name="Log" width="90" x="181" y="27">
                    <list key="log">
                      <parameter key="att1" value="operator.Extract Log Value.value.data_value"/>
                    </list>
                  </operator>
                  <connect from_port="input 1" to_op="Extract Log Value" to_port="example set"/>
                  <connect from_op="Extract Log Value" from_port="example set" to_op="Log" to_port="through 1"/>
                  <connect from_op="Log" from_port="through 1" to_port="output 1"/>
                  <portSpacing port="source_input 1" spacing="0"/>
                  <portSpacing port="source_input 2" spacing="0"/>
                  <portSpacing port="sink_output 1" spacing="0"/>
                  <portSpacing port="sink_output 2" spacing="0"/>
                </process>
                <process expanded="true" height="376" width="253">
                  <connect from_port="input 1" to_port="output 1"/>
                  <portSpacing port="source_input 1" spacing="0"/>
                  <portSpacing port="source_input 2" spacing="0"/>
                  <portSpacing port="sink_output 1" spacing="0"/>
                  <portSpacing port="sink_output 2" spacing="0"/>
                </process>
              </operator>
              <operator activated="true" class="set_macro" expanded="true" height="76" name="change run" width="90" x="189" y="32">
                <parameter key="macro" value="run"/>
                <parameter key="value" value="2"/>
              </operator>
              <connect from_port="input 1" to_op="do something or not" to_port="input 1"/>
              <connect from_op="do something or not" from_port="output 1" to_op="change run" to_port="through 1"/>
              <connect from_op="change run" from_port="through 1" to_port="output 1"/>
              <portSpacing port="source_input 1" spacing="0"/>
              <portSpacing port="source_input 2" spacing="0"/>
              <portSpacing port="sink_output 1" spacing="0"/>
              <portSpacing port="sink_output 2" spacing="0"/>
            </process>
          </operator>
          <connect from_port="example set" to_op="reset runs" to_port="through 1"/>
          <connect from_op="reset runs" from_port="through 1" to_op="just loop 10 times" to_port="input 1"/>
          <portSpacing port="source_example set" spacing="0"/>
          <portSpacing port="sink_example set" spacing="0"/>
          <portSpacing port="sink_output 1" spacing="0"/>
        </process>
      </operator>
      <connect from_op="Generate Data" from_port="output" to_op="Loop Examples" to_port="example set"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
    </process>
  </operator>
</process>

Best regards,
chero

colo

Thank you for your good advices. I added both feature requests to the bug tracker even if I didn't really know something witty to write into bug description.
It seems cherokee's workaround is not as dark as Sebastian's way I didn't even take the "Select Subprocess" operator into consideration because I never used or needed it before. But in this case it should be useful. I chose another way and simply added a flag attribute into my first example set from where the inner loop is started. Not so nice and clean but a quick way to get my process running  ::)

Sebastian do you have any idea when the current SVN version will be released to public?

Best regards,
Matthias


By the way: macro handling could perhaps need some other revision. I needed an iteration counter variable for my loop (because %{a} doesn't begin with 1 for each start of the loop). Seemed easy, "Generate Macro" before my "Loop" operator and increase count by "Set Macro" inside the loop. But the latter didn't allow me something like %{counter}+1 because the value was interpreted as string. So I had to overwrite the existing macro with "Generate Macro" which allows some functions and calculations. If I didn't absolutely fail in using "Set Macro" this non-uniform value setting and handling is a bit confusing.

land

Hi Matthias,
in fact a better support for macros is already planned for RapidMiner 6.0. Then macros will be added to the meta data transformation to allow checking and selecting the already defined macros in the gui...

Greetings,
  Sebastian

colo

Hi Sebastian,

this sounds good!

But for now I have to refer to the old story about Branch and Macros once again But this time it's just a short note in the margin.
I'm now using the latest version of RM with text and web extensions (by the way - great that extensions are available through subversion now) from subversion which already has "expression" as condition type implemented for the Branch operator. I just selected the condition type "macro_defined" unintentionally and discovered a change for this parameter. This selection does no longer activate a text parameter field where you can enter the respective macro name but shows an "io object" selection field instead. I guess this was not intended and you might want to fix it.

Regards,
Matthias

land

Hi,
thanks a lot for this hint.

Greetings,
  Sebastian

land

Hi,
as I just heard: It's already fixed.

Greetings,
  Sebastian

fstarsinic

Neither of these seem to be working for me when doing a setMacro to update the "round" variable which was initialized to 1 elsewhere:   

• eval(%{round} + 1)
• eval(%{round}) + 1

the results either stay 1 or end up with the following result:  "eval (eval(1 + 1) + 1)"   and so on.

I thought eval was supposed to .... evaluate as numeric?

BalazsBarany

Hi @fstarsinic, 

what are you trying to do? Can you send us your process?

Are you using Set Macro or Generate Macro? Only Generate Macro evaluates expressions. You can see the difference in this process:

<?xml version="1.0" encoding="UTF-8"?><process version="9.2.001">

  <context>

    <input/>

    <output/>

    <macros/>

  </context>

  <operator activated="true" class="process" compatibility="9.2.001" expanded="true" name="Process">

    <parameter key="logverbosity" value="init"/>

    <parameter key="random_seed" value="2001"/>

    <parameter key="send_mail" value="never"/>

    <parameter key="notification_email" value=""/>

    <parameter key="process_duration_for_mail" value="30"/>

    <parameter key="encoding" value="SYSTEM"/>

    <process expanded="true">

      <operator activated="true" class="set_macro" compatibility="9.2.001" expanded="true" height="82" name="Set Macro" width="90" x="45" y="34">

        <parameter key="macro" value="round"/>

        <parameter key="value" value="1.5"/>

      </operator>

      <operator activated="true" class="generate_macro" compatibility="9.2.001" expanded="true" height="82" name="Generate Macro" width="90" x="179" y="34">

        <list key="function_descriptions">

          <parameter key="generated" value="round(eval(%{round}) + 1)"/>

        </list>

      </operator>

      <connect from_op="Set Macro" from_port="through 1" to_op="Generate Macro" to_port="through 1"/>

      <connect from_op="Generate Macro" from_port="through 1" to_port="result 1"/>

      <portSpacing port="source_input 1" spacing="0"/>

      <portSpacing port="sink_result 1" spacing="0"/>

      <portSpacing port="sink_result 2" spacing="0"/>

    </process>

  </operator>

</process>

Regards,
Balázs