Loop through filtered values

asav_yu

I am having trouble putting together a loop.

I have a dataset with multiple products. Let's say I want to run the model three times for 3 different products. I created 3 macros with product names.

Then in my filter Product Name = Macro 1, run model on that filtered dataset. Then how do I create a loop so it goes Product Name = Macro 2 etc. run the model again for this dataset and so on for all specified macros ?

Really appreciate any help.

varunm1

Hello @asav_yu

Looks like a macro inside another macro is not possible. Did you try "loop values" operator for your requirement? This will filter data based on attribute value automatically. Please see below XML with titanic dataset.

<?xml version="1.0" encoding="UTF-8"?><process version="9.3.001">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="9.3.001" expanded="true" name="Process">
<parameter key="logverbosity" value="init"/>
<parameter key="random_seed" value="2001"/>
<parameter key="send_mail" value="never"/>
<parameter key="notification_email" value=""/>
<parameter key="process_duration_for_mail" value="30"/>
<parameter key="encoding" value="SYSTEM"/>
<process expanded="true">
<operator activated="true" class="retrieve" compatibility="9.3.001" expanded="true" height="68" name="Retrieve Titanic Training" width="90" x="45" y="34">
<parameter key="repository_entry" value="//Samples/data/Titanic Training"/>
</operator>
<operator activated="true" class="concurrency:loop_values" compatibility="9.3.001" expanded="true" height="82" name="Loop Values" width="90" x="380" y="34">
<parameter key="attribute" value="Passenger Class"/>
<parameter key="iteration_macro" value="loop_value"/>
<parameter key="reuse_results" value="false"/>
<parameter key="enable_parallel_execution" value="true"/>
<process expanded="true">
<operator activated="true" class="filter_examples" compatibility="9.3.001" expanded="true" height="103" name="Filter Examples" width="90" x="246" y="34">
<parameter key="parameter_expression" value=""/>
<parameter key="condition_class" value="custom_filters"/>
<parameter key="invert_filter" value="false"/>
<list key="filters_list">
<parameter key="filters_entry_key" value="Passenger Class.equals.%{loop_value}"/>
</list>
<parameter key="filters_logic_and" value="true"/>
<parameter key="filters_check_metadata" value="true"/>
</operator>
<connect from_port="input 1" to_op="Filter Examples" to_port="example set input"/>
<connect from_op="Filter Examples" from_port="example set output" to_port="output 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
<portSpacing port="sink_output 2" spacing="0"/>
</process>
</operator>
<connect from_op="Retrieve Titanic Training" from_port="output" to_op="Loop Values" to_port="input 1"/>
<connect from_op="Loop Values" from_port="output 1" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>

Please inform if you need more information.

varunm1

Hello @asav_yu

Once you set the number of iterations to 3 in loop operator, you can write Product Name = Macro %{execution_count} . If possible rename macros to Macro_1, Macro_2, Macro_3 and try Macro_%{execution_count}. I suggested this renaming cause, I am not sure if rapidminer can deal with spaces. 

The said statement will give value Macro_1 for first execution, Macro_2 for the second execution of the loop and soon. I am just guessing your process and proposing this but if you could provide your XML the solution can be more specific.

asav_yu

@varunm1 This is great. It almost worked but after I apply a filter there are no examples left in the dataset so it can't find the filter value. Product name is polynominal so for Macro_1 I set value "Product name" in quotation marks. I can't figure out why it doesn't work. Here is a simplified version:

<div><?xml version="1.0" encoding="UTF-8"?><process version="9.3.000"></div><div>&nbsp; <context></div><div>&nbsp; &nbsp; <input/></div><div>&nbsp; &nbsp; <output/></div><div>&nbsp; &nbsp; <macros/></div><div>&nbsp; </context></div><div>&nbsp; <operator activated="true" class="process" compatibility="9.3.000" expanded="true" name="Process"></div><div>&nbsp; &nbsp; <parameter key="logverbosity" value="init"/></div><div>&nbsp; &nbsp; <parameter key="random_seed" value="2001"/></div><div>&nbsp; &nbsp; <parameter key="send_mail" value="never"/></div><div>&nbsp; &nbsp; <parameter key="notification_email" value=""/></div><div>&nbsp; &nbsp; <parameter key="process_duration_for_mail" value="30"/></div><div>&nbsp; &nbsp; <parameter key="encoding" value="SYSTEM"/></div><div>&nbsp; &nbsp; <process expanded="true"></div><div>&nbsp; &nbsp; &nbsp; <operator activated="true" class="concurrency:loop" compatibility="9.3.000" expanded="true" height="82" name="Loop" width="90" x="179" y="85"></div><div>&nbsp; &nbsp; &nbsp; &nbsp; <parameter key="number_of_iterations" value="2"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; <parameter key="iteration_macro" value="iteration"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; <parameter key="reuse_results" value="false"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; <parameter key="enable_parallel_execution" value="true"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; <process expanded="true"></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <operator activated="true" class="set_macros" compatibility="9.3.000" expanded="true" height="82" name="Set Macros" width="90" x="246" y="136"></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <list key="macros"></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <parameter key="Macro_1" value="&quot;Coke&quot;"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <parameter key="Macro_2" value="&quot;Pepsi&quot;"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </list></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </operator></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <operator activated="true" class="filter_examples" compatibility="9.3.000" expanded="true" height="103" name="Filter Examples" width="90" x="380" y="136"></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <parameter key="parameter_expression" value=""/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <parameter key="condition_class" value="custom_filters"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <parameter key="invert_filter" value="false"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <list key="filters_list"></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <parameter key="filters_entry_key" value="Model.equals.Macro_%{execution_count}"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </list></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <parameter key="filters_logic_and" value="true"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <parameter key="filters_check_metadata" value="true"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </operator></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <connect from_op="Set Macros" from_port="through 1" to_op="Filter Examples" to_port="example set input"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <connect from_op="Filter Examples" from_port="example set output" to_port="output 1"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <portSpacing port="source_input 1" spacing="0"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <portSpacing port="sink_output 1" spacing="0"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <portSpacing port="sink_output 2" spacing="0"/></div><div>&nbsp; &nbsp; &nbsp; &nbsp; </process></div><div>&nbsp; &nbsp; &nbsp; </operator></div><div>&nbsp; &nbsp; &nbsp; <connect from_op="Loop" from_port="output 1" to_port="result 1"/></div><div>&nbsp; &nbsp; &nbsp; <portSpacing port="source_input 1" spacing="0"/></div><div>&nbsp; &nbsp; &nbsp; <portSpacing port="sink_result 1" spacing="0"/></div><div>&nbsp; &nbsp; &nbsp; <portSpacing port="sink_result 2" spacing="0"/></div><div>&nbsp; &nbsp; </process></div><div>&nbsp; </operator></div><div></process></div>

asav_yu

@varunm1 Thank you. That however will cycle through all possible products for me (100+) but I also would like define which ones exactly. But I will do it another way, I will eliminate the ones i don't wont prior to Loop and then it will just loop through the ones I want.

Thanks again for your help!