The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

how to continue a loop after a timeout

suleymansahalsuleymansahal Member Posts: 27 Contributor II
edited November 2018 in Help

There is an option in loop operator which allows to limit the duration of a loop step. But when the the duration of the process exceeds the time limit it aborts the entire loop. Is there a solution, workaround to just skip that loop step and continue from the next step of the loop?

Edit: I just thought something. Will putting a loop which has one iteration with a time limit inside the real loop work?

Tagged:

Answers

  • suleymansahalsuleymansahal Member Posts: 27 Contributor II

    It did not work. The loop operator always finished the first step regardless of the timeout preference. When timeout happened the loop operator was aborted after finishing the first step. What is the rationale for this?

  • phellingerphellinger Employee-RapidMiner, Member Posts: 103 RM Engineering

    Hi,

     

    the purpose of the limit time option is to stop the entire loop, when its runtime exceeds the specified interval. It is not intended to stop an interation in the middle. It simply does not start the next iteration, if the runtime already exceeded the specified interval. Use cases that I can think of right now: a) detecting and reporting an error, when the Loop should finish in the specified time b) setting a time constraint, when there is an optimization going on inside the loop, etc.

     

    I am afraid I am not aware of an operator that could accomplish what you expect: stopping an arbitrary subprocess precisely after a specified timeout has been reached.

     

    Best,

    Peter

  • suleymansahalsuleymansahal Member Posts: 27 Contributor II

    Thank you for your prompt answer.

     

    I was comparing different models inside the loop. I was logging both performance values and execution times. Performance is not the sole consideration. Since some models like ANN take considerably higher times, I intended to skip those models which last longer than my computer can handle. Right now, I observe the log values and if I realize one of the loop steps runs too long I manually take that model out from the loop and rerun the whole process. I cannot think of any solution either.

     

    Thank you for your efforts.

  • JEdwardJEdward RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 578 Unicorn

    An interesting one... I imagine this might be possible with RapidMiner Server. 

     

    My basic flow would probably be like this.  (Please note, this is only draft to give you an idea... it's not meant to be functional.)

    1. Schedule Process inside a loop or optimize parameters to schedule the ANN training on RapidMiner Server.
    2. Delay operator to wait for a set period of time.  (the maximum timeout)
    3. Try to read the results of the ANN training after set delay, if not finished then there would be no results to read.
    4. Use Handle Exception to deal with the still running cases and stop the running ANN training process.
    5. Continue to loop until done.
    <?xml version="1.0" encoding="UTF-8"?><process version="7.3.000">
      <context>
        <input/>
        <output/>
        <macros/>
      </context>
      <operator activated="true" class="process" compatibility="7.3.000" expanded="true" name="Process">
        <process expanded="true">
          <operator activated="true" class="jdbc_connectors:read_database" compatibility="7.3.000" expanded="true" height="68" name="Read Database" width="90" x="112" y="85">
            <enumeration key="parameters"/>
          </operator>
          <operator activated="true" class="store" compatibility="7.3.000" expanded="true" height="68" name="Store" width="90" x="246" y="85"/>
          <operator activated="true" class="jdbc_connectors:read_database" compatibility="7.3.000" expanded="true" height="68" name="Read Database (2)" width="90" x="112" y="289">
            <enumeration key="parameters"/>
          </operator>
          <operator activated="true" class="store" compatibility="7.3.000" expanded="true" height="68" name="Store (2)" width="90" x="246" y="238"/>
          <operator activated="true" class="optimize_parameters_grid" compatibility="7.3.000" expanded="true" height="124" name="Optimize Parameters (Grid)" width="90" x="447" y="85">
            <list key="parameters">
              <parameter key="Param1.value" value="[1;10;9;linear]"/>
              <parameter key="Param2.value" value="[5;105;100;linear]"/>
            </list>
            <process expanded="true">
              <operator activated="true" class="subprocess" compatibility="7.3.000" expanded="true" height="82" name="Set Parameters for ANN" width="90" x="112" y="187">
                <process expanded="true">
                  <operator activated="true" class="set_macro" compatibility="7.3.000" expanded="true" height="82" name="Param1" width="90" x="45" y="34">
                    <parameter key="macro" value="param1"/>
                    <parameter key="value" value="2"/>
                  </operator>
                  <operator activated="true" class="set_macro" compatibility="7.3.000" expanded="true" height="82" name="Param2" width="90" x="246" y="136">
                    <parameter key="macro" value="param2"/>
                    <parameter key="value" value="43"/>
                  </operator>
                  <operator activated="true" class="generate_macro" compatibility="7.3.000" expanded="true" height="82" name="Unique Result Name" width="90" x="447" y="187">
                    <list key="function_descriptions">
                      <parameter key="UniqueResultName" value="eval(%{execution_count})"/>
                    </list>
                  </operator>
                  <connect from_op="Param1" from_port="through 1" to_op="Param2" to_port="through 1"/>
                  <connect from_op="Param2" from_port="through 1" to_op="Unique Result Name" to_port="through 1"/>
                  <portSpacing port="source_in 1" spacing="0"/>
                  <portSpacing port="sink_out 1" spacing="0"/>
                  <portSpacing port="sink_out 2" spacing="0"/>
                </process>
              </operator>
              <operator activated="true" class="process_scheduling:schedule_process" compatibility="7.3.000" expanded="true" height="82" name="Schedule Process" width="90" x="246" y="85">
                <parameter key="process_location" value="/process/ANN"/>
                <enumeration key="process_input">
                  <parameter key="input_location" value="/processresults/input1"/>
                  <parameter key="input_location" value="/processresults/input2"/>
                </enumeration>
                <enumeration key="process_output">
                  <parameter key="output_location" value="/processresults/%{UniqueResultName}%"/>
                </enumeration>
                <list key="macros">
                  <parameter key="param1" value="%{param1}"/>
                  <parameter key="param2" value="%{param2}"/>
                </list>
                <description align="center" color="transparent" colored="false" width="126">Schedule the ANN process to run on the server.</description>
              </operator>
              <operator activated="true" class="delay" compatibility="7.3.000" expanded="true" height="82" name="Delay" width="90" x="380" y="85">
                <parameter key="delay_amount" value="10000"/>
                <description align="center" color="transparent" colored="false" width="126">Wait for the set timeout... and then continue. The handle.</description>
              </operator>
              <operator activated="true" class="handle_exception" compatibility="7.3.000" expanded="true" height="103" name="Handle Exception" width="90" x="514" y="85">
                <process expanded="true">
                  <operator activated="true" class="retrieve" compatibility="7.3.000" expanded="true" height="68" name="Retrieve" width="90" x="45" y="85">
                    <parameter key="repository_entry" value="/processresults/%{UniqueResultName}%"/>
                    <description align="center" color="transparent" colored="false" width="126">Get the result of the process.</description>
                  </operator>
                  <connect from_op="Retrieve" from_port="output" to_port="out 1"/>
                  <portSpacing port="source_in 1" spacing="0"/>
                  <portSpacing port="source_in 2" spacing="0"/>
                  <portSpacing port="sink_out 1" spacing="0"/>
                  <portSpacing port="sink_out 2" spacing="0"/>
                  <portSpacing port="sink_out 3" spacing="0"/>
                </process>
                <process expanded="true">
                  <operator activated="true" class="productivity:execute_process" compatibility="7.3.000" expanded="true" height="82" name="Stop Running Process" width="90" x="45" y="34">
                    <parameter key="process_location" value="/utility/stopRunningProcess"/>
                    <parameter key="use_input" value="false"/>
                    <list key="macros"/>
                    <description align="center" color="transparent" colored="false" width="126">No idea how to do this part, but ideally you'd want to stop the correct ANNprocess on the server from running and receive a confirmation back.</description>
                  </operator>
                  <connect from_port="in 1" to_op="Stop Running Process" to_port="input 1"/>
                  <connect from_op="Stop Running Process" from_port="result 1" to_port="out 1"/>
                  <portSpacing port="source_in 1" spacing="0"/>
                  <portSpacing port="source_in 2" spacing="0"/>
                  <portSpacing port="sink_out 1" spacing="0"/>
                  <portSpacing port="sink_out 2" spacing="0"/>
                  <portSpacing port="sink_out 3" spacing="0"/>
                  <description align="center" color="yellow" colored="false" height="105" resized="false" width="180" x="219" y="142">Skip by if there are no errors, or deliver 0 performance values... something like that.</description>
                </process>
              </operator>
              <connect from_op="Set Parameters for ANN" from_port="out 1" to_op="Schedule Process" to_port="through 1"/>
              <connect from_op="Schedule Process" from_port="through 1" to_op="Delay" to_port="through 1"/>
              <connect from_op="Delay" from_port="through 1" to_op="Handle Exception" to_port="in 1"/>
              <connect from_op="Handle Exception" from_port="out 1" to_port="performance"/>
              <connect from_op="Handle Exception" from_port="out 2" to_port="result 1"/>
              <portSpacing port="source_input 1" spacing="0"/>
              <portSpacing port="source_input 2" spacing="0"/>
              <portSpacing port="source_input 3" spacing="0"/>
              <portSpacing port="sink_performance" spacing="0"/>
              <portSpacing port="sink_result 1" spacing="0"/>
              <portSpacing port="sink_result 2" spacing="0"/>
            </process>
          </operator>
          <connect from_op="Read Database" from_port="output" to_op="Store" to_port="input"/>
          <connect from_op="Store" from_port="through" to_op="Optimize Parameters (Grid)" to_port="input 1"/>
          <connect from_op="Read Database (2)" from_port="output" to_op="Store (2)" to_port="input"/>
          <connect from_op="Store (2)" from_port="through" to_op="Optimize Parameters (Grid)" to_port="input 2"/>
          <connect from_op="Optimize Parameters (Grid)" from_port="performance" to_port="result 1"/>
          <connect from_op="Optimize Parameters (Grid)" from_port="parameter" to_port="result 2"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>
          <portSpacing port="sink_result 3" spacing="0"/>
        </process>
      </operator>
    </process>

     

     

  • Marco_BoeckMarco_Boeck Administrator, Moderator, Employee-RapidMiner, Member, University Professor Posts: 1,996 RM Engineering

    Hi,

     

    don't worry about the Loop timeout parameter. It will be not show up anymore with the next release as it was just a crutch for some very obscure use cases.

     

    Regards,

    Marco

Sign In or Register to comment.