Does Windowing operator accept a macro for "window size"?

vgpowered · March 22

Hello,
Is it possible to use a macro in the Windowing for "window size" parameter?

I have read the thread "Forecasting process - doing it right?" and it seems the operator can do, but I have the error "Required parameter missing"

What happens?

ceaperez · March 23

Hi @vgpowered,

here is the XML:

<?xml version="1.0" encoding="UTF-8"?><process version="10.3.001">

  <context>

    <input/>

    <output/>

    <macros/>

  </context>

  <operator activated="true" class="process" compatibility="10.3.001" expanded="true" name="Process" origin="GENERATED_TUTORIAL">

    <parameter key="logverbosity" value="init"/>

    <parameter key="random_seed" value="2001"/>

    <parameter key="send_mail" value="never"/>

    <parameter key="notification_email" value=""/>

    <parameter key="process_duration_for_mail" value="30"/>

    <parameter key="encoding" value="SYSTEM"/>

    <process expanded="true">

      <operator activated="true" class="retrieve" compatibility="10.3.001" expanded="true" height="68" name="Retrieve Lake Huron" origin="GENERATED_TUTORIAL" width="90" x="45" y="34">

        <parameter key="repository_entry" value="//Samples/Time Series/data sets/Lake Huron"/>

      </operator>

      <operator activated="true" class="concurrency:optimize_parameters_grid" compatibility="10.3.001" expanded="true" height="124" name="Optimize Parameters (Grid)" width="90" x="246" y="34">

        <list key="parameters">

          <parameter key="Set Macro (Real).value" value="[2;10;5;linear]"/>

        </list>

        <parameter key="error_handling" value="fail on error"/>

        <parameter key="log_performance" value="true"/>

        <parameter key="log_all_criteria" value="false"/>

        <parameter key="synchronize" value="false"/>

        <parameter key="enable_parallel_execution" value="true"/>

        <process expanded="true">

          <operator activated="true" class="operator_toolbox:set_macro_real" compatibility="2.17.000" expanded="true" height="82" name="Set Macro (Real)" width="90" x="45" y="187">

            <parameter key="macro" value="uno"/>

            <parameter key="value" value="3.5"/>

          </operator>

          <operator activated="true" class="generate_macro" compatibility="10.3.001" expanded="true" height="82" name="Generate Macro" width="90" x="179" y="238">

            <list key="function_descriptions">

              <parameter key="dos" value="floor(eval(%{uno}))"/>

            </list>

          </operator>

          <operator activated="true" class="time_series:windowing" compatibility="10.3.001" expanded="true" height="103" name="Windowing (Series)" origin="GENERATED_TUTORIAL" width="90" x="179" y="34">

            <parameter key="attribute_filter_type" value="single"/>

            <parameter key="attribute" value="Lake surface level / feet"/>

            <parameter key="attributes" value=""/>

            <parameter key="use_except_expression" value="false"/>

            <parameter key="value_type" value="numeric"/>

            <parameter key="use_value_type_exception" value="false"/>

            <parameter key="except_value_type" value="real"/>

            <parameter key="block_type" value="value_series"/>

            <parameter key="use_block_type_exception" value="false"/>

            <parameter key="except_block_type" value="value_series_end"/>

            <parameter key="invert_selection" value="false"/>

            <parameter key="include_special_attributes" value="false"/>

            <parameter key="has_indices" value="true"/>

            <parameter key="indices_attribute" value="Date"/>

            <parameter key="sort_time_series" value="false"/>

            <parameter key="expert_settings" value="false"/>

            <parameter key="unit" value="example based"/>

            <parameter key="windows_defined" value="from start"/>

            <parameter key="custom_start_point" value="5"/>

            <parameter key="custom_end_point" value="100"/>

            <parameter key="window_size" value="%{dos}"/>

            <parameter key="custom_start_time" value="2000-01-01 00:00:00"/>

            <parameter key="custom_end_time" value="2030-01-01 00:00:00"/>

            <parameter key="date_format" value="yyyy-MM-dd HH:mm:ss"/>

            <parameter key="window_size_time" value="1.Hours"/>

            <parameter key="windows_stop_definition" value="from next window start"/>

            <parameter key="window_start_attribute" value=""/>

            <parameter key="window_stop_attribute" value=""/>

            <parameter key="no_overlapping_windows" value="false"/>

            <parameter key="step_size" value="1"/>

            <parameter key="step_size_time" value="1.Minutes"/>

            <parameter key="create_horizon_(labels)" value="true"/>

            <parameter key="horizon_attribute" value="Lake surface level / feet"/>

            <parameter key="horizon_size" value="1"/>

            <parameter key="horizon_offset" value="0"/>

            <parameter key="horizon_size_time" value="1.Hours"/>

            <parameter key="horizon_offset_time" value="0.Minutes"/>

            <parameter key="horizon_start_attribute" value=""/>

            <parameter key="horizon_stop_attribute" value=""/>

            <parameter key="empty_window_handling" value="add empty exampleset"/>

          </operator>

          <operator activated="true" class="h2o:deep_learning" compatibility="10.3.001" expanded="true" height="103" name="Deep Learning" width="90" x="313" y="34">

            <parameter key="activation" value="Rectifier"/>

            <enumeration key="hidden_layer_sizes">

              <parameter key="hidden_layer_sizes" value="50"/>

              <parameter key="hidden_layer_sizes" value="50"/>

            </enumeration>

            <enumeration key="hidden_dropout_ratios"/>

            <parameter key="reproducible_(uses_1_thread)" value="false"/>

            <parameter key="use_local_random_seed" value="false"/>

            <parameter key="local_random_seed" value="1992"/>

            <parameter key="epochs" value="10.0"/>

            <parameter key="compute_variable_importances" value="false"/>

            <parameter key="train_samples_per_iteration" value="-2"/>

            <parameter key="adaptive_rate" value="true"/>

            <parameter key="epsilon" value="1.0E-8"/>

            <parameter key="rho" value="0.99"/>

            <parameter key="learning_rate" value="0.005"/>

            <parameter key="learning_rate_annealing" value="1.0E-6"/>

            <parameter key="learning_rate_decay" value="1.0"/>

            <parameter key="momentum_start" value="0.0"/>

            <parameter key="momentum_ramp" value="1000000.0"/>

            <parameter key="momentum_stable" value="0.0"/>

            <parameter key="nesterov_accelerated_gradient" value="true"/>

            <parameter key="standardize" value="true"/>

            <parameter key="L1" value="1.0E-5"/>

            <parameter key="L2" value="0.0"/>

            <parameter key="max_w2" value="10.0"/>

            <parameter key="loss_function" value="Automatic"/>

            <parameter key="distribution_function" value="AUTO"/>

            <parameter key="early_stopping" value="false"/>

            <parameter key="stopping_rounds" value="1"/>

            <parameter key="stopping_metric" value="AUTO"/>

            <parameter key="stopping_tolerance" value="0.001"/>

            <parameter key="missing_values_handling" value="MeanImputation"/>

            <parameter key="max_runtime_seconds" value="0"/>

            <list key="expert_parameters"/>

            <list key="expert_parameters_"/>

          </operator>

          <operator activated="true" class="apply_model" compatibility="10.3.001" expanded="true" height="82" name="Apply Model" width="90" x="447" y="34">

            <list key="application_parameters"/>

          </operator>

          <operator activated="true" class="performance_regression" compatibility="10.3.001" expanded="true" height="82" name="Performance (2)" width="90" x="581" y="34">

            <parameter key="main_criterion" value="first"/>

            <parameter key="root_mean_squared_error" value="true"/>

            <parameter key="absolute_error" value="false"/>

            <parameter key="relative_error" value="false"/>

            <parameter key="relative_error_lenient" value="false"/>

            <parameter key="relative_error_strict" value="false"/>

            <parameter key="normalized_absolute_error" value="false"/>

            <parameter key="root_relative_squared_error" value="false"/>

            <parameter key="squared_error" value="false"/>

            <parameter key="correlation" value="false"/>

            <parameter key="squared_correlation" value="false"/>

            <parameter key="prediction_average" value="false"/>

            <parameter key="spearman_rho" value="false"/>

            <parameter key="kendall_tau" value="false"/>

            <parameter key="skip_undefined_labels" value="true"/>

            <parameter key="use_example_weights" value="true"/>

          </operator>

          <connect from_port="input 1" to_op="Windowing (Series)" to_port="example set"/>

          <connect from_op="Set Macro (Real)" from_port="through 1" to_op="Generate Macro" to_port="through 1"/>

          <connect from_op="Windowing (Series)" from_port="windowed example set" to_op="Deep Learning" to_port="training set"/>

          <connect from_op="Deep Learning" from_port="model" to_op="Apply Model" to_port="model"/>

          <connect from_op="Deep Learning" from_port="exampleSet" to_op="Apply Model" to_port="unlabelled data"/>

          <connect from_op="Apply Model" from_port="labelled data" to_op="Performance (2)" to_port="labelled data"/>

          <connect from_op="Performance (2)" from_port="performance" to_port="performance"/>

          <portSpacing port="source_input 1" spacing="0"/>

          <portSpacing port="source_input 2" spacing="0"/>

          <portSpacing port="sink_performance" spacing="0"/>

          <portSpacing port="sink_model" spacing="0"/>

          <portSpacing port="sink_output 1" spacing="0"/>

        </process>

      </operator>

      <connect from_op="Retrieve Lake Huron" from_port="output" to_op="Optimize Parameters (Grid)" to_port="input 1"/>

      <connect from_op="Optimize Parameters (Grid)" from_port="performance" to_port="result 1"/>

      <portSpacing port="source_input 1" spacing="0"/>

      <portSpacing port="sink_result 1" spacing="0"/>

      <portSpacing port="sink_result 2" spacing="0"/>

    </process>

  </operator>

</process>

ceaperez · March 23

Hi @vgpowered,
yes, is possible, please find attached a simple sample using a modification of the windows size operator tutorial.

best,

Cesar

vgpowered · March 23

Hi Cesar,
Sorry, Where is the attachment?
By the way I'm using Windowing operator inside a Optimize parameter (Grid)...

ceaperez · March 23

Hi @vgpowered,
The attached file is in the bottom of my message.

Can you share your process?

Best,

Cesar

vgpowered · March 23

Hi Cesar,
I can only see an icon (it is a page with the upper right corner folded). When I move the mouse over this icon the "Macrowinsize.rmp" message appear, but nothing else. I think you have attached an image, not a file.

My process is very complex, but only a draft (Accepts my apology)

Víctor

vgpowered · March 23

Hi Cesar
Maybe the Rapidminer Web is not working. I'm testing my own message and it shows somthing similar like yours (about attached files)...

ceaperez · March 23

Hi @vgpowered,
I can download your process. Anyway this is the xml

<?xml version="1.0" encoding="UTF-8"?><process version="10.3.001">

  <context>

    <input/>

    <output/>

    <macros/>

  </context>

  <operator activated="true" class="process" compatibility="10.3.001" expanded="true" name="Process" origin="GENERATED_TUTORIAL">

    <parameter key="logverbosity" value="init"/>

    <parameter key="random_seed" value="2001"/>

    <parameter key="send_mail" value="never"/>

    <parameter key="notification_email" value=""/>

    <parameter key="process_duration_for_mail" value="30"/>

    <parameter key="encoding" value="SYSTEM"/>

    <process expanded="true">

      <operator activated="true" class="retrieve" compatibility="10.3.001" expanded="true" height="68" name="Retrieve Lake Huron" origin="GENERATED_TUTORIAL" width="90" x="112" y="34">

        <parameter key="repository_entry" value="//Samples/Time Series/data sets/Lake Huron"/>

      </operator>

      <operator activated="true" class="set_macro" compatibility="10.3.001" expanded="true" height="82" name="Set Macro" width="90" x="112" y="238">

        <parameter key="macro" value="Winsiz"/>

        <parameter key="value" value="4"/>

      </operator>

      <operator activated="true" class="time_series:process_windows" compatibility="10.3.001" expanded="true" height="103" name="Process Windows" origin="GENERATED_TUTORIAL" width="90" x="313" y="34">

        <parameter key="attribute_filter_type" value="all"/>

        <parameter key="attribute" value=""/>

        <parameter key="attributes" value=""/>

        <parameter key="use_except_expression" value="false"/>

        <parameter key="value_type" value="numeric"/>

        <parameter key="use_value_type_exception" value="false"/>

        <parameter key="except_value_type" value="real"/>

        <parameter key="block_type" value="value_series"/>

        <parameter key="use_block_type_exception" value="false"/>

        <parameter key="except_block_type" value="value_series_end"/>

        <parameter key="invert_selection" value="false"/>

        <parameter key="include_special_attributes" value="false"/>

        <parameter key="has_indices" value="true"/>

        <parameter key="indices_attribute" value="Date"/>

        <parameter key="sort_time_series" value="false"/>

        <parameter key="expert_settings" value="false"/>

        <parameter key="unit" value="example based"/>

        <parameter key="windows_defined" value="from start"/>

        <parameter key="custom_start_point" value="5"/>

        <parameter key="custom_end_point" value="100"/>

        <parameter key="window_size" value="%{Winsiz}"/>

        <parameter key="custom_start_time" value="2000-01-01 00:00:00"/>

        <parameter key="custom_end_time" value="2030-01-01 00:00:00"/>

        <parameter key="date_format" value="yyyy-MM-dd HH:mm:ss"/>

        <parameter key="window_size_time" value="1.Hours"/>

        <parameter key="windows_stop_definition" value="from next window start"/>

        <parameter key="window_start_attribute" value=""/>

        <parameter key="window_stop_attribute" value=""/>

        <parameter key="no_overlapping_windows" value="false"/>

        <parameter key="step_size" value="1"/>

        <parameter key="step_size_time" value="1.Minutes"/>

        <parameter key="create_horizon_(labels)" value="true"/>

        <parameter key="horizon_attribute" value="Lake surface level / feet"/>

        <parameter key="horizon_size" value="1"/>

        <parameter key="horizon_offset" value="0"/>

        <parameter key="horizon_size_time" value="1.Hours"/>

        <parameter key="horizon_offset_time" value="0.Minutes"/>

        <parameter key="horizon_start_attribute" value=""/>

        <parameter key="horizon_stop_attribute" value=""/>

        <parameter key="empty_window_handling" value="add empty exampleset"/>

        <parameter key="add_last_index_in_window_attribute" value="true"/>

        <parameter key="enable_parallel_execution" value="true"/>

        <process expanded="true">

          <operator activated="true" class="time_series:extract_std_descriptive_features" compatibility="10.3.001" expanded="true" height="82" name="Extract Aggregates" origin="GENERATED_TUTORIAL" width="90" x="313" y="34">

            <parameter key="attribute_filter_type" value="all"/>

            <parameter key="attribute" value=""/>

            <parameter key="attributes" value=""/>

            <parameter key="use_except_expression" value="false"/>

            <parameter key="value_type" value="numeric"/>

            <parameter key="use_value_type_exception" value="false"/>

            <parameter key="except_value_type" value="real"/>

            <parameter key="block_type" value="value_series"/>

            <parameter key="use_block_type_exception" value="false"/>

            <parameter key="except_block_type" value="value_series_end"/>

            <parameter key="invert_selection" value="false"/>

            <parameter key="include_special_attributes" value="false"/>

            <parameter key="sum" value="true"/>

            <parameter key="mean" value="true"/>

            <parameter key="geometric_mean" value="true"/>

            <parameter key="first_quartile" value="true"/>

            <parameter key="median" value="true"/>

            <parameter key="third_quartile" value="true"/>

            <parameter key="min" value="true"/>

            <parameter key="max" value="true"/>

            <parameter key="std_deviation" value="true"/>

            <parameter key="kurtosis" value="true"/>

            <parameter key="skewness" value="true"/>

            <parameter key="add_time_series_name" value="false"/>

            <parameter key="ignore_invalid_values" value="false"/>

          </operator>

          <connect from_port="windowed example set" to_op="Extract Aggregates" to_port="example set"/>

          <connect from_op="Extract Aggregates" from_port="features" to_port="output 1"/>

          <portSpacing port="source_windowed example set" spacing="0"/>

          <portSpacing port="source_input 1" spacing="0"/>

          <portSpacing port="sink_output 1" spacing="0"/>

          <portSpacing port="sink_output 2" spacing="0"/>

        </process>

      </operator>

      <operator activated="true" class="append" compatibility="10.3.001" expanded="true" height="82" name="Append" origin="GENERATED_TUTORIAL" width="90" x="514" y="34">

        <parameter key="datamanagement" value="double_array"/>

        <parameter key="data_management" value="auto"/>

        <parameter key="merge_type" value="all"/>

      </operator>

      <connect from_op="Retrieve Lake Huron" from_port="output" to_op="Process Windows" to_port="example set"/>

      <connect from_op="Set Macro" from_port="through 1" to_op="Process Windows" to_port="custom windows"/>

      <connect from_op="Process Windows" from_port="output 1" to_op="Append" to_port="example set 1"/>

      <connect from_op="Append" from_port="merged set" to_port="result 1"/>

      <portSpacing port="source_input 1" spacing="0"/>

      <portSpacing port="sink_result 1" spacing="0"/>

      <portSpacing port="sink_result 2" spacing="0"/>

    </process>

  </operator>

</process>

vgpowered · March 23

Thank Cesar,
I think the problem is the Optimization operator.
I have build a little example that demostrates there is a limitation.
Do not lose your time with the previous complex procces, simply test the next example.

Víctor

ceaperez · March 23

Hola Victor,
Just put the macro outside of the optimization operator.

Saludos,

Cesar

vgpowered · March 23

Hi Cesar,
But in this case you cannot optimize its value. You could say, you can optimize it directly, but in my real process there are several windowing operators (inside the Optimize process) that share this same value but not others. This is the issue.

Víctor

ceaperez · March 23

Hi @vgpowered,
I have a possible solution:

Best,

César

vgpowered · March 23

Hi Cesar,
The attachment does not work.

Regards
Víctor

vgpowered · March 23

Hi Cesar,
Í cannot download the "windowing with optimiza.rmp" . Icon doesn't work,

Víctor

vgpowered · March 25

Hello César,
Yes, this is the solution. I did'n have installed the Operator Toolbox set of operators. In all cases this is a real case of trick solutión. I think Ripidminer has a problem if we have to use this approach.

It should be more efficient to create a new versión of the optimize operator.

Thank you very much.

Víctor

ceaperez · March 26

Hi @vgpowered,
You are right, this is a tricky solution, but these kinds of issues help the RM team improve the product. I'm not part of the RM team so we need help from someone there to review this issue or find other way, maybe @rjones13 can follow up this issue.

best

Cesar

Howdy, Stranger!

Quick Links

Categories

Altair RapidMiner Community

GET HELP. LEARN BEST PRACTICES. NETWORK WITH YOUR PEERS.

Does Windowing operator accept a macro for "window size"?

Best Answer

Answers