Loop Data

TASHRM

I am trying to create multiple data sets each current data set as to split this by the number of months defined by a generate attribute attribute i have labeled "term" which works out the length of a contract and i want to expand this dataset having values that will be the same per a contract split across the entire contract period showing each month separately ie 2020/03/01-2021/02/28.

I have tried to create a loop but im not sure i have set it up correctly.

Please could anyone assist me.

 <operator activated="true" class="generate_attributes" compatibility="9.10.000" expanded="true" height="82" name="Generate Attributes (13)" width="90" x="4534" y="544">

        <list key="function_descriptions">

          <parameter key="TERM" value="if(date_get(BEGIN_COV_DT,DATE_UNIT_YEAR)==(date_get(date_parse_custom(REPORTING_DT,&quot;yyyy/MM/dd&quot;),DATE_UNIT_YEAR) - 1),date_get(BEGIN_COV_DT,DATE_UNIT_MONTH)-(date_get(date_parse_custom(REPORTING_DT,&quot;yyyy/MM/dd&quot;),DATE_UNIT_MONTH)+1),if(date_get(BEGIN_COV_DT,DATE_UNIT_YEAR)==(date_get(date_parse_custom(REPORTING_DT,&quot;yyyy/MM/dd&quot;),DATE_UNIT_YEAR)),date_get(BEGIN_COV_DT,DATE_UNIT_MONTH)+(11-date_get(date_parse_custom(REPORTING_DT,&quot;yyyy/MM/dd&quot;),DATE_UNIT_MONTH)),&quot;&quot;))"/>

        </list>

        <parameter key="keep_all" value="true"/>

      </operator>

      <operator activated="false" class="loop_examples" compatibility="9.10.000" expanded="true" height="82" name="Loop Examples" width="90" x="4400" y="442">

        <parameter key="iteration_macro" value="example"/>

        <process expanded="true">

          <operator activated="false" class="concurrency:loop_attributes" compatibility="9.10.000" expanded="true" height="82" name="Loop Attributes (2)" width="90" x="514" y="187">

            <parameter key="attribute_filter_type" value="all"/>

            <parameter key="attribute" value=""/>

            <parameter key="attributes" value=""/>

            <parameter key="use_except_expression" value="false"/>

            <parameter key="value_type" value="attribute_value"/>

            <parameter key="use_value_type_exception" value="false"/>

            <parameter key="except_value_type" value="time"/>

            <parameter key="block_type" value="attribute_block"/>

            <parameter key="use_block_type_exception" value="false"/>

            <parameter key="except_block_type" value="value_matrix_row_start"/>

            <parameter key="invert_selection" value="false"/>

            <parameter key="include_special_attributes" value="false"/>

            <parameter key="attribute_name_macro" value="loop_attribute"/>

            <parameter key="reuse_results" value="false"/>

            <parameter key="enable_parallel_execution" value="true"/>

            <process expanded="true">

              <operator activated="true" class="generate_attributes" compatibility="9.10.000" expanded="true" height="82" name="Generate Attributes (7)" width="90" x="179" y="34">

                <list key="function_descriptions">

                  <parameter key="End_Cov_Date" value="date_add(date_parse_custom(BEGIN_COV_DT,+1,DATE_UNIT_MONTH))"/>

                </list>

                <parameter key="keep_all" value="true"/>

              </operator>

              <connect from_port="input 1" to_op="Generate Attributes (7)" to_port="example set input"/>

              <connect from_op="Generate Attributes (7)" from_port="example set output" to_port="output 1"/>

              <portSpacing port="source_input 1" spacing="0"/>

              <portSpacing port="source_input 2" spacing="0"/>

              <portSpacing port="sink_output 1" spacing="0"/>

              <portSpacing port="sink_output 2" spacing="0"/>

            </process>

          </operator>

          <operator activated="true" class="filter_examples" compatibility="9.10.000" expanded="true" height="103" name="Filter Examples (4)" width="90" x="179" y="34">

            <parameter key="parameter_expression" value=""/>

            <parameter key="condition_class" value="custom_filters"/>

            <parameter key="invert_filter" value="false"/>

            <list key="filters_list"/>

            <parameter key="filters_logic_and" value="true"/>

            <parameter key="filters_check_metadata" value="true"/>

          </operator>

          <operator activated="true" class="concurrency:loop_values" compatibility="9.10.000" expanded="true" height="82" name="Loop Values" width="90" x="313" y="34">

            <parameter key="attribute" value="TERM"/>

            <parameter key="iteration_macro" value="loop_value"/>

            <parameter key="reuse_results" value="false"/>

            <parameter key="enable_parallel_execution" value="true"/>

            <process expanded="true">

              <portSpacing port="source_input 1" spacing="0"/>

              <portSpacing port="source_input 2" spacing="0"/>

              <portSpacing port="sink_output 1" spacing="0"/>

              <portSpacing port="sink_output 2" spacing="0"/>

            </process>

          </operator>

          <connect from_port="example set" to_op="Filter Examples (4)" to_port="example set input"/>

          <connect from_op="Filter Examples (4)" from_port="example set output" to_op="Loop Values" to_port="input 1"/>

          <connect from_op="Loop Values" from_port="output 1" to_port="example set"/>

          <portSpacing port="source_example set" spacing="0"/>

          <portSpacing port="sink_example set" spacing="0"/>

          <portSpacing port="sink_output 1" spacing="0"/>

        </process>

      </operator>

      <operator activated="true" class="order_attributes" compatibility="9.10.000" expanded="true" height="82" name="Reorder Attributes (18)" width="90" x="4668" y="544">

        <parameter key="sort_mode" value="user specified"/>

        <parameter key="attribute_ordering" value="REPORTING_DT|ENTITY_ID|INSURANCE_CONTRACT_ID|COVERAGE_CD|CEDED_FLG|CASHFLOW_TYPE_CD|CURRENCY_CD|INCURRED_CLAIM_DT|CASHFLOW_DT|ACQ_COSTS_AMT|ADM_COSTS_AMT|CLAIMS_AMT|ENTITY_FEE_AMT|GUAR_FV_CHNG_AMT|GUAR_FV_AMT|GUAR_PMT_AMT|PREM_AMT|PROVENUES_AMT|RE_COMM_AMT|RE_DEF_AMT|RSK_ADJ_AMT|SETTL_COSTS_AMT|UNDERL_FV_AMT|UNDERL_FV_CHNG_AMT|INVESTMENT_AMT"/>

        <parameter key="use_regular_expressions" value="false"/>

        <parameter key="handle_unmatched" value="append"/>

        <parameter key="sort_direction" value="ascending"/>

      </operator>

      <operator activated="true" breakpoints="after" class="concurrency:loop_attributes" compatibility="9.10.000" expanded="true" height="82" name="Loop Attributes" width="90" x="4802" y="544">

        <parameter key="attribute_filter_type" value="single"/>

        <parameter key="attribute" value="TERM"/>

        <parameter key="attributes" value=""/>

        <parameter key="use_except_expression" value="false"/>

        <parameter key="value_type" value="attribute_value"/>

        <parameter key="use_value_type_exception" value="false"/>

        <parameter key="except_value_type" value="time"/>

        <parameter key="block_type" value="attribute_block"/>

        <parameter key="use_block_type_exception" value="false"/>

        <parameter key="except_block_type" value="value_matrix_row_start"/>

        <parameter key="invert_selection" value="false"/>

        <parameter key="include_special_attributes" value="false"/>

        <parameter key="attribute_name_macro" value="loop_attribute"/>

        <parameter key="reuse_results" value="true"/>

        <parameter key="enable_parallel_execution" value="true"/>

        <process expanded="true">

          <operator activated="true" class="filter_examples" compatibility="9.10.000" expanded="true" height="103" name="Filter Examples (5)" width="90" x="45" y="34">

            <parameter key="parameter_expression" value=""/>

            <parameter key="condition_class" value="custom_filters"/>

            <parameter key="invert_filter" value="false"/>

            <list key="filters_list">

              <parameter key="filters_entry_key" value="INSURANCE_CONTRACT_ID.equals.%{loop_attributes}"/>

            </list>

            <parameter key="filters_logic_and" value="true"/>

            <parameter key="filters_check_metadata" value="true"/>

          </operator>

          <operator activated="true" class="concurrency:loop_values" compatibility="9.10.000" expanded="true" height="82" name="Loop Values (4)" width="90" x="246" y="34">

            <parameter key="attribute" value="END_COV_DT"/>

            <parameter key="iteration_macro" value="loop_value"/>

            <parameter key="reuse_results" value="true"/>

            <parameter key="enable_parallel_execution" value="false"/>

            <process expanded="true">

              <operator activated="true" class="generate_attributes" compatibility="9.10.000" expanded="true" height="82" name="Generate Attributes (8)" width="90" x="246" y="34">

                <list key="function_descriptions">

                  <parameter key="End_Cov_Date" value="date_add(BEGIN_COV_DT,+1,DATE_UNIT_MONTH)"/>

                </list>

                <parameter key="keep_all" value="true"/>

              </operator>

              <connect from_port="input 1" to_op="Generate Attributes (8)" to_port="example set input"/>

              <connect from_op="Generate Attributes (8)" from_port="example set output" to_port="output 1"/>

              <portSpacing port="source_input 1" spacing="0"/>

              <portSpacing port="source_input 2" spacing="0"/>

              <portSpacing port="sink_output 1" spacing="0"/>

              <portSpacing port="sink_output 2" spacing="0"/>

            </process>

          </operator>

          <connect from_port="input 1" to_op="Filter Examples (5)" to_port="example set input"/>

          <connect from_op="Filter Examples (5)" from_port="example set output" to_op="Loop Values (4)" to_port="input 1"/>

          <connect from_op="Loop Values (4)" from_port="output 1" to_port="output 1"/>

          <portSpacing port="source_input 1" spacing="0"/>

          <portSpacing port="source_input 2" spacing="0"/>

          <portSpacing port="sink_output 1" spacing="0"/>

          <portSpacing port="sink_output 2" spacing="0"/>

        </process>

      </operator>

kayman

Can you export your process (as rmp)  and attach it instead of copy pasting? It seems some of the code is missing so no way to reproduce your workflow now.

TASHRM

Good Day please see below the rmp process,Thank you for your response 

kayman

Not too sure what the outcome should be, but I believe you try to get the amount of months between 2 dates, being your BEGIN_COVER_DT (lets call this BCD) and REPORTING_DT (in short RD). If so there seems to be an issue with you calculation when the BCD years are different.

Assume 2 dates, BCD is 2020-11-20 and RD is 2021/01/09 this part of the function is triggered : 

date_get(date_parse_custom(BEGIN_COV_DT,"yyyy-MM-dd"),DATE_UNIT_YEAR)==(date_get(date_parse_custom(REPORTING_DT,"yyyy/MM/dd"),DATE_UNIT_YEAR) - 1)

and the provided result would be 

date_get(date_parse_custom(BEGIN_COV_DT,"yyyy-MM-dd"),DATE_UNIT_MONTH)-(date_get(date_parse_custom(REPORTING_DT,"yyyy/MM/dd"),DATE_UNIT_MONTH)+1)

or in this example 11 - (1+1) = 11 while it should be 2 (or 3 depending on how you deal with started months)

So wouldn't it be better to work with datediff instead? 

You convert first both BCD and RD to proper dates using the nominal to date operator, and then use the date_diff operator. This gives you the difference in milliseconds so you can then say that one month is (in average) around 2629800000 ms so dividing the date_diff with the amount of ms in a month gives you the difference in months. 

I've attached an example, if I understood wrong feel free to correct.

TASHRM

Good Day

Thank you for your assistance, i have have managed to correct the term and it was easier to use the date_diff and convert the milliseconds to months, so for the loop in the process i want the loop to create a number of iterations based on the term so ie the term is 3 so for that CONTRACT_iD needs to be replicated 3 times but its been creating more iterations than the term specified as well as the loop needs to produce one file and its currently creating individual files per a CONTRACT_ID and i have tried to append the results but that has not worked, im not too sure why this is happening.

Please could you assist?

Kind Regards

kayman

I suggest to create a new question for this new topic as your original is (somewhat) solved. This increases the possibility someone else but me looks at it to support you. This is just a general tip.

Next, also provide some more data or simplified examples showing what you want to do and what fails. This could be dummy partial processes where you add (partial) data exports so people can see, recreate and try to support with solutions. It is just very hard to try to do this based on just a little explanation and no domain knowledge.

TASHRM

Thank you for your feedback on this issue I will create a new Question for my current issue.