Using Loop

robin

I am unable to get loop to work with me, maybe you can tell me how I can work better with loop. In the process below I am reading from a SQL server to enrich data. I need to limit the number of lines selected in the extract to 1, this bit I am getting correct, also I am correctly selecting the second row of data when I manually loop the process. 

 

I have tried numerous loop operators but none of them play along. I need to loop the below process based on the number of entries that there are in a sql table. I need to select one line at a time, run the process and then loop it. Help!

 

robin

Thank you, sent me in the correct direction. Perviois experience with loops was clouding my understanding. 

 

Did not realise that I had to place the process inside the loop operator, once I figured that it was all good. 

 

Telcontar120

The screenshot isn't that helpful for understanding what you are actually trying to do.  You should paste the XML of your process instead.  Also, you mentioned you can get your loop to read the db table one line at a time, so can you be more descriptive about what else you want it to do and where it is failing?  The process based on the picture is quite long so it would take another community memeber a while to try to figure it all out for troubleshooting.

 

 

robin

The problem is more understanding which loop opperator to use and where to place it. There are a number of loop operators, however which one should I be using to extract a single entry from a MySQL database? When I run the process, as the data inside the database is enriched with information it is removed from the table that I cam calling from and placed on another table. The process I am running can only handle one entry at a time, which is why I need to take this approach.

 

Secondly where does one place the loop, when coding a loop it normally goes at the end of a process, however from the various tutorials I have watched as well as sample processes I have used it seems to be in different locations. What determines where it should be placed?

 

I will attempt to create a sample process and post here to aid the discussion. 

 

robin

<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="generate_data" compatibility="7.6.001" expanded="true" height="68" name="Generate Data" width="90" x="112" y="340">
    <parameter key="target_function" value="random"/>
    <parameter key="number_examples" value="100"/>
    <parameter key="number_of_attributes" value="5"/>
    <parameter key="attributes_lower_bound" value="-10.0"/>
    <parameter key="attributes_upper_bound" value="10.0"/>
    <parameter key="gaussian_standard_deviation" value="10.0"/>
    <parameter key="largest_radius" value="10.0"/>
    <parameter key="use_local_random_seed" value="false"/>
    <parameter key="local_random_seed" value="1992"/>
    <parameter key="datamanagement" value="double_array"/>
    <parameter key="data_management" value="auto"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="generate_id" compatibility="7.6.001" expanded="true" height="82" name="Generate ID (2)" width="90" x="246" y="340">
    <parameter key="create_nominal_ids" value="true"/>
    <parameter key="offset" value="0"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="set_role" compatibility="7.6.001" expanded="true" height="82" name="Set Role (4)" width="90" x="380" y="340">
    <parameter key="attribute_name" value="id"/>
    <parameter key="target_role" value="regular"/>
    <list key="set_additional_roles"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="web:enrich_data_by_webservice" compatibility="7.3.000" expanded="true" height="68" name="Enrich Data by Webservice" width="90" x="514" y="340">
    <parameter key="query_type" value="String Matching"/>
    <list key="string_machting_queries"/>
    <parameter key="attribute_type" value="Nominal"/>
    <list key="regular_expression_queries"/>
    <list key="regular_region_queries"/>
    <list key="xpath_queries"/>
    <list key="namespaces"/>
    <parameter key="ignore_CDATA" value="true"/>
    <parameter key="assume_html" value="true"/>
    <list key="index_queries"/>
    <list key="jsonpath_queries"/>
    <parameter key="request_method" value="GET"/>
    <parameter key="url" value="ww.sample.address.com"/>
    <parameter key="delay" value="0"/>
    <list key="request_properties"/>
    <parameter key="encoding" value="SYSTEM"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="text:data_to_documents" compatibility="7.5.000" expanded="true" height="68" name="Data to Documents (2)" width="90" x="648" y="340">
    <parameter key="select_attributes_and_weights" value="false"/>
    <list key="specify_weights"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="text:json_to_data" compatibility="7.5.000" expanded="true" height="82" name="JSON To Data" width="90" x="782" y="340">
    <parameter key="ignore_arrays" value="false"/>
    <parameter key="limit_attributes" value="false"/>
    <parameter key="skip_invalid_documents" value="false"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="text:data_to_documents" compatibility="7.5.000" expanded="true" height="68" name="Data to Documents" width="90" x="916" y="340">
    <parameter key="select_attributes_and_weights" value="true"/>
    <list key="specify_weights">
      <parameter key="id" value="1.0"/>
    </list>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="text:combine_documents" compatibility="7.5.000" expanded="true" height="82" name="Combine Documents" width="90" x="1050" y="340"/>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="text:write_document" compatibility="7.5.000" expanded="true" height="82" name="Write Document" width="90" x="1184" y="340">
    <parameter key="file" value="/Users/Robin/Dropbox/keceptiv_key/write_document_folder/wrtie_documet.*"/>
    <parameter key="overwrite" value="true"/>
    <parameter key="encoding" value="SYSTEM"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="text:documents_to_data" compatibility="7.5.000" expanded="true" height="82" name="Documents to Data" width="90" x="1318" y="340">
    <parameter key="text_attribute" value="text"/>
    <parameter key="label_attribute" value="text"/>
    <parameter key="add_meta_information" value="true"/>
    <parameter key="datamanagement" value="double_sparse_array"/>
    <parameter key="data_management" value="auto"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="select_attributes" compatibility="7.6.001" expanded="true" height="82" name="Select Attributes" width="90" x="1452" y="340">
    <parameter key="attribute_filter_type" value="all"/>
    <parameter key="attribute" value=""/>
    <parameter key="attributes" value=""/>
    <parameter key="use_except_expression" value="false"/>
    <parameter key="value_type" value="attribute_value"/>
    <parameter key="use_value_type_exception" value="false"/>
    <parameter key="except_value_type" value="time"/>
    <parameter key="block_type" value="attribute_block"/>
    <parameter key="use_block_type_exception" value="false"/>
    <parameter key="except_block_type" value="value_matrix_row_start"/>
    <parameter key="invert_selection" value="false"/>
    <parameter key="include_special_attributes" value="false"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <operator activated="true" class="jdbc_connectors:write_database" compatibility="7.6.001" expanded="true" height="68" name="Write Database" width="90" x="1586" y="340">
    <parameter key="define_connection" value="predefined"/>
    <parameter key="connection" value="Personality Profile"/>
    <parameter key="database_system" value="MySQL"/>
    <parameter key="use_default_schema" value="true"/>
    <parameter key="table_name" value="personality_header"/>
    <parameter key="overwrite_mode" value="append"/>
    <parameter key="set_default_varchar_length" value="false"/>
    <parameter key="default_varchar_length" value="128"/>
    <parameter key="add_generated_primary_keys" value="false"/>
    <parameter key="db_key_attribute_name" value="generated_primary_key"/>
    <parameter key="batch_size" value="1"/>
  </operator>
</process>

FBT

Hi Robin,

 

your xml code always seems to be corrupted, as it includes several <process></process> tags, which would indicate several processes. The easiest way to correctly copy the xml code is to go to the xml tab, click the right mouse button and chose "Select All" in the drop-down. Then just copy it and paste it into the community forum within code tags.

 

Having said all that, I think what you could try to do, in order to solve your processing problem, is to work with the "Loop Examples" operator in combination with the "Extract Macro". Instead of manually entering the row number in the "Extract Macro" operator, you could just enter a macro with %{MACRO_NAME} for the example index. This would be the same name that you gave the iteration macro in the "Loop Examples" operator. The "Extract Macro" operator and anything that you would want to do with the respective selected database output row needs to be nested within the "Loop Examples" operator.

 

Without having access to your correct XML code, it's difficult to be more specific. But the explanation above should give you an idea of how you can go about it.