"How to generate attributes using attributes from two datasets (sources)"

bgracian

Hello! I am new with rapidminer, and i am taking the first steps with ETL processes. Now, i have found a problem that i don't know how to solve it. In particular, I am looking for a way to generating new attributes applying function expressions where I need to use attributes from two database tables (dataset1, dataset2), where one of the tables (dataset2) has attributes that i need to use to calculate new attributes in the other table (dataset1).

Tables have different number of rows, and need to calculate a particular value of att1 in dataset1 to apply as a constant to calculate new attribute.

dataset1

                    att1 att2 att3
        row1    1.1  1.2  2.3
        row2    1      2.2 3.3
        row3    1      2.4  5.3

dataset2

                    att1 att2 att3
        row1    1.1  2    3
        row2    1.2  2    3
        row3    1.3  2    3
        row4    1.4  2    3
        row5    1.6  2    3
        row6    1.7  2    3


Using Generate attributes

new attribute (dataset1)

dataset1.att4 = dataset1.att1 * dataset2.att1


Is it possible to use attributes from other datasets to calculate new attributes in other dataset?

Best regards,

alberto

wessel

Hello,

Based on your example, what should be the result?
Should I simply discard
      row4    1.4  2    3
        row5    1.6  2    3
        row6    1.7  2    3

Best regards,

Wessel

Edit:

See process below, with join and generate attributes.

wessel

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.3.013">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.3.013" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="generate_data" compatibility="5.3.013" expanded="true" height="60" name="Generate Data" width="90" x="228" y="101">
        <parameter key="number_examples" value="3"/>
      </operator>
      <operator activated="true" class="generate_id" compatibility="5.3.013" expanded="true" height="76" name="Generate ID" width="90" x="453" y="122"/>
      <operator activated="true" class="generate_data" compatibility="5.3.013" expanded="true" height="60" name="Generate Data (2)" width="90" x="223" y="203">
        <parameter key="number_examples" value="5"/>
      </operator>
      <operator activated="true" class="generate_id" compatibility="5.3.013" expanded="true" height="76" name="Generate ID (2)" width="90" x="415" y="272"/>
      <operator activated="true" class="rename_by_replacing" compatibility="5.3.013" expanded="true" height="76" name="Rename by Replacing" width="90" x="547" y="420">
        <parameter key="replace_what" value="$"/>
        <parameter key="replace_by" value="_2"/>
      </operator>
      <operator activated="true" class="join" compatibility="5.3.013" expanded="true" height="76" name="Join" width="90" x="573" y="216">
        <list key="key_attributes"/>
      </operator>
      <operator activated="true" class="generate_attributes" compatibility="5.3.013" expanded="true" height="76" name="Generate Attributes" width="90" x="735" y="162">
        <list key="function_descriptions">
          <parameter key="att1*att1_2" value="att1*att1_2"/>
        </list>
      </operator>
      <connect from_op="Generate Data" from_port="output" to_op="Generate ID" to_port="example set input"/>
      <connect from_op="Generate ID" from_port="example set output" to_op="Join" to_port="left"/>
      <connect from_op="Generate Data (2)" from_port="output" to_op="Generate ID (2)" to_port="example set input"/>
      <connect from_op="Generate ID (2)" from_port="example set output" to_op="Rename by Replacing" to_port="example set input"/>
      <connect from_op="Rename by Replacing" from_port="example set output" to_op="Join" to_port="right"/>
      <connect from_op="Join" from_port="join" to_op="Generate Attributes" to_port="example set input"/>
      <connect from_op="Generate Attributes" from_port="example set output" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>