The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

decision tree does not fit the roc

cam8cam8 Member Posts: 2 Learner III
edited July 2020 in Help

Hello,

 

i work actually at a data set for a decision tree solution inspired from your video "finding the right model".

 

The roc based on the data set shows me a good result for decision tree and random forest.

But when i want to create a simple decision tree model i get a very low recall of 1,12% for TruePositiv.

With Optimize Parameters i can increase it to 47% for TruePositiv.

But i see in documentations, that a TruePositiv of 99% is possible.

 

Is something wrong with my process?

 

The XML-Code:

<?xml version="1.0" encoding="UTF-8"?><process version="8.2.000">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="8.2.000" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="retrieve" compatibility="8.2.000" expanded="true" height="68" name="Retrieve transfusion.dataset1" width="90" x="45" y="34">
        <parameter key="repository_entry" value="../data/transfusion.dataset1"/>
      </operator>
      <operator activated="true" class="rename" compatibility="8.2.000" expanded="true" height="82" name="Rename" width="90" x="179" y="34">
        <parameter key="old_name" value="whether he/she donated blood in March 2007"/>
        <parameter key="new_name" value="Donation"/>
        <list key="rename_additional_attributes">
          <parameter key="Frequency (times)" value="Frequency"/>
          <parameter key="Monetary (c.c. blood)" value="Monetary"/>
          <parameter key="Recency (months)" value="Recency"/>
          <parameter key="Time (months)" value="Time"/>
          <parameter key="ID (customer)" value="ID"/>
        </list>
      </operator>
      <operator activated="true" class="numerical_to_binominal" compatibility="8.2.000" expanded="true" height="82" name="Numerical to Binominal" width="90" x="313" y="34">
        <parameter key="attribute_filter_type" value="single"/>
        <parameter key="attribute" value="Donation"/>
      </operator>
      <operator activated="true" class="set_role" compatibility="8.2.000" expanded="true" height="82" name="Set Role" width="90" x="447" y="34">
        <parameter key="attribute_name" value="Donation"/>
        <parameter key="target_role" value="label"/>
        <list key="set_additional_roles">
          <parameter key="ID" value="id"/>
          <parameter key="Frequency" value="regular"/>
          <parameter key="Monetary" value="regular"/>
          <parameter key="Recency" value="regular"/>
          <parameter key="Time" value="regular"/>
        </list>
      </operator>
      <operator activated="true" class="select_attributes" compatibility="8.2.000" expanded="true" height="82" name="Select Attributes" width="90" x="581" y="34">
        <parameter key="attribute_filter_type" value="subset"/>
        <parameter key="attributes" value="Donation|Frequency|Recency|Time|Monetary"/>
        <parameter key="include_special_attributes" value="true"/>
      </operator>
      <operator activated="true" class="multiply" compatibility="8.2.000" expanded="true" height="124" name="Multiply (2)" width="90" x="715" y="34"/>
      <operator activated="true" class="concurrency:optimize_parameters_grid" compatibility="8.2.000" expanded="true" height="124" name="DT Optimize Parameters" width="90" x="849" y="289">
        <list key="parameters">
          <parameter key="Decision Tree (2).criterion" value="gini_index"/>
          <parameter key="Decision Tree (2).maximal_depth" value="[15;30;15;linear]"/>
          <parameter key="Decision Tree (2).minimal_gain" value="[0.040;0.050;10;linear]"/>
          <parameter key="Decision Tree (2).apply_prepruning" value="true"/>
          <parameter key="Decision Tree (2).minimal_size_for_split" value="[5;10;6;linear]"/>
          <parameter key="Decision Tree (2).number_of_prepruning_alternatives" value="[10;15;5;linear]"/>
        </list>
        <parameter key="log_all_criteria" value="true"/>
        <process expanded="true">
          <operator activated="true" class="concurrency:cross_validation" compatibility="8.2.000" expanded="true" height="145" name="DT Cross Validation (2)" width="90" x="45" y="34">
            <process expanded="true">
              <operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree (2)" width="90" x="112" y="34">
                <parameter key="criterion" value="gini_index"/>
              </operator>
              <connect from_port="training set" to_op="Decision Tree (2)" to_port="training set"/>
              <connect from_op="Decision Tree (2)" from_port="model" to_port="model"/>
              <portSpacing port="source_training set" spacing="0"/>
              <portSpacing port="sink_model" spacing="0"/>
              <portSpacing port="sink_through 1" spacing="0"/>
            </process>
            <process expanded="true">
              <operator activated="true" class="apply_model" compatibility="8.2.000" expanded="true" height="82" name="Apply Model (4)" width="90" x="112" y="34">
                <list key="application_parameters"/>
              </operator>
              <operator activated="true" class="performance_binominal_classification" compatibility="8.2.000" expanded="true" height="82" name="Performance (4)" width="90" x="246" y="34">
                <parameter key="main_criterion" value="recall"/>
                <parameter key="AUC (optimistic)" value="true"/>
                <parameter key="AUC" value="true"/>
                <parameter key="AUC (pessimistic)" value="true"/>
                <parameter key="precision" value="true"/>
                <parameter key="recall" value="true"/>
              </operator>
              <connect from_port="model" to_op="Apply Model (4)" to_port="model"/>
              <connect from_port="test set" to_op="Apply Model (4)" to_port="unlabelled data"/>
              <connect from_op="Apply Model (4)" from_port="labelled data" to_op="Performance (4)" to_port="labelled data"/>
              <connect from_op="Performance (4)" from_port="performance" to_port="performance 1"/>
              <portSpacing port="source_model" spacing="0"/>
              <portSpacing port="source_test set" spacing="0"/>
              <portSpacing port="source_through 1" spacing="0"/>
              <portSpacing port="sink_test set results" spacing="0"/>
              <portSpacing port="sink_performance 1" spacing="0"/>
              <portSpacing port="sink_performance 2" spacing="0"/>
            </process>
          </operator>
          <connect from_port="input 1" to_op="DT Cross Validation (2)" to_port="example set"/>
          <connect from_op="DT Cross Validation (2)" from_port="model" to_port="model"/>
          <connect from_op="DT Cross Validation (2)" from_port="performance 1" to_port="performance"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="source_input 2" spacing="0"/>
          <portSpacing port="sink_performance" spacing="0"/>
          <portSpacing port="sink_model" spacing="0"/>
          <portSpacing port="sink_output 1" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="concurrency:cross_validation" compatibility="8.2.000" expanded="true" height="145" name="Cross Validation" width="90" x="849" y="136">
        <process expanded="true">
          <operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree" width="90" x="112" y="34"/>
          <connect from_port="training set" to_op="Decision Tree" to_port="training set"/>
          <connect from_op="Decision Tree" from_port="model" to_port="model"/>
          <portSpacing port="source_training set" spacing="0"/>
          <portSpacing port="sink_model" spacing="0"/>
          <portSpacing port="sink_through 1" spacing="0"/>
        </process>
        <process expanded="true">
          <operator activated="true" class="apply_model" compatibility="8.2.000" expanded="true" height="82" name="Apply Model" width="90" x="112" y="34">
            <list key="application_parameters"/>
          </operator>
          <operator activated="true" class="performance_binominal_classification" compatibility="8.2.000" expanded="true" height="82" name="Performance" width="90" x="246" y="34">
            <parameter key="main_criterion" value="recall"/>
            <parameter key="precision" value="true"/>
            <parameter key="recall" value="true"/>
          </operator>
          <connect from_port="model" to_op="Apply Model" to_port="model"/>
          <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
          <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
          <connect from_op="Performance" from_port="performance" to_port="performance 1"/>
          <portSpacing port="source_model" spacing="0"/>
          <portSpacing port="source_test set" spacing="0"/>
          <portSpacing port="source_through 1" spacing="0"/>
          <portSpacing port="sink_test set results" spacing="0"/>
          <portSpacing port="sink_performance 1" spacing="0"/>
          <portSpacing port="sink_performance 2" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="compare_rocs" compatibility="8.2.000" expanded="true" height="82" name="Compare ROCs (3)" width="90" x="849" y="34">
        <parameter key="sampling_type" value="automatic"/>
        <parameter key="use_example_weights" value="false"/>
        <process expanded="true">
          <operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree (5)" width="90" x="447" y="34"/>
          <operator activated="true" class="naive_bayes" compatibility="8.2.000" expanded="true" height="82" name="Naive Bayes (3)" width="90" x="447" y="136"/>
          <operator activated="true" class="rule_induction" compatibility="8.2.000" expanded="true" height="82" name="Rule Induction (3)" width="90" x="447" y="238"/>
          <operator activated="true" class="random_tree" compatibility="8.2.000" expanded="true" height="82" name="Random Tree (3)" width="90" x="447" y="340"/>
          <operator activated="true" class="k_nn" compatibility="8.2.000" expanded="true" height="82" name="k-NN (5)" width="90" x="447" y="442"/>
          <connect from_port="train 1" to_op="Decision Tree (5)" to_port="training set"/>
          <connect from_port="train 2" to_op="Naive Bayes (3)" to_port="training set"/>
          <connect from_port="train 3" to_op="Rule Induction (3)" to_port="training set"/>
          <connect from_port="train 4" to_op="Random Tree (3)" to_port="training set"/>
          <connect from_port="train 5" to_op="k-NN (5)" to_port="training set"/>
          <connect from_op="Decision Tree (5)" from_port="model" to_port="model 1"/>
          <connect from_op="Naive Bayes (3)" from_port="model" to_port="model 2"/>
          <connect from_op="Rule Induction (3)" from_port="model" to_port="model 3"/>
          <connect from_op="Random Tree (3)" from_port="model" to_port="model 4"/>
          <connect from_op="k-NN (5)" from_port="model" to_port="model 5"/>
          <portSpacing port="source_train 1" spacing="0"/>
          <portSpacing port="source_train 2" spacing="0"/>
          <portSpacing port="source_train 3" spacing="0"/>
          <portSpacing port="source_train 4" spacing="0"/>
          <portSpacing port="source_train 5" spacing="0"/>
          <portSpacing port="source_train 6" spacing="0"/>
          <portSpacing port="sink_model 1" spacing="0"/>
          <portSpacing port="sink_model 2" spacing="0"/>
          <portSpacing port="sink_model 3" spacing="0"/>
          <portSpacing port="sink_model 4" spacing="0"/>
          <portSpacing port="sink_model 5" spacing="0"/>
          <portSpacing port="sink_model 6" spacing="0"/>
        </process>
      </operator>
      <connect from_op="Retrieve transfusion.dataset1" from_port="output" to_op="Rename" to_port="example set input"/>
      <connect from_op="Rename" from_port="example set output" to_op="Numerical to Binominal" to_port="example set input"/>
      <connect from_op="Numerical to Binominal" from_port="example set output" to_op="Set Role" to_port="example set input"/>
      <connect from_op="Set Role" from_port="example set output" to_op="Select Attributes" to_port="example set input"/>
      <connect from_op="Select Attributes" from_port="example set output" to_op="Multiply (2)" to_port="input"/>
      <connect from_op="Multiply (2)" from_port="output 1" to_op="Compare ROCs (3)" to_port="example set"/>
      <connect from_op="Multiply (2)" from_port="output 2" to_op="Cross Validation" to_port="example set"/>
      <connect from_op="Multiply (2)" from_port="output 3" to_op="DT Optimize Parameters" to_port="input 1"/>
      <connect from_op="DT Optimize Parameters" from_port="performance" to_port="result 4"/>
      <connect from_op="DT Optimize Parameters" from_port="model" to_port="result 5"/>
      <connect from_op="Cross Validation" from_port="model" to_port="result 2"/>
      <connect from_op="Cross Validation" from_port="performance 1" to_port="result 3"/>
      <connect from_op="Compare ROCs (3)" from_port="rocComparison" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
      <portSpacing port="sink_result 4" spacing="0"/>
      <portSpacing port="sink_result 5" spacing="0"/>
      <portSpacing port="sink_result 6" spacing="0"/>
    </process>
  </operator>
</process>


roc.png

Tagged:

Best Answer

  • lionelderkrikorlionelderkrikor RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    Solution Accepted

    Hi @cam8,

     

    There isn't wrong in your process.

    You have an imbalanced dataset (ratio true/false = 178/570). That's lead generally to a bad Recall.

    To increase the Recall, you can sample your data. By sampling your data, I'm obtaining Recall = 60% with a decision tree.

    However, if your goal is the research of the maximum performance, decision tree is not the best choice : a better choice

    is the kNN model. After training this model, you can obtain a recall > 70 %, (with  1< k < 10) and you can obtain a theoretical recall > 90% by optimizing the value of k. But use with caution, this last case correspond to a situation of overfitting : Your model has, in deed, good performances on your training dataset, but will have bad performances on future 'unseen data'.

     

    I hope it helps,

     

    Regards,

     

    Lionel

     

    NB : The process:

    <?xml version="1.0" encoding="UTF-8"?><process version="8.2.000">
      <context>
        <input/>
        <output/>
        <macros/>
      </context>
      <operator activated="true" class="process" compatibility="8.2.000" expanded="true" name="Process">
        <process expanded="true">
          <operator activated="true" class="read_csv" compatibility="8.2.000" expanded="true" height="68" name="Read CSV" width="90" x="45" y="34">
            <parameter key="csv_file" value="C:\Users\Lionel\Documents\Formations_DataScience\Rapidminer\Tests_Rapidminer\Donators_DT\Donators.csv"/>
            <parameter key="column_separators" value=",\s*|;\s*"/>
            <parameter key="first_row_as_names" value="false"/>
            <list key="annotations">
              <parameter key="0" value="Name"/>
            </list>
            <parameter key="encoding" value="windows-1252"/>
            <list key="data_set_meta_data_information">
              <parameter key="0" value="R&#0;e&#0;c&#0;e&#0;n&#0;c&#0;y&#0;(&#0;m&#0;o&#0;n&#0;t&#0;h&#0;s&#0;).true.polynominal.attribute"/>
              <parameter key="1" value="F&#0;r&#0;e&#0;q&#0;u&#0;e&#0;n&#0;c&#0;y&#0; &#0;(&#0;t&#0;i&#0;m&#0;e&#0;s&#0;).true.polynominal.attribute"/>
              <parameter key="2" value="M&#0;o&#0;n&#0;e&#0;t&#0;a&#0;r&#0;y&#0; &#0;(&#0;c&#0;\.&#0;c&#0;\.&#0; &#0;b&#0;l&#0;o&#0;o&#0;d&#0;).true.polynominal.attribute"/>
              <parameter key="3" value="T&#0;i&#0;m&#0;e&#0; &#0;(&#0;m&#0;o&#0;n&#0;t&#0;h&#0;s&#0;).true.polynominal.attribute"/>
              <parameter key="4" value="w&#0;h&#0;e&#0;t&#0;h&#0;e&#0;r&#0; &#0;h&#0;e&#0;/&#0;s&#0;h&#0;e&#0; &#0;d&#0;o&#0;n&#0;a&#0;t&#0;e&#0;d&#0; &#0;b&#0;l&#0;o&#0;o&#0;d&#0; &#0;i&#0;n&#0; &#0;M&#0;a&#0;r&#0;c&#0;h&#0; &#0;2&#0;0&#0;0&#0;7.true.integer.attribute"/>
            </list>
          </operator>
          <operator activated="true" class="rename" compatibility="8.2.000" expanded="true" height="82" name="Rename" width="90" x="179" y="34">
            <parameter key="old_name" value="w&#0;h&#0;e&#0;t&#0;h&#0;e&#0;r&#0; &#0;h&#0;e&#0;/&#0;s&#0;h&#0;e&#0; &#0;d&#0;o&#0;n&#0;a&#0;t&#0;e&#0;d&#0; &#0;b&#0;l&#0;o&#0;o&#0;d&#0; &#0;i&#0;n&#0; &#0;M&#0;a&#0;r&#0;c&#0;h&#0; &#0;2&#0;0&#0;0&#0;7"/>
            <parameter key="new_name" value="Donation"/>
            <list key="rename_additional_attributes">
              <parameter key="F&#0;r&#0;e&#0;q&#0;u&#0;e&#0;n&#0;c&#0;y&#0; &#0;(&#0;t&#0;i&#0;m&#0;e&#0;s&#0;)" value="Frequency"/>
              <parameter key="M&#0;o&#0;n&#0;e&#0;t&#0;a&#0;r&#0;y&#0; &#0;(&#0;c&#0;.&#0;c&#0;.&#0; &#0;b&#0;l&#0;o&#0;o&#0;d&#0;)" value="Monetary"/>
              <parameter key="R&#0;e&#0;c&#0;e&#0;n&#0;c&#0;y&#0;(&#0;m&#0;o&#0;n&#0;t&#0;h&#0;s&#0;)" value="Recency"/>
              <parameter key="T&#0;i&#0;m&#0;e&#0; &#0;(&#0;m&#0;o&#0;n&#0;t&#0;h&#0;s&#0;)" value="Time"/>
            </list>
          </operator>
          <operator activated="true" class="numerical_to_binominal" compatibility="8.2.000" expanded="true" height="82" name="Numerical to Binominal" width="90" x="313" y="34">
            <parameter key="attribute_filter_type" value="single"/>
            <parameter key="attribute" value="Donation"/>
          </operator>
          <operator activated="true" class="set_role" compatibility="8.2.000" expanded="true" height="82" name="Set Role" width="90" x="447" y="34">
            <parameter key="attribute_name" value="Donation"/>
            <parameter key="target_role" value="label"/>
            <list key="set_additional_roles"/>
          </operator>
          <operator activated="true" class="select_attributes" compatibility="8.2.000" expanded="true" height="82" name="Select Attributes" width="90" x="581" y="34">
            <parameter key="attribute_filter_type" value="subset"/>
            <parameter key="attributes" value="Donation|Frequency|Recency|Time|Monetary"/>
            <parameter key="include_special_attributes" value="true"/>
          </operator>
          <operator activated="true" class="multiply" compatibility="8.2.000" expanded="true" height="187" name="Multiply (2)" width="90" x="313" y="187"/>
          <operator activated="true" class="concurrency:optimize_parameters_grid" compatibility="8.2.000" expanded="true" height="124" name="DT Optimize Parameters" width="90" x="849" y="289">
            <list key="parameters">
              <parameter key="Decision Tree (2).criterion" value="gain_ratio"/>
              <parameter key="Sample (2).sample_size" value="[30;40;10;linear]"/>
              <parameter key="Decision Tree (2).confidence" value="[0.45;0.5;10;linear]"/>
              <parameter key="Decision Tree (2).minimal_leaf_size" value="[1.0;5;10;linear]"/>
              <parameter key="Decision Tree (2).minimal_size_for_split" value="[1.0;100.0;10;linear]"/>
            </list>
            <process expanded="true">
              <operator activated="true" class="extract_macro" compatibility="8.2.000" expanded="true" height="68" name="Extract Macro (2)" width="90" x="112" y="34">
                <parameter key="macro" value="exampleCount"/>
                <parameter key="macro_type" value="statistics"/>
                <parameter key="statistics" value="count"/>
                <parameter key="attribute_name" value="Donation"/>
                <parameter key="attribute_value" value="true"/>
                <list key="additional_macros"/>
              </operator>
              <operator activated="true" class="concurrency:cross_validation" compatibility="8.2.000" expanded="true" height="145" name="DT Cross Validation (2)" width="90" x="581" y="34">
                <process expanded="true">
                  <operator activated="true" class="sample" compatibility="8.2.000" expanded="true" height="82" name="Sample (2)" width="90" x="179" y="34">
                    <parameter key="balance_data" value="true"/>
                    <parameter key="sample_size" value="-1"/>
                    <list key="sample_size_per_class">
                      <parameter key="true" value="%{exampleCount}"/>
                      <parameter key="false" value="%{exampleCount}"/>
                    </list>
                    <list key="sample_ratio_per_class"/>
                    <list key="sample_probability_per_class"/>
                  </operator>
                  <operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree (2)" width="90" x="380" y="34">
                    <parameter key="criterion" value="gini_index"/>
                    <parameter key="apply_prepruning" value="false"/>
                    <parameter key="minimal_gain" value="0.01"/>
                    <parameter key="minimal_leaf_size" value="4"/>
                  </operator>
                  <connect from_port="training set" to_op="Sample (2)" to_port="example set input"/>
                  <connect from_op="Sample (2)" from_port="example set output" to_op="Decision Tree (2)" to_port="training set"/>
                  <connect from_op="Decision Tree (2)" from_port="model" to_port="model"/>
                  <portSpacing port="source_training set" spacing="0"/>
                  <portSpacing port="sink_model" spacing="0"/>
                  <portSpacing port="sink_through 1" spacing="0"/>
                </process>
                <process expanded="true">
                  <operator activated="true" class="apply_model" compatibility="8.2.000" expanded="true" height="82" name="Apply Model (4)" width="90" x="112" y="34">
                    <list key="application_parameters"/>
                  </operator>
                  <operator activated="true" class="performance_binominal_classification" compatibility="8.2.000" expanded="true" height="82" name="Performance (4)" width="90" x="246" y="34">
                    <parameter key="main_criterion" value="recall"/>
                    <parameter key="AUC (optimistic)" value="true"/>
                    <parameter key="AUC" value="true"/>
                    <parameter key="AUC (pessimistic)" value="true"/>
                    <parameter key="precision" value="true"/>
                    <parameter key="recall" value="true"/>
                  </operator>
                  <connect from_port="model" to_op="Apply Model (4)" to_port="model"/>
                  <connect from_port="test set" to_op="Apply Model (4)" to_port="unlabelled data"/>
                  <connect from_op="Apply Model (4)" from_port="labelled data" to_op="Performance (4)" to_port="labelled data"/>
                  <connect from_op="Performance (4)" from_port="performance" to_port="performance 1"/>
                  <portSpacing port="source_model" spacing="0"/>
                  <portSpacing port="source_test set" spacing="0"/>
                  <portSpacing port="source_through 1" spacing="0"/>
                  <portSpacing port="sink_test set results" spacing="0"/>
                  <portSpacing port="sink_performance 1" spacing="0"/>
                  <portSpacing port="sink_performance 2" spacing="0"/>
                </process>
              </operator>
              <connect from_port="input 1" to_op="Extract Macro (2)" to_port="example set"/>
              <connect from_op="Extract Macro (2)" from_port="example set" to_op="DT Cross Validation (2)" to_port="example set"/>
              <connect from_op="DT Cross Validation (2)" from_port="model" to_port="model"/>
              <connect from_op="DT Cross Validation (2)" from_port="performance 1" to_port="performance"/>
              <portSpacing port="source_input 1" spacing="0"/>
              <portSpacing port="source_input 2" spacing="0"/>
              <portSpacing port="sink_performance" spacing="0"/>
              <portSpacing port="sink_model" spacing="0"/>
              <portSpacing port="sink_output 1" spacing="0"/>
            </process>
          </operator>
          <operator activated="true" class="concurrency:optimize_parameters_grid" compatibility="8.2.000" expanded="true" height="124" name="kNN Optimize Parameters (2)" width="90" x="849" y="493">
            <list key="parameters">
              <parameter key="k-NN (2).k" value="[1.0;500;500;linear]"/>
            </list>
            <process expanded="true">
              <operator activated="true" class="extract_macro" compatibility="8.2.000" expanded="true" height="68" name="Extract Macro (3)" width="90" x="112" y="34">
                <parameter key="macro" value="exampleCount"/>
                <parameter key="macro_type" value="statistics"/>
                <parameter key="statistics" value="count"/>
                <parameter key="attribute_name" value="Donation"/>
                <parameter key="attribute_value" value="true"/>
                <list key="additional_macros"/>
              </operator>
              <operator activated="true" class="concurrency:cross_validation" compatibility="8.2.000" expanded="true" height="145" name="DT Cross Validation (3)" width="90" x="581" y="34">
                <process expanded="true">
                  <operator activated="true" class="sample" compatibility="8.2.000" expanded="true" height="82" name="Sample (3)" width="90" x="179" y="34">
                    <parameter key="balance_data" value="true"/>
                    <parameter key="sample_size" value="-1"/>
                    <list key="sample_size_per_class">
                      <parameter key="true" value="%{exampleCount}"/>
                      <parameter key="false" value="%{exampleCount}"/>
                    </list>
                    <list key="sample_ratio_per_class"/>
                    <list key="sample_probability_per_class"/>
                  </operator>
                  <operator activated="true" class="k_nn" compatibility="8.2.000" expanded="true" height="82" name="k-NN (2)" width="90" x="380" y="34">
                    <parameter key="k" value="5"/>
                    <parameter key="weighted_vote" value="true"/>
                  </operator>
                  <connect from_port="training set" to_op="Sample (3)" to_port="example set input"/>
                  <connect from_op="Sample (3)" from_port="example set output" to_op="k-NN (2)" to_port="training set"/>
                  <connect from_op="k-NN (2)" from_port="model" to_port="model"/>
                  <portSpacing port="source_training set" spacing="0"/>
                  <portSpacing port="sink_model" spacing="0"/>
                  <portSpacing port="sink_through 1" spacing="0"/>
                </process>
                <process expanded="true">
                  <operator activated="true" class="apply_model" compatibility="8.2.000" expanded="true" height="82" name="Apply Model (2)" width="90" x="112" y="34">
                    <list key="application_parameters"/>
                  </operator>
                  <operator activated="true" class="performance_binominal_classification" compatibility="8.2.000" expanded="true" height="82" name="Performance (2)" width="90" x="246" y="34">
                    <parameter key="main_criterion" value="recall"/>
                    <parameter key="AUC (optimistic)" value="true"/>
                    <parameter key="AUC" value="true"/>
                    <parameter key="AUC (pessimistic)" value="true"/>
                    <parameter key="precision" value="true"/>
                    <parameter key="recall" value="true"/>
                  </operator>
                  <connect from_port="model" to_op="Apply Model (2)" to_port="model"/>
                  <connect from_port="test set" to_op="Apply Model (2)" to_port="unlabelled data"/>
                  <connect from_op="Apply Model (2)" from_port="labelled data" to_op="Performance (2)" to_port="labelled data"/>
                  <connect from_op="Performance (2)" from_port="performance" to_port="performance 1"/>
                  <portSpacing port="source_model" spacing="0"/>
                  <portSpacing port="source_test set" spacing="0"/>
                  <portSpacing port="source_through 1" spacing="0"/>
                  <portSpacing port="sink_test set results" spacing="0"/>
                  <portSpacing port="sink_performance 1" spacing="0"/>
                  <portSpacing port="sink_performance 2" spacing="0"/>
                </process>
              </operator>
              <connect from_port="input 1" to_op="Extract Macro (3)" to_port="example set"/>
              <connect from_op="Extract Macro (3)" from_port="example set" to_op="DT Cross Validation (3)" to_port="example set"/>
              <connect from_op="DT Cross Validation (3)" from_port="model" to_port="model"/>
              <connect from_op="DT Cross Validation (3)" from_port="performance 1" to_port="performance"/>
              <portSpacing port="source_input 1" spacing="0"/>
              <portSpacing port="source_input 2" spacing="0"/>
              <portSpacing port="sink_performance" spacing="0"/>
              <portSpacing port="sink_model" spacing="0"/>
              <portSpacing port="sink_output 1" spacing="0"/>
            </process>
          </operator>
          <operator activated="true" class="compare_rocs" compatibility="8.2.000" expanded="true" height="82" name="Compare ROCs (3)" width="90" x="849" y="34">
            <parameter key="sampling_type" value="automatic"/>
            <parameter key="use_example_weights" value="false"/>
            <process expanded="true">
              <operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree (5)" width="90" x="447" y="34"/>
              <operator activated="true" class="naive_bayes" compatibility="8.2.000" expanded="true" height="82" name="Naive Bayes (3)" width="90" x="447" y="136"/>
              <operator activated="true" class="rule_induction" compatibility="8.2.000" expanded="true" height="82" name="Rule Induction (3)" width="90" x="447" y="238"/>
              <operator activated="true" class="random_tree" compatibility="8.2.000" expanded="true" height="82" name="Random Tree (3)" width="90" x="447" y="340"/>
              <operator activated="true" class="k_nn" compatibility="8.2.000" expanded="true" height="82" name="k-NN (5)" width="90" x="447" y="442"/>
              <connect from_port="train 1" to_op="Decision Tree (5)" to_port="training set"/>
              <connect from_port="train 2" to_op="Naive Bayes (3)" to_port="training set"/>
              <connect from_port="train 3" to_op="Rule Induction (3)" to_port="training set"/>
              <connect from_port="train 4" to_op="Random Tree (3)" to_port="training set"/>
              <connect from_port="train 5" to_op="k-NN (5)" to_port="training set"/>
              <connect from_op="Decision Tree (5)" from_port="model" to_port="model 1"/>
              <connect from_op="Naive Bayes (3)" from_port="model" to_port="model 2"/>
              <connect from_op="Rule Induction (3)" from_port="model" to_port="model 3"/>
              <connect from_op="Random Tree (3)" from_port="model" to_port="model 4"/>
              <connect from_op="k-NN (5)" from_port="model" to_port="model 5"/>
              <portSpacing port="source_train 1" spacing="0"/>
              <portSpacing port="source_train 2" spacing="0"/>
              <portSpacing port="source_train 3" spacing="0"/>
              <portSpacing port="source_train 4" spacing="0"/>
              <portSpacing port="source_train 5" spacing="0"/>
              <portSpacing port="source_train 6" spacing="0"/>
              <portSpacing port="sink_model 1" spacing="0"/>
              <portSpacing port="sink_model 2" spacing="0"/>
              <portSpacing port="sink_model 3" spacing="0"/>
              <portSpacing port="sink_model 4" spacing="0"/>
              <portSpacing port="sink_model 5" spacing="0"/>
              <portSpacing port="sink_model 6" spacing="0"/>
            </process>
          </operator>
          <operator activated="true" class="concurrency:cross_validation" compatibility="8.2.000" expanded="true" height="145" name="Cross Validation" width="90" x="849" y="136">
            <process expanded="true">
              <operator activated="true" class="extract_macro" compatibility="8.2.000" expanded="true" height="68" name="Extract Macro" width="90" x="112" y="34">
                <parameter key="macro" value="exampleCount"/>
                <parameter key="macro_type" value="statistics"/>
                <parameter key="statistics" value="count"/>
                <parameter key="attribute_name" value="Donation"/>
                <parameter key="attribute_value" value="true"/>
                <list key="additional_macros"/>
              </operator>
              <operator activated="true" class="sample" compatibility="8.2.000" expanded="true" height="82" name="Sample" width="90" x="246" y="34">
                <parameter key="balance_data" value="true"/>
                <parameter key="sample_size" value="-1"/>
                <list key="sample_size_per_class">
                  <parameter key="true" value="%{exampleCount}"/>
                  <parameter key="false" value="%{exampleCount}"/>
                </list>
                <list key="sample_ratio_per_class"/>
                <list key="sample_probability_per_class"/>
              </operator>
              <operator activated="true" class="concurrency:parallel_decision_tree" compatibility="8.2.000" expanded="true" height="103" name="Decision Tree" width="90" x="380" y="34"/>
              <connect from_port="training set" to_op="Extract Macro" to_port="example set"/>
              <connect from_op="Extract Macro" from_port="example set" to_op="Sample" to_port="example set input"/>
              <connect from_op="Sample" from_port="example set output" to_op="Decision Tree" to_port="training set"/>
              <connect from_op="Decision Tree" from_port="model" to_port="model"/>
              <portSpacing port="source_training set" spacing="0"/>
              <portSpacing port="sink_model" spacing="0"/>
              <portSpacing port="sink_through 1" spacing="0"/>
            </process>
            <process expanded="true">
              <operator activated="true" class="apply_model" compatibility="8.2.000" expanded="true" height="82" name="Apply Model" width="90" x="112" y="34">
                <list key="application_parameters"/>
              </operator>
              <operator activated="true" class="performance_binominal_classification" compatibility="8.2.000" expanded="true" height="82" name="Performance" width="90" x="246" y="34">
                <parameter key="main_criterion" value="recall"/>
                <parameter key="precision" value="true"/>
                <parameter key="recall" value="true"/>
              </operator>
              <connect from_port="model" to_op="Apply Model" to_port="model"/>
              <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
              <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
              <connect from_op="Performance" from_port="performance" to_port="performance 1"/>
              <portSpacing port="source_model" spacing="0"/>
              <portSpacing port="source_test set" spacing="0"/>
              <portSpacing port="source_through 1" spacing="0"/>
              <portSpacing port="sink_test set results" spacing="0"/>
              <portSpacing port="sink_performance 1" spacing="0"/>
              <portSpacing port="sink_performance 2" spacing="0"/>
            </process>
          </operator>
          <connect from_op="Read CSV" from_port="output" to_op="Rename" to_port="example set input"/>
          <connect from_op="Rename" from_port="example set output" to_op="Numerical to Binominal" to_port="example set input"/>
          <connect from_op="Numerical to Binominal" from_port="example set output" to_op="Set Role" to_port="example set input"/>
          <connect from_op="Set Role" from_port="example set output" to_op="Select Attributes" to_port="example set input"/>
          <connect from_op="Select Attributes" from_port="example set output" to_op="Multiply (2)" to_port="input"/>
          <connect from_op="Multiply (2)" from_port="output 1" to_op="Compare ROCs (3)" to_port="example set"/>
          <connect from_op="Multiply (2)" from_port="output 2" to_port="result 4"/>
          <connect from_op="Multiply (2)" from_port="output 3" to_op="kNN Optimize Parameters (2)" to_port="input 1"/>
          <connect from_op="Multiply (2)" from_port="output 4" to_op="Cross Validation" to_port="example set"/>
          <connect from_op="Multiply (2)" from_port="output 5" to_port="result 8"/>
          <connect from_op="Multiply (2)" from_port="output 6" to_op="DT Optimize Parameters" to_port="input 1"/>
          <connect from_op="DT Optimize Parameters" from_port="performance" to_port="result 9"/>
          <connect from_op="DT Optimize Parameters" from_port="model" to_port="result 10"/>
          <connect from_op="DT Optimize Parameters" from_port="parameter set" to_port="result 11"/>
          <connect from_op="kNN Optimize Parameters (2)" from_port="performance" to_port="result 5"/>
          <connect from_op="kNN Optimize Parameters (2)" from_port="model" to_port="result 6"/>
          <connect from_op="kNN Optimize Parameters (2)" from_port="parameter set" to_port="result 7"/>
          <connect from_op="Compare ROCs (3)" from_port="rocComparison" to_port="result 1"/>
          <connect from_op="Cross Validation" from_port="model" to_port="result 2"/>
          <connect from_op="Cross Validation" from_port="performance 1" to_port="result 3"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>
          <portSpacing port="sink_result 3" spacing="0"/>
          <portSpacing port="sink_result 4" spacing="0"/>
          <portSpacing port="sink_result 5" spacing="0"/>
          <portSpacing port="sink_result 6" spacing="0"/>
          <portSpacing port="sink_result 7" spacing="0"/>
          <portSpacing port="sink_result 8" spacing="0"/>
          <portSpacing port="sink_result 9" spacing="0"/>
          <portSpacing port="sink_result 10" spacing="0"/>
          <portSpacing port="sink_result 11" spacing="0"/>
          <portSpacing port="sink_result 12" spacing="0"/>
        </process>
      </operator>
    </process>

     

Answers

  • cam8cam8 Member Posts: 2 Learner III

    Hey @lionelderkrikor,

     

    thank you for your support.

    I was already worried about the model.

    I test your solution with the macro- an the sample-operator.

    The accuracy increased instantly.

     

    With your solution i tested a few other things and see, that the cross validation was a important fact for decreasing the accuracy.

    Now I am working with the Optimize Parameters and directly below with your solution and the model without the cross validation.

     

    Again: Thank you very much.

    Best Regards

    car8

Sign In or Register to comment.