Evolutionary Optimization - Performance Target

michaelgloven

I am using the optimize evolutionary operator and my interest is working towards a "best" sensitivity or positive binominal class recall for my learner - I believe its an iterative or trade-off process to learn a model to an acceptable positive class recall, but was curious if there was a criterion which would support my objective? I see there are weighted mean recall and mean precision criterion in the performance operator, but there's no documentation on how they are calculated. Does someone have this documentation and\or ideas on optimization to positive class recall? Thanks!

earmijo

If I understand correctly your question you want to optimize the parameters of an algorithm so as to maximize positive recall. 

 

If that's the cas (and you have a binomial label) you can change the operator Performance (Classification) to the more specialized Performance (Binomial Classification) and set "Recall" as your goal (main criterion). 

 

<?xml version="1.0" encoding="UTF-8"?><process version="8.0.001">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="6.0.002" expanded="true" name="Root">
    <parameter key="random_seed" value="2000"/>
    <process expanded="true">
      <operator activated="true" breakpoints="after" class="retrieve" compatibility="8.0.001" expanded="true" height="68" name="Sonar Dataset" width="90" x="112" y="136">
        <parameter key="repository_entry" value="//Samples/data/Sonar"/>
      </operator>
      <operator activated="true" class="optimize_selection_evolutionary" compatibility="8.0.001" expanded="true" height="103" name="Optimize Selection (Evolutionary)" width="90" x="380" y="136">
        <parameter key="keep_best_individual" value="true"/>
        <process expanded="true">
          <operator activated="true" class="split_validation" compatibility="8.0.001" expanded="true" height="124" name="Validation" width="90" x="313" y="30">
            <process expanded="true">
              <operator activated="true" class="support_vector_machine" compatibility="8.0.001" expanded="true" height="124" name="SVM" width="90" x="112" y="30"/>
              <connect from_port="training" to_op="SVM" to_port="training set"/>
              <connect from_op="SVM" from_port="model" to_port="model"/>
              <portSpacing port="source_training" spacing="0"/>
              <portSpacing port="sink_model" spacing="0"/>
              <portSpacing port="sink_through 1" spacing="0"/>
            </process>
            <process expanded="true">
              <operator activated="true" class="apply_model" compatibility="7.1.001" expanded="true" height="82" name="Apply Model" width="90" x="45" y="30">
                <list key="application_parameters"/>
              </operator>
              <operator activated="true" class="performance_binominal_classification" compatibility="8.0.001" expanded="true" height="82" name="Performance" width="90" x="246" y="32">
                <parameter key="main_criterion" value="recall"/>
              </operator>
              <connect from_port="model" to_op="Apply Model" to_port="model"/>
              <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
              <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
              <connect from_op="Performance" from_port="performance" to_port="averagable 1"/>
              <portSpacing port="source_model" spacing="0"/>
              <portSpacing port="source_test set" spacing="0"/>
              <portSpacing port="source_through 1" spacing="0"/>
              <portSpacing port="sink_averagable 1" spacing="0"/>
              <portSpacing port="sink_averagable 2" spacing="0"/>
            </process>
          </operator>
          <connect from_port="example set" to_op="Validation" to_port="training"/>
          <connect from_op="Validation" from_port="averagable 1" to_port="performance"/>
          <portSpacing port="source_example set" spacing="0"/>
          <portSpacing port="source_through 1" spacing="0"/>
          <portSpacing port="sink_performance" spacing="36"/>
        </process>
      </operator>
      <connect from_op="Sonar Dataset" from_port="output" to_op="Optimize Selection (Evolutionary)" to_port="example set in"/>
      <connect from_op="Optimize Selection (Evolutionary)" from_port="example set out" to_port="result 1"/>
      <connect from_op="Optimize Selection (Evolutionary)" from_port="performance" to_port="result 2"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="18"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>

michaelgloven

thanks for pointing this out, I've been using the classification performance operator all this time and did not realize the binominal criterion based operator was available...