Speculative Rounds In Forward Selection

Sasch

Hello all,

I've got a short question:
What feature/attribue chooses the Forward Selection Operator in a speculative round?

Let's assume we have 5 features/attributes, "without increase" - stopping criterium, maximal number of attributes set to 5 and speculative rounds = 2.

1.Round :
 - att1 = 70 %
 - att2 = 60 %
 - att3 = 50 %
 - att4 = 45 %
 - att5 = 80 %
 => attribute 5 will be chosen

2.Round :
- (att5 + att1) = 60 %
- (att5 + att2) = 70 %
- (att5 + att3) = 80 %
- (att5 + att4) = 90 %
=> attribute 4 will be chosen

3.Round :
 - (att5 + att4 + att1) = 70 %
 - (att5 + att4 + att2) = 60 %
 - (att5 + att4 + att3) = 50 %
=> No increase => speculative round

So here's my question again: what attribute will now be chosen?
And what will happen next?

Sorry if this is a stupid question but I didn't find any answer neither here in this forum nor with google.

Thanx a lot in advance,
Sasch

MariusHelf

Hi Sasch,

actually, your third round is *not* a speculative round - the Forward Selection always adds one attribute and looks for increases. In your case, the best result was obtained in round 2. Since round 3 does not deliver a better result, the algorithm stops and delivers the best results found, i.e. att5+att4 from round 2.

If you configure one speculative round, the Forward Selection would try a 4th round, even though no increase could be achieved in round 3.

In any case, the attributes that delivered the best performance are returned.

Best regards,
Marius

MariusHelf

Experiment with this process to get a better feeling for what is happening:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.3.013">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.3.013" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="retrieve" compatibility="5.3.013" expanded="true" height="60" name="Retrieve Iris" width="90" x="45" y="30">
        <parameter key="repository_entry" value="//Samples/data/Iris"/>
      </operator>
      <operator activated="true" class="optimize_selection_forward" compatibility="5.3.013" expanded="true" height="94" name="Forward Selection" width="90" x="179" y="30">
        <process expanded="true">
          <operator activated="true" class="x_validation" compatibility="5.3.013" expanded="true" height="112" name="Validation" width="90" x="45" y="30">
            <description>A cross-validation evaluating a decision tree model.</description>
            <process expanded="true">
              <operator activated="true" class="decision_tree" compatibility="5.3.013" expanded="true" height="76" name="Decision Tree" width="90" x="45" y="30"/>
              <connect from_port="training" to_op="Decision Tree" to_port="training set"/>
              <connect from_op="Decision Tree" from_port="model" to_port="model"/>
              <portSpacing port="source_training" spacing="0"/>
              <portSpacing port="sink_model" spacing="0"/>
              <portSpacing port="sink_through 1" spacing="0"/>
            </process>
            <process expanded="true">
              <operator activated="true" class="apply_model" compatibility="5.3.013" expanded="true" height="76" name="Apply Model" width="90" x="45" y="30">
                <list key="application_parameters"/>
              </operator>
              <operator activated="true" class="performance" compatibility="5.3.013" expanded="true" height="76" name="Performance" width="90" x="179" y="30"/>
              <connect from_port="model" to_op="Apply Model" to_port="model"/>
              <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
              <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
              <connect from_op="Performance" from_port="performance" to_port="averagable 1"/>
              <portSpacing port="source_model" spacing="0"/>
              <portSpacing port="source_test set" spacing="0"/>
              <portSpacing port="source_through 1" spacing="0"/>
              <portSpacing port="sink_averagable 1" spacing="0"/>
              <portSpacing port="sink_averagable 2" spacing="0"/>
            </process>
          </operator>
          <operator activated="true" class="log" compatibility="5.3.013" expanded="true" height="76" name="Log" width="90" x="246" y="30">
            <list key="log">
              <parameter key="performance" value="operator.Validation.value.performance"/>
              <parameter key="attributes" value="operator.Forward Selection.value.feature_names"/>
            </list>
          </operator>
          <connect from_port="example set" to_op="Validation" to_port="training"/>
          <connect from_op="Validation" from_port="averagable 1" to_op="Log" to_port="through 1"/>
          <connect from_op="Log" from_port="through 1" to_port="performance"/>
          <portSpacing port="source_example set" spacing="0"/>
          <portSpacing port="sink_performance" spacing="0"/>
        </process>
      </operator>
      <connect from_op="Retrieve Iris" from_port="output" to_op="Forward Selection" to_port="example set"/>
      <connect from_op="Forward Selection" from_port="attribute weights" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

Sasch

Hello Marius,
thanks a lot for your answer.

I understood everything so far but I'm still interested in what attribute the Forward Selection picks in round 3
if I configure one speculative round?
=>
3.Round :
  - (att5 + att4 + att1) = 70 %
  - (att5 + att4 + att2) = 60 %
  - (att5 + att4 + att3) = 50 %
=> No increase => speculative round

Will it be att1 because it has the highest rate?

Do have any links to some literature about Forward Selection with speculative rounds?

Thanks again,
Sasch