Neural Network Validation

bkruger

Hi,

I have a (basic - I think) question.  When I do a NN model and performance validation, I can clearly see that 1 case was incorrectly classified (refer below).  (There were more, but ignore that for now).

true CC true CF true GA class precision
pred. CC 1 0 2 33.33%
pred. CF 0 2 0 100.00%
pred. GA 1 0 11 91.67%
class recall 50.00% 100.00% 84.62%

This data is about authorship attribution, so I know that the author CC was mistakingly identified as author GA.  So now I want to look at this specific record and investigate it further.  How do I identify this specific case?

Any help would be appreciated.

Thanks
BK

SebastianLoh

Hi BK,

if I understood you right, you want to know which examples are classified wrong, is this correct?

In RM this is very easy. The "Filter Examples" operator allows you to filter wrong labeled examples. Take a look at the example process below to see how it works.

Ciao Sebastian

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" expanded="true" name="Process">
    <process expanded="true" height="651" width="722">
      <operator activated="true" class="retrieve" expanded="true" height="60" name="Retrieve (2)" width="90" x="45" y="210">
        <parameter key="repository_entry" value="//Samples/data/Golf-Testset"/>
      </operator>
      <operator activated="true" class="retrieve" expanded="true" height="60" name="Retrieve (3)" width="90" x="15" y="89">
        <parameter key="repository_entry" value="//Samples/data/Golf"/>
      </operator>
      <operator activated="true" class="decision_tree" expanded="true" height="76" name="Decision Tree" width="90" x="179" y="30"/>
      <operator activated="true" class="apply_model" expanded="true" height="76" name="Apply Model" width="90" x="246" y="210">
        <list key="application_parameters"/>
      </operator>
      <operator activated="true" class="multiply" expanded="true" height="94" name="Multiply" width="90" x="380" y="210"/>
      <operator activated="true" class="filter_examples" expanded="true" height="76" name="Filter Examples" width="90" x="581" y="255">
        <parameter key="condition_class" value="wrong_predictions"/>
      </operator>
      <operator activated="true" class="performance" expanded="true" height="76" name="Performance" width="90" x="447" y="30"/>
      <connect from_op="Retrieve (2)" from_port="output" to_op="Apply Model" to_port="unlabelled data"/>
      <connect from_op="Retrieve (3)" from_port="output" to_op="Decision Tree" to_port="training set"/>
      <connect from_op="Decision Tree" from_port="model" to_op="Apply Model" to_port="model"/>
      <connect from_op="Apply Model" from_port="labelled data" to_op="Multiply" to_port="input"/>
      <connect from_op="Multiply" from_port="output 1" to_op="Performance" to_port="labelled data"/>
      <connect from_op="Multiply" from_port="output 2" to_op="Filter Examples" to_port="example set input"/>
      <connect from_op="Filter Examples" from_port="example set output" to_port="result 2"/>
      <connect from_op="Performance" from_port="performance" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>

bkruger

Thank you Sebastian,  this is 100% what I was looking for.

Cheers BK