Saving Perfromance Vector

tobix10

Is it possible to save performance vector(all measurements, roc curve ...) ?

I've tried Write Performance, but I couldn't read it from file. Also, I've used the store operator, but results were wrong.

earmijo

You could use the operator "Performance to Data". This saves the performance vectors to an Example Set and then export it to Excel/Csv or store it as an Example Dataset.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.3.008">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.3.008" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="retrieve" compatibility="5.3.008" expanded="true" height="60" name="Retrieve Sonar" width="90" x="112" y="75">
        <parameter key="repository_entry" value="//Samples/data/Sonar"/>
      </operator>
      <operator activated="true" class="x_validation" compatibility="5.1.002" expanded="true" height="112" name="Validation" width="90" x="246" y="75">
        <description>A cross-validation evaluating a decision tree model.</description>
        <process expanded="true">
          <operator activated="true" class="decision_tree" compatibility="5.3.008" expanded="true" height="76" name="Decision Tree" width="90" x="45" y="30"/>
          <connect from_port="training" to_op="Decision Tree" to_port="training set"/>
          <connect from_op="Decision Tree" from_port="model" to_port="model"/>
          <portSpacing port="source_training" spacing="0"/>
          <portSpacing port="sink_model" spacing="0"/>
          <portSpacing port="sink_through 1" spacing="0"/>
        </process>
        <process expanded="true">
          <operator activated="true" class="apply_model" compatibility="5.3.008" expanded="true" height="76" name="Apply Model" width="90" x="45" y="30">
            <list key="application_parameters"/>
          </operator>
          <operator activated="true" class="performance_binominal_classification" compatibility="5.3.008" expanded="true" height="76" name="Performance" width="90" x="231" y="30">
            <parameter key="classification_error" value="true"/>
            <parameter key="kappa" value="true"/>
            <parameter key="AUC (optimistic)" value="true"/>
            <parameter key="AUC" value="true"/>
            <parameter key="AUC (pessimistic)" value="true"/>
            <parameter key="precision" value="true"/>
            <parameter key="recall" value="true"/>
            <parameter key="lift" value="true"/>
            <parameter key="fallout" value="true"/>
            <parameter key="f_measure" value="true"/>
            <parameter key="false_positive" value="true"/>
            <parameter key="false_negative" value="true"/>
            <parameter key="true_positive" value="true"/>
            <parameter key="true_negative" value="true"/>
            <parameter key="sensitivity" value="true"/>
            <parameter key="specificity" value="true"/>
            <parameter key="youden" value="true"/>
            <parameter key="positive_predictive_value" value="true"/>
            <parameter key="negative_predictive_value" value="true"/>
            <parameter key="psep" value="true"/>
          </operator>
          <connect from_port="model" to_op="Apply Model" to_port="model"/>
          <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
          <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
          <connect from_op="Performance" from_port="performance" to_port="averagable 1"/>
          <portSpacing port="source_model" spacing="0"/>
          <portSpacing port="source_test set" spacing="0"/>
          <portSpacing port="source_through 1" spacing="0"/>
          <portSpacing port="sink_averagable 1" spacing="0"/>
          <portSpacing port="sink_averagable 2" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="performance_to_data" compatibility="5.3.008" expanded="true" height="76" name="Performance to Data" width="90" x="380" y="120"/>
      <operator activated="true" class="store" compatibility="5.3.008" expanded="true" height="60" name="Store" width="90" x="514" y="165">
        <parameter key="repository_entry" value="//Otros/performance-example-set"/>
      </operator>
      <connect from_op="Retrieve Sonar" from_port="output" to_op="Validation" to_port="training"/>
      <connect from_op="Validation" from_port="averagable 1" to_op="Performance to Data" to_port="performance vector"/>
      <connect from_op="Performance to Data" from_port="example set" to_op="Store" to_port="input"/>
      <connect from_op="Store" from_port="through" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

tobix10

Thx, but it saves only performances so I think write as text operator will be better.

It's weird that you can easily save model, but not performance vector ;/.

MariusHelf

I suppose you are referring to the Write Model operator - that operator is deprecated!
With the help of the Store operator you can store anything that comes out of an operator into the repository. From there, you can open and inspect it, or load it via the Retrieve operator in other processes.

Best regards,
Marius