Extracting the real SVM data point values (used to plot the ROC curve)

virex

Hi, i'm using the Lib SVM in rapidminer. I would like to obtain the real data point values obtained after the SVM model is determined, that is used to plot the ROC curve. Is there anyway to obtain these values via the GUI itself? I searched and found in the rapidminer api that there is a container ROCData that holds all the ROC data points for a single ROC curve(http://rapid-i.com/api/rapidminer-5.1/com/rapidminer/tools/math/package-use.html). Is there any way to retrieve this data points?

AMT

virex wrote:

Hi, i'm using the Lib SVM in rapidminer. I would like to obtain the real data point values obtained after the SVM model is determined, that is used to plot the ROC curve. Is there anyway to obtain these values via the GUI itself? I searched and found in the rapidminer api that there is a container ROCData that holds all the ROC data points for a single ROC curve(http://rapid-i.com/api/rapidminer-5.1/com/rapidminer/tools/math/package-use.html). Is there any way to retrieve this data points?

I am also trying to use LIBSVM and I have the following doubts concerning the parameters:
w(attr1) and w(attr2) and bias.
1- if I use linear kernel it does not seem to me that the bias is correct . Sometimes seems more one the hyperplanes parallel to the decision one.
2- if I use an RBF kernel. These parameters (w and bias) are also shown and I do not know what is the meaning of such values. Any hint?
AMT

IngoRM

Hi Virex,

well, there is no simple GUI option like "Export ROC Points" or something similar. But of course you could create a process which generates the data points itself: Apply the model, create the confidences, sort them, count the true and false positive sums, create the rates etc. Advanced process design, but certainly possible.

Hi AMT,

well, your request does actually not have anything to do with the original topic - please start a new topic thread in future. Anyway, here are two comments:

1- well, if I do this anything looks fine (compare the process below, the bias is close to 0). By the way, in high dimensional space the intercept becomes completely irrelevant since it is only a single degree of freedom which can be more or less safely be ignored. This is also reflected by most SVM implementation so there might indeed be a notable difference for low-dimensional data sets.

2- No. You cannot interpret those values as easy as for the linear case for any other non-linear kernel function, sorry.

Cheers,
Ingo

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.1.003">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.1.003" expanded="true" name="Process">
    <process expanded="true" height="161" width="413">
      <operator activated="true" class="generate_data" compatibility="5.1.003" expanded="true" height="60" name="Generate Data" width="90" x="45" y="30">
        <parameter key="target_function" value="sum classification"/>
        <parameter key="number_examples" value="500"/>
        <parameter key="number_of_attributes" value="2"/>
      </operator>
      <operator activated="true" class="support_vector_machine_libsvm" compatibility="5.1.003" expanded="true" height="76" name="SVM" width="90" x="179" y="30">
        <parameter key="kernel_type" value="linear"/>
        <list key="class_weights"/>
      </operator>
      <connect from_op="Generate Data" from_port="output" to_op="SVM" to_port="training set"/>
      <connect from_op="SVM" from_port="model" to_port="result 1"/>
      <connect from_op="SVM" from_port="exampleSet" to_port="result 2"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>

AMT

Ingo Mierswa wrote:

Hi Virex, well, there is no simple GUI option like "Export ROC Points" or something similar. But of course you could create a process which generates the data points itself: Apply the model, create the confidences, sort them, count the true and false positive sums, create the rates etc. Advanced process design, but certainly possible. Hi AMT, well, your request does actually not have anything to do with the original topic - please start a new topic thread in future. Anyway, here are two comments: 1- well, if I do this anything looks fine (compare the process below, the bias is close to 0). By the way, in high dimensional space the intercept becomes completely irrelevant since it is only a single degree of freedom which can be more or less safely be ignored. This is also reflected by most SVM implementation so there might indeed be a notable difference for low-dimensional data sets. 2- No. You cannot interpret those values as easy as for the linear case for any other non-linear kernel function, sorry. Cheers, Ingo <?xml version="1.0" encoding="UTF-8" standalone="no"?> <process version="5.1.003">   <context>     <input/>     <output/>     <macros/>   </context>   <operator activated="true" class="process" compatibility="5.1.003" expanded="true" name="Process">     <process expanded="true" height="161" width="413">       <operator activated="true" class="generate_data" compatibility="5.1.003" expanded="true" height="60" name="Generate Data" width="90" x="45" y="30">         <parameter key="target_function" value="sum classification"/>         <parameter key="number_examples" value="500"/>         <parameter key="number_of_attributes" value="2"/>       </operator>       <operator activated="true" class="support_vector_machine_libsvm" compatibility="5.1.003" expanded="true" height="76" name="SVM" width="90" x="179" y="30">         <parameter key="kernel_type" value="linear"/>         <list key="class_weights"/>       </operator>       <connect from_op="Generate Data" from_port="output" to_op="SVM" to_port="training set"/>       <connect from_op="SVM" from_port="model" to_port="result 1"/>       <connect from_op="SVM" from_port="exampleSet" to_port="result 2"/>       <portSpacing port="source_input 1" spacing="0"/>       <portSpacing port="sink_result 1" spacing="0"/>       <portSpacing port="sink_result 2" spacing="0"/>       <portSpacing port="sink_result 3" spacing="0"/>     </process>   </operator> </process>

Yes you are right I should not put my question here...
In what concerns  the first question: I do not agree with you. This bias determines where the hyperplane pass in the multidimensional space, so it has a very concrete meaning.
And in what concerns the second question: the w that is written in the interface can not be related with the decision surface. For the RBF (or all other except linear)  the dual form for w should be used and only integrated on the dot operation product in order to use the so-called kernel tricky. So my question is what is the information that w is providing in this case?

AMT

IngoRM

Hi,

In what concerns  the first question: I do not agree with you. This bias determines where the hyperplane pass in the multidimensional space, so it has a very concrete meaning.

No problem - everybody has a right for his or her own opinion 

Just kidding: Did you imagine what it means for the optimization during SVM training if you keep the bias fixed to, for example, 0? Right, you reduce the degree of freedom by one - which is not much especially not for high-dimensional spaces. This is what many SVM implementations actually do anyway: ignore the bias during optimization and estimate it at the end based on the data and the values for alpha. This often leads to less accurate (but still good enough) estimations as it would be possible but you don't have to check the condition that the sum over all alphas and all labels always stays 0.

I of course know the meaning of the bias, I am just stating that it is less important than many people believe which might be caused by the fact that we are not really able to transfer our imaginations and visualizations from 2 to high numbers of dimensions. In 2 it is much more important than in 20,000. That's all what I said.

If you are interested in more details about this you might want to look into the SVM tutorial of Burges - he is the original author of the statement above as far as I remember.

So my question is what is the information that w is providing in this case?

And my answer already was: none. At least for me and you. I remember that I have seen a paper a couple of years ago where some interpretation was suggested - but I did not found it really convincing.

Cheers,
Ingo