How to visualise and interpret Kernel Model SVM?

Fred12

hi,

I have seen there is a chart option for LibSVM Kernel Model, but I have difficulties to understand of what use the visualisation has, or how to interpret the model...

I don't know how to use the visualization and therefore I have no idea how to interpret any signs... is the chart model of any use? e.g. can I visualise the decision boundary in some way?

 

e.g the number of support vectors is given, but how can I relate them to the classes or similar?

 

can somebody help me in how to interpret that chart?

Thomas_Ott

I think the best person to explain SVM's in RapidMiner would be @IngoRM

IngoRM

Sure, the Master of SVM to the rescue :smileytongue:

 

First of all: There is indeed not a lot use of those charts.  For low dimensional data sets, however, you can try and visualize decision boundaries by plotting the function values in the data dimensions.  But you can actually get the same effect by just applying the model to the training data and visualize that one.  The only advantage of doing this in the model is that you can also overlay the data points which became support vectors vs. not.  But since there is typically no surprises there (since data points close to the boundaries always become support vectors...) the use is somewhat, well, limited.  But for education purposes it sometimes can be helpful to show things like this...

 

Hope this helps (a bit),

Ingo

 

 

Fred12

ok thanks, I already thought something similar, 

my second question is: is there any tool out there maybe that can do something like visualiszing the decision boundaries of the libSVM? (or other SVM)? I have seen many complex SVM boundaries already online, but I don't know if it works also with higher dimensional data (like 6 or 7 dimensions)...

but there must be some program that is able to plot decision boundaries I think, maybe there is a R extension?

IngoRM

Hi,

 

There is not a lot of options in general.  The basic idea is to use a dimensionality reduction technique like Self-Organizing Maps (SOM) or Singular Value Decomposition (SVD) to reduce the number of dimensions to 2 and show the decision frontiers in this new space.  But of course then you can no longer really interpret those results easily so I always wonder a bit what actually the point of this approach is...

 

Anyway, you can do this in RM with the operator "Visualize Model by SOM".  Below is a quick sample process.  Press the "Calculate" button in the model visualization and you can also set the point color to "label".

 

Cheers,

Ingo

 

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="generate_data" compatibility="7.4.000" expanded="true" height="68" name="Generate Data" width="90" x="112" y="85">
        <parameter key="target_function" value="quadratic classification"/>
        <parameter key="number_examples" value="1000"/>
      </operator>
      <operator activated="true" class="support_vector_machine_libsvm" compatibility="7.4.000" expanded="true" height="82" name="SVM" width="90" x="246" y="85">
        <parameter key="C" value="1000.0"/>
        <list key="class_weights"/>
      </operator>
      <operator activated="true" class="visualize_model_by_som" compatibility="7.4.000" expanded="true" height="103" name="Visualize Model by SOM" width="90" x="514" y="85"/>
      <connect from_op="Generate Data" from_port="output" to_op="SVM" to_port="training set"/>
      <connect from_op="SVM" from_port="model" to_op="Visualize Model by SOM" to_port="model"/>
      <connect from_op="SVM" from_port="exampleSet" to_op="Visualize Model by SOM" to_port="example set"/>
      <connect from_op="Visualize Model by SOM" from_port="visualization" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

Fred12

thanks probably still better than nothing...

 

is it somehow possible to plot this in 3D Space? like in a scatterplot 3D but with decision boundary?