About singular value decomposition

morphism

Howdi?

I am working on Text Mining project.

In text mining, SVD(singular value decomposition) is often used.

If M is a matrix given, SVD decomposes M = U D V^t ..... like this, where D is diagonal matrix

In RapidMiner, if I use SVD, it gives only matrix U and D, then where can I get the component V ??

Thank you...

sgenzer

hi @morphism sorry no one has chimed in here. Is this still a problem?

LevenD

has anybody an answer ?

yyhuang

Hi @LevenD & @morphism,

Let's do some cool linear algebra.

Here is a nice handbook about SVD
https://www.cs.cmu.edu/~venkatg/teaching/CStheory-infoage/book-chapter-4.pdf

On page 29, Exercise 1.6

if we have A= UDV', U is the left singular vector of A, V' is the transpose of V (right singular vector of A),
thus A'=  (UDV')' = V DU'.
D'=D because D is a diagonal matrix, (V')'=V is obvious.
So, we can get the component V if we apply SVD operator on the transposed data matrix.

Attached is the example process on sonar data, you can extract the svd vectors by using converters extensions from marketplace.

<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="6.0.002" expanded="true" name="Process" origin="GENERATED_TUTORIAL">
    <parameter key="logverbosity" value="init"/>
    <parameter key="random_seed" value="2001"/>
    <parameter key="send_mail" value="never"/>
    <parameter key="notification_email" value=""/>
    <parameter key="process_duration_for_mail" value="30"/>
    <parameter key="encoding" value="SYSTEM"/>
    <process expanded="true">
      <operator activated="true" breakpoints="after" class="retrieve" compatibility="9.6.000" expanded="true" height="68" name="Sonar" origin="GENERATED_TUTORIAL" width="90" x="179" y="34">
        <parameter key="repository_entry" value="//Samples/data/Sonar"/>
      </operator>
      <operator activated="true" class="singular_value_decomposition" compatibility="9.6.000" expanded="true" height="103" name="SVD on Sonar data matrix A" origin="GENERATED_TUTORIAL" width="90" x="380" y="34">
        <parameter key="dimensionality_reduction" value="none"/>
        <parameter key="percentage_threshold" value="0.95"/>
        <parameter key="dimensions" value="10"/>
      </operator>
      <operator activated="true" class="transpose" compatibility="9.6.000" expanded="true" height="82" name="Transpose" width="90" x="514" y="289"/>
      <operator activated="true" class="filter_examples" compatibility="9.6.000" expanded="true" height="103" name="Filter Examples" width="90" x="648" y="289">
        <parameter key="parameter_expression" value=""/>
        <parameter key="condition_class" value="custom_filters"/>
        <parameter key="invert_filter" value="false"/>
        <list key="filters_list">
          <parameter key="filters_entry_key" value="id.does_not_equal.class"/>
        </list>
        <parameter key="filters_logic_and" value="true"/>
        <parameter key="filters_check_metadata" value="true"/>
      </operator>
      <operator activated="true" class="parse_numbers" compatibility="9.6.000" expanded="true" height="82" name="Parse Numbers" width="90" x="782" y="289">
        <parameter key="attribute_filter_type" value="all"/>
        <parameter key="attribute" value=""/>
        <parameter key="attributes" value=""/>
        <parameter key="use_except_expression" value="false"/>
        <parameter key="value_type" value="nominal"/>
        <parameter key="use_value_type_exception" value="false"/>
        <parameter key="except_value_type" value="file_path"/>
        <parameter key="block_type" value="single_value"/>
        <parameter key="use_block_type_exception" value="false"/>
        <parameter key="except_block_type" value="single_value"/>
        <parameter key="invert_selection" value="false"/>
        <parameter key="include_special_attributes" value="false"/>
        <parameter key="decimal_character" value="."/>
        <parameter key="grouped_digits" value="false"/>
        <parameter key="grouping_character" value=","/>
        <parameter key="infinity_representation" value=""/>
        <parameter key="unparsable_value_handling" value="fail"/>
      </operator>
      <operator activated="true" class="singular_value_decomposition" compatibility="9.6.000" expanded="true" height="103" name="SVD on A'" origin="GENERATED_TUTORIAL" width="90" x="916" y="289">
        <parameter key="dimensionality_reduction" value="none"/>
        <parameter key="percentage_threshold" value="0.95"/>
        <parameter key="dimensions" value="10"/>
      </operator>
      <connect from_op="Sonar" from_port="output" to_op="SVD on Sonar data matrix A" to_port="example set input"/>
      <connect from_op="SVD on Sonar data matrix A" from_port="example set output" to_port="result 1"/>
      <connect from_op="SVD on Sonar data matrix A" from_port="original" to_op="Transpose" to_port="example set input"/>
      <connect from_op="SVD on Sonar data matrix A" from_port="preprocessing model" to_port="result 2"/>
      <connect from_op="Transpose" from_port="example set output" to_op="Filter Examples" to_port="example set input"/>
      <connect from_op="Filter Examples" from_port="example set output" to_op="Parse Numbers" to_port="example set input"/>
      <connect from_op="Parse Numbers" from_port="example set output" to_op="SVD on A'" to_port="example set input"/>
      <connect from_op="SVD on A'" from_port="example set output" to_port="result 3"/>
      <connect from_op="SVD on A'" from_port="preprocessing model" to_port="result 4"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
      <portSpacing port="sink_result 4" spacing="0"/>
      <portSpacing port="sink_result 5" spacing="0"/>
      <description align="left" color="yellow" colored="false" height="125" resized="true" width="639" x="107" y="153">A= UDV', U is the left singular vector of A, V' is the transpose of V (right sigular vector of A)&lt;br&gt;A = UDV'&lt;br&gt;thus A'= V DU'.&lt;br&gt;According to Page 29 in https://www.cs.cmu.edu/~venkatg/teaching/CStheory-infoage/book-chapter-4.pdf&lt;br&gt;&lt;br&gt;</description&gt;
    </process>
  </operator>
</process>