Slope Function?

Ghostrider

Is there a function in RapidMiner TimeSeries library which can return the slope from the given point to previous point?  I searched, doesn't look like it.

awchisholm

Hello

It could be that the "Extract Linear Coefficients" operator might do the job as well as "Extract Linear Coeeficients (Piecewise)". But it can be difficult to be sure.

regards

Andrew

Ghostrider

The linear coefficient operators simply return a table with the coefficients.  I'd like to do a sliding window where I walk across the time series and for each point, use the previous n points to calculate the slope at the last point.  Best way to do this would be to use Fit Trend function (Time Series plug-in), however, I don't want to use all the data in the data set, only a window of the previous n points used to predict the slope at the last (rightmost) point in the window.  Anyone know how to do this either through RapidMiner or some other open-source software?

awchisholm

It can be done in Rapidminer but I just tried it and I couldn't make it work. I suspect a bug.

If I get time, I will try and isolate the issue and post what I get.

Andrew

Ghostrider

Thanks Andrew, I appreciate.

awchisholm

Hello Ghostrider

Well that was hard work. I've chopped a long wavelength sine wave into 4 bits and found the slope of each and it looks right.

I think you will have trouble using it but here it is.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.0.11" expanded="true" name="Process">
    <process expanded="true" height="598" width="927">
      <operator activated="true" class="series:generate_series_sinus" compatibility="5.0.2" expanded="true" height="60" name="Generate Series (Sinus)" width="90" x="112" y="75">
        <list key="frequency">
          <parameter key="0.45" value="1.0"/>
        </list>
      </operator>
      <operator activated="true" class="series:windowing_series" compatibility="5.0.2" expanded="true" height="60" name="Windowing (Series)" width="90" x="380" y="75">
        <parameter key="step_size" value="1000"/>
        <parameter key="window_size" value="4000"/>
        <process expanded="true" height="616" width="945">
          <operator activated="true" class="series:extract_linear_coefficients" compatibility="5.0.2" expanded="true" height="60" name="Extract Coefficients" width="90" x="447" y="30"/>
          <connect from_port="series source" to_op="Extract Coefficients" to_port="series"/>
          <connect from_op="Extract Coefficients" from_port="series" to_port="series sink"/>
          <portSpacing port="source_series source" spacing="0"/>
          <portSpacing port="sink_series sink" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="series:series_to_data" compatibility="5.0.2" expanded="true" height="60" name="Series To Data" width="90" x="648" y="75"/>
      <connect from_op="Generate Series (Sinus)" from_port="output" to_op="Windowing (Series)" to_port="series in"/>
      <connect from_op="Windowing (Series)" from_port="series out" to_op="Series To Data" to_port="series"/>
      <connect from_op="Series To Data" from_port="example set" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

regards

Andrew

Ghostrider

Thanks Andrew, I was looking for something that given an array of inputs gives an array of outputs where each value is the slope (or derivative) computed across the inputs at that point -- the output array would be the same length as the input array -- the output array would be the derivative of the input array at each point.  Your XML above returns a result window, I'm looking for the data in an example set.  If I figure out how to do this, I'll post it here.  Using R might be the best approach...

awchisholm

Hello again Ghostrider,

The output from my previous process is an example set - a small one - but an example set nonetheless. It's not obvious how it relates to the input data so I can imagine it's hard to use.

You can use the differentiate operator for series and I've shown an example that does this. The example set contains the original data and its differential at the same point. Not sure it it helps and I think I agree with you that R is the way to go

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.0.11" expanded="true" name="Process">
    <process expanded="true" height="598" width="927">
      <operator activated="true" class="series:generate_series_sinus" compatibility="5.0.2" expanded="true" height="60" name="Generate Series (Sinus)" width="90" x="112" y="165">
        <list key="frequency">
          <parameter key="0.45" value="1.0"/>
        </list>
      </operator>
      <operator activated="true" class="multiply" compatibility="5.0.11" expanded="true" height="94" name="Multiply" width="90" x="246" y="165"/>
      <operator activated="true" class="series:series_to_data" compatibility="5.0.2" expanded="true" height="60" name="Series To Data (2)" width="90" x="514" y="165"/>
      <operator activated="true" class="generate_id" compatibility="5.0.11" expanded="true" height="76" name="Generate ID (2)" width="90" x="645" y="163"/>
      <operator activated="true" class="series:differentiate_series" compatibility="5.0.2" expanded="true" height="60" name="Differentiate (2)" width="90" x="380" y="75"/>
      <operator activated="true" class="series:series_to_data" compatibility="5.0.2" expanded="true" height="60" name="Series To Data" width="90" x="514" y="75"/>
      <operator activated="true" class="generate_id" compatibility="5.0.11" expanded="true" height="76" name="Generate ID" width="90" x="646" y="74"/>
      <operator activated="true" class="join" compatibility="5.0.11" expanded="true" height="76" name="Join" width="90" x="813" y="73"/>
      <connect from_op="Generate Series (Sinus)" from_port="output" to_op="Multiply" to_port="input"/>
      <connect from_op="Multiply" from_port="output 1" to_op="Differentiate (2)" to_port="series"/>
      <connect from_op="Multiply" from_port="output 2" to_op="Series To Data (2)" to_port="series"/>
      <connect from_op="Series To Data (2)" from_port="example set" to_op="Generate ID (2)" to_port="example set input"/>
      <connect from_op="Generate ID (2)" from_port="example set output" to_op="Join" to_port="right"/>
      <connect from_op="Differentiate (2)" from_port="series" to_op="Series To Data" to_port="series"/>
      <connect from_op="Series To Data" from_port="example set" to_op="Generate ID" to_port="example set input"/>
      <connect from_op="Generate ID" from_port="example set output" to_op="Join" to_port="left"/>
      <connect from_op="Join" from_port="join" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="36"/>
      <portSpacing port="sink_result 2" spacing="36"/>
    </process>
  </operator>
</process>

regards

Andrew

Ghostrider

Yeah, the problem is that the output is not indexed to the same reference as the input.  Oh, there is a differentiate operator, nice.  Do you know what the equation would be for a 3rd order different operator?  I'm assuming it would be something like:

y = 3*x - x[n-1] - x[n-2] - x[n-3]

awchisholm

It's over to the code to answer that one - all the more reason to use something like R to create your own in an interpreted environment without the burden of having to learn Java and install a development environment.

Andrew

Ghostrider

True...although I already have a great development environment (Eclipse) and Java already is my favorite programming language...don't know R yet, but hope to learn soon! :-)