Yahoo Historical Stock Price Operator is not returing data

pradeep

Yahoo Historical Data is returning empty values in rapidminer, is there any bug ?

I was able to download rawdat from yahoo finance website, but this operator is unable to do it.

Thomas_Ott

Hi Pardeep. 

 

Just do a search for how to do it on the Community. Here's a great response from Ingo: http://community.rapidminer.com/t5/RapidMiner-Studio-Forum/Import-XML-code-to-process/m-p/32606

qwertz2

 

Dear all,

 

In the current version you have to click on menu view -> show panel -> XML. Then you can paste the XML code in the new XML windows and confirm with the green checkmark.

 

Cheers

Sachs

keith_drake

Yes, I noticed the same thing. And another app I use to download data from Yahoo (YLoader) won't work either -- it throws a "HTTP response error 301"

Anyone else?

keith_drake

Yes, I noticed the same thing. And another app I use to download data from Yahoo (YLoader) won't work either -- it throws a "HTTP response error 301"

Anyone else?

Vayao

I'm suffering this malfunction also. Please can someone help?

Thanks a lot

 

keith_drake

I've confirmed with another software vendor the problem is with Yahoo changing their protocol: "The main fix in version 4.10 is for error 301 when downloading Yahoo data, which was due to a change in the Yahoo protocol."

The Yahoo Historical Stock Data operator is in the Finance and Economic Data extension provided by BroadReachAnalytics. Visiting their URL provided in the RM documentation (http://www.broadreachanalytics.com/) it appears they are no longer active.

RapidMiner: Is there any opportunity to update this operator? Thanks.

gglsbmt

Hello my friend thankyou so much

Thomas_Ott

Yes, I can confirm that this is no longer working and BroadReachAnalytics appears to be gone.

 

With RapidMiner, you can always find a work around and here it is. 

 

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File" width="90" x="45" y="34">
        <parameter key="resource_type" value="URL"/>
        <parameter key="url" value="http://www.google.com/finance/historical?q=NASDAQ:GOOGL&amp;amp;ei=u5z8WJGtC8ejeeqWgbgJ&amp;amp;output=csv"/>
      </operator>
      <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV" width="90" x="179" y="34">
        <parameter key="column_separators" value=","/>
        <list key="annotations"/>
        <list key="data_set_meta_data_information"/>
      </operator>
      <connect from_op="Open File" from_port="file" to_op="Read CSV" to_port="file"/>
      <connect from_op="Read CSV" from_port="output" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

sgenzer

Hi...I have moved over to "openexchangerates.com" which has a nice API. I built a simple app for an accounting client using the API on RM server. Send me a PM if you want to see it.

Scott

qwertz2

 

Hi Thomas,

 

Thanks for that work around. I can confirm that it works pretty well with a single symbol. However, is there also a way to load multiple symbols at a time?

 

From your tutorials I know that it is possible to append several single symbols but in this case I want to download about 20. Searching the internet I found a site (LINK) which describes a select query but I couldn't manage to get this working in Rapidminer...

 

 

Kind regards

Sachs

pradeep

Hello Sachs,

 

 

I am new for Rapidminer,

could you please help me out in using 'xml code' for Yahoo Historical Stock price operator, which has been provided Thomas,

using it in Rapidminer?

 

Thanks

pradeep

Hello Thomas,

 

 

Thank you for providing the work around solution,

 

 

I am new for Rapidminer,

could you please help me out in using 'xml code' for Yahoo Historical Stock price operator, using it in Rapidminer?

 

Thanks

Pradeep.

keith_drake

Hello Thomas ... thanks for the solution. How do I implement the XML script? I copied it into the Execute Script operator and got an error (see attached). Thanks.

Thomas_Ott

Hi Sachs,

 

Piece of cake. Just make a text file with the header Ticker and then a new symbol on each line. Then use this process below. You should save them seperately in a repository by appending %{symbol}_stock

 

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File" width="90" x="45" y="34">
        <parameter key="filename" value="C:\Users\ThomasOtt\Desktop\Ticker File.txt"/>
      </operator>
      <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV" width="90" x="179" y="34">
        <list key="annotations"/>
        <list key="data_set_meta_data_information"/>
      </operator>
      <operator activated="true" class="extract_macro" compatibility="7.4.000" expanded="true" height="68" name="Extract Macro (2)" width="90" x="313" y="34">
        <parameter key="macro" value="num"/>
        <list key="additional_macros"/>
      </operator>
      <operator activated="true" class="concurrency:loop" compatibility="7.4.000" expanded="true" height="82" name="Loop" width="90" x="514" y="34">
        <parameter key="number_of_iterations" value="%{num}"/>
        <process expanded="true">
          <operator activated="true" class="extract_macro" compatibility="7.4.000" expanded="true" height="68" name="Extract Macro" width="90" x="112" y="34">
            <parameter key="macro" value="symbol"/>
            <parameter key="macro_type" value="data_value"/>
            <parameter key="attribute_name" value="Ticker"/>
            <parameter key="example_index" value="%{iteration}"/>
            <list key="additional_macros"/>
          </operator>
          <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File (2)" width="90" x="112" y="136">
            <parameter key="resource_type" value="URL"/>
            <parameter key="filename" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
            <parameter key="url" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
          </operator>
          <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV (2)" width="90" x="447" y="85">
            <parameter key="csv_file" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
            <parameter key="column_separators" value=","/>
            <list key="annotations"/>
            <list key="data_set_meta_data_information"/>
          </operator>
          <connect from_port="input 1" to_op="Extract Macro" to_port="example set"/>
          <connect from_op="Open File (2)" from_port="file" to_op="Read CSV (2)" to_port="file"/>
          <connect from_op="Read CSV (2)" from_port="output" to_port="output 1"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="source_input 2" spacing="0"/>
          <portSpacing port="sink_output 1" spacing="0"/>
          <portSpacing port="sink_output 2" spacing="0"/>
        </process>
      </operator>
      <connect from_op="Open File" from_port="file" to_op="Read CSV" to_port="file"/>
      <connect from_op="Read CSV" from_port="output" to_op="Extract Macro (2)" to_port="example set"/>
      <connect from_op="Extract Macro (2)" from_port="example set" to_op="Loop" to_port="input 1"/>
      <connect from_op="Loop" from_port="output 1" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

pradeep

Thank you Sachs.

pradeep

Thank you @Thomas_Ott.

qwertz2

 

Hi Thomas,

 

This already looks very promising - and brings the advantage to control stock symbols in separate external file. Pretty cool!

 

Now I am struggling with two more problems:

1) The process provides a collection but I rather want to have each symbol in its own column in a single example set / table.

2) I tried to access the yahoo web api with the open file operator:

http://ichart.finance.yahoo.com/table.csv?s=AAPL&d=4&e=10&f=2016&g=d&a=11&b=12&c=2015&ignore=table.csv

But I couldn't manage to make the yahoo links work though they would do fine a web browser.

 

Lookling forward to getting some advice...

 

 

Cheers

Sachs

 

 

Thomas_Ott

Yahoo did something to bork the whole thing recently, so I would default to Google right now. 

With respect to putting it all in a table, I'd have to think about it but it would include using a Rename by Replacing so you can do AAPL_Close and XOM_Close and then something to merge them all together, maybe with a combination of a Remember and Recall operator but I need go AFK know.