The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Web mining, get numerical data through going through provided calendar

felix_wfelix_w Member Posts: 61 Contributor II
edited September 2019 in Help

Dear Rapidminer Community, 

 

only recently I discovered that RM also offers webmining operators which is of great help for me! 

But unfortunately I encountered a problem when I tried to download data from this webpage 

https://www.eex.com/en/market-data/power/spot-market/auction#!/2018/01/03

I would like to download all hourly data 00-01 etc for a specific time (e.g. 01.01.2015 - 30.09.2017) but I can only do this through manually changing the dates day per day in the provided calendar on the website, which is not very handy. 

 

My question now, is there any way in RM to configure the Webcrawler to go through the calender automatically and draw the hourly data?

 

I hope you can help me with my question. 

 

Best regards

Felix

 

Tagged:

Answers

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    This might be tricky indeed. I don't think the webcrawler from RapidMiner can do this and you might have to resort to some outside script to do it (like node.js).  You can use the Execute Program operator to call the script, change the date, load the new page and then run the RM crawler. 

  • felix_wfelix_w Member Posts: 61 Contributor II

    Thank you for your reply! I thought already that this could be slightly tricky ;)

Sign In or Register to comment.