The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
"Historical Twitter data"
Learner III
Hi all
Relatively new to rapidminer but looking at it as a useful analysis tool for some research I am doing with colleagues. In particular we are looking to use historical Twitter data but the biggest stumbling block seems to be how to import the data into Rapidminer in an efficient manner - our current 'manual' system of creating an excel sheet with the data separated into categories (ID, date, location, retweets, tweet text etc.) is somewhat labour intensive.
I have looked online but not found anything that would work for us, yet! For example, http://vancouverdata.blogspot.co.uk/2010/11/text-analytics-with-rapidminer-loading.html - a brilliant set of tutorials but hasn't greatly helped in getting what we need at the start. ???
I have had a look at the Twitter connector - fantastic and the data is ready to go - but I'm not sure it works for historical data. We are looking at tweets from around 18 months ago and when I limit the results by date nothing comes back. Does anyone know if the Twitter connector would work here, and if so how? If not, can anyone point me in the right direction of how best to import historical Twitter data into rapid miner.
Thanks
Relatively new to rapidminer but looking at it as a useful analysis tool for some research I am doing with colleagues. In particular we are looking to use historical Twitter data but the biggest stumbling block seems to be how to import the data into Rapidminer in an efficient manner - our current 'manual' system of creating an excel sheet with the data separated into categories (ID, date, location, retweets, tweet text etc.) is somewhat labour intensive.
I have looked online but not found anything that would work for us, yet! For example, http://vancouverdata.blogspot.co.uk/2010/11/text-analytics-with-rapidminer-loading.html - a brilliant set of tutorials but hasn't greatly helped in getting what we need at the start. ???
I have had a look at the Twitter connector - fantastic and the data is ready to go - but I'm not sure it works for historical data. We are looking at tweets from around 18 months ago and when I limit the results by date nothing comes back. Does anyone know if the Twitter connector would work here, and if so how? If not, can anyone point me in the right direction of how best to import historical Twitter data into rapid miner.
Thanks
Tagged:
0
Answers
i think twitter limites the api for 2 weeks back or something. There is nothing RM can do about this.
Cheers,
Martin
Dortmund, Germany
I don't know their pricing, but you can look at some of the partners plugged into their feed to see if any of those have RapidMiner connectors: Splunk for example.
https://www.gnip.com/partners/plugged-in/ (Although, not all these partners get as much as others, Alteryx only gets the last 30 days of Tweets through their Gnip connector). Brandwatch was the one I used to use.
Another potential option is free API tools like Snapbird https://github.com/remy/snapbird who claim to circumvent limits with their search API. I haven't tested I'm afraid.