Using RapidMiner to search Twitter accounts for all tweets related to a specific topic/query.

christopher_bak

Hi there,

I have just started using RapidMiner and have a limited grasp of XML. I am trying to use RapidMiner to search a set of Twitter accounts (news outlets) for all tweets (news stories) related to a specific topic/query, I would then analyse the headlines' sentiment with Aylien.  

 

I only seem to be able to use 'Search Twitter' to pull up all tweets (recent or popular) on a topic but not from the users I want data on, or I use 'Get Twitter User Statuses' and receive superfluous information on topics I do not want. Is there any way to achieve the results I want without creating my own process (which I am unable to do) using advanced XML?

 

I am sorry if there isn't enough information there, or if I have missed something that may help anyone answering the question. Any help or advice anybody can provide would be greatly appreciated.

 

Thanks.

 

Chris

Telcontar120

Thanks for the clarification.  Just to be sure, when you say "manually" you understand that RapidMiner is still doing the work using the Filter Examples operator, where you can use the "contains" clause to find those topics you are interested in.  So no human work required other than setting up the process.  It's actually pretty efficient, just not in a single operator.

I would add the product suggestion though for the Twitter search, it sounds like something that would be helpful.

 

sgenzer

hello @christopher_bak welcome to the community. Some suggestions for you:

• I would HIGHLY recommend learning the general functionality of RapidMiner. We have a whole "RapidMiner University" here on the community which will teach you about Filter Examples and much more.
• There is almost an exact solution to your problem in the Community Samples repository that you can use any time:

 

Hope that helps.

Scott

 

Telcontar120

So you want to search for all posts on a given topic only for a given set of Twitter users?

There is no "simple" way to do this (e.g., using a single operator) but it is definitely do-able with RapidMiner.  If the user set is not that large, I'd probably start by pulling all the tweets for those users (Get User Statuses), and then simply filtering them by those that contain your hashtags of interest with the Filter Examples operator on the text field, and then Append all the results together.  If the user set is large, you might need to do this with a loop/macro setup.

 

P.S.  If you think this is a worthwhile use case you could suggest the idea to add such a query term (user id) to the Search Twitter operator in the Product Ideas board.

 

christopher_bak

Hi Brian,

 

Thanks for the reply. 

 

So you want to search for all posts on a given topic only for a given set of Twitter users?

Yes, that is essentially it. I wanted to pull all mentions of Bitcoin in 2018 by a select number of news media outlets on Twitter.

 

If the user set is not that large, I'd probably start by pulling all the tweets for those users (Get User Statuses), and then simply filtering them by those that contain your hashtags of interest with the Filter Examples operator on the text field, and then Append all the results together.

This was what I thought I might have to do; run single user searches and sort through them manually, I was just hoping someone with more advanced knowledge of the platform may know a simpler more efficient way to complete the task. 

 

If you think this is a worthwhile use case you could suggest the idea to add such a query term (user id) to the Search Twitter operator in the Product Ideas board.

If you think it would be worthwhile I would be happy to share my query with the Product Ideas Board.

 

Thanks for your advice.

 

Chris

 

christopher_bak

I didn't even realise there was a 'Filter Examples' operator. Thank you for the tip!