The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Dynamic modelling in Twitter

aileenzhouaileenzhou Member Posts: 12 Contributor II
in Help
How to capture behavior change over time in Twitter, for example, perception towards vaccination during and post pandemic.
Hello, has any one done dynamic topic modelling that can reflect topic change over a certain period? Thank you. 
Tagged:

Answers

  • rfuentealbarfuentealba RapidMiner Certified Analyst, Member, University Professor Posts: 568 Unicorn
    Hello @aileenzhou,

    That's work for NLP, and RapidMiner does that very well (you could also use Python NLTK together with RapidMiner). However, extracting data from Twitter is proven to be complicated because they put limits on how many tweets you get (max. 3200 per profile). If you can solve that, the rest is matter of: sorting information, tokenizing, getting parts of speech, lemmatization, stop words...

    If you are interested in the second part, ping us; the other one, I am pretty sure someone here did such a task but I don't remember who.

    All the best,

    Rod.
  • aileenzhouaileenzhou Member Posts: 12 Contributor II
    Thank you, Rod. To get Twitter sorted as suggested, ie, tokenising, lemmatisation, stop words ... is part of a standardised process. However, how to extract topic and the topic evolution over the time is where I stuck.  
  • rfuentealbarfuentealba RapidMiner Certified Analyst, Member, University Professor Posts: 568 Unicorn
    Hello @aileenzhou

    There are two operators that can help you:
    • Extract topics from documents
    • Extract topics from data
    Those are the same, the input varies. Have you tried these already?
  • aileenzhouaileenzhou Member Posts: 12 Contributor II
    Thanks a lot. It has been awhile since I tried the database last time. I will try again and let you know later. 
  • rfuentealbarfuentealba RapidMiner Certified Analyst, Member, University Professor Posts: 568 Unicorn
    Sure! If you can share a sample process for us to see what you're doing, that would be marvelous!
Sign In or Register to comment.