The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Problem with text preprocessing

ghamgham Member Posts: 24 Learner III
edited December 2018 in Help

hi

How do I use the stop word dictionary to abbreviate words and replace them with the original words and delete negative words like not?
Where do I download the stopword dictinary?
Thankful

Tagged:

Answers

  • ghamgham Member Posts: 24 Learner III

    help me...:smileysad:

  • kypexinkypexin RapidMiner Certified Analyst, Member Posts: 291 Unicorn

    Hi @gham

     

    I suggest you to look at 'Text processing' extension for RapidMiner which contains lots of very useful operators for working with texts; among them, there's a group of operators named 'Filter stopwords' which are doing the exact thing you need, for different languages. 

     

    As for downloading the dictionary, I guess there's no 'unified' dictionary as every text processing task would basically need different stoword list, so this part need to be completed manually. Maybe you can just google and download any ready dictionaries / lists someone has built before and shared online.

    -- 
    Vladimir
    http://whatthefraud.wtf
  • Telcontar120Telcontar120 RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    If you are using one of the supported languages, there are operators with built-in dictionaries (e.g., English, German, etc.) , so you don't need to download them.  If you are using another language, or if you want a custom stopword dictionary, you will need to create it yourself, or find one on the web and download it.  That's not functionality which is handled inside RapidMiner, but a little bit of web searching will turn up many useful references.

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    hello @gham Some quick recommendations for you:
    • Post your XML process here in this thread (see https://youtu.be/KkgB5QXWXJ8 and "Read Before Posting" on right when you reply)
    • Attach your dataset if possible (use a fictionalized version if there are privacy concerns)
    • Make sure you have all necessary extensions installed (see https://youtu.be/pjBqG3xtXx4)

    Scott

  • ghamgham Member Posts: 24 Learner III

    Please help my friend help me out.

     

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    @gham did you post your process like what @sgenzer said? You have to do some work here, help us troubleshoot were you're getting stuck. We're not going to buildyour process for you. 

  • ghamgham Member Posts: 24 Learner III

    Hello
    How do I remove the negation?

    help me please

    12.JPG

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    I'm sorry - I have no idea what "remove the negation" means.


    Scott

     

Sign In or Register to comment.