The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Include Negations in Dictionary based Sentiment Approach

katrin_kandlbinkatrin_kandlbin Member Posts: 2 Contributor I
edited June 2019 in Help

Hi all,

 

I want to use the dictionary based sentiment approach for texts by using a self-created dictionary.

The dictionary sentiment approach does not account for negations as far as I know.

For example: Today I was not really productive.

The word "productive" is positive but because of the "not" the whole sentence has a negative meaning instead of positive.

 

How does the dictionary based sentiment approach of Rapidminer account for this issue?

 

All the best,

Katrin

Tagged:

Answers

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist

    Hi Katrin,

     

    welcome to the community! I've written the operator myself and I am very happy that you use it. The current version of the operator does not handle negations. Negations are in general a tricky thing also in more sophisticated approaches like SVMs. I am not sure how this dictionary based approach could handle this. If you have a solution in mind I am happy to have a look if we can implement this.

     

    Cheers,

    Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • DocMusherDocMusher Member Posts: 333 Unicorn

    Hi,

    Negation is in fact only a small part of the problem: imagine the sentence "probably a possible tumor" in the notes of a radiologist. 

    :)

    Sven

  • katrin_kandlbinkatrin_kandlbin Member Posts: 2 Contributor I

    Hi Martin,

    first of all: happy new year. Thank you very much for your reply.

    One possible solution to account for negations might be to include another "dictionary" of self-created word list which consists of negation words.

    This word list is applied to the same text corpus as the sentiment dictionary in the following way:

    1. all words in the text corpus should be numbered consecutively
    2. the sentiment dictionary operator recognizes that (for example) word Nr. 12 is a sentiment word and assigns the corresponding value (1 if positive and -1 if negative)
    3. the new negation operator then concentrates only on the sentiment words of the text corpus (here word Nr. 12) and examines the surrounding words - let's say 5 words before the sentiment word and 5 words after.
    4. If there is a negation word included 5 words before or after the sentiment word, the value of the sentiment word will be multiplied by (-1) in order to reverse the sentiment value.

    Do you think this might work?

     

    Best,

    Katrin

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist

    Hi @katrin_kandlbin,

    i've implemented this (without counting but same idea). I will share a preview version of this via e-mail since i need to write some docs and it needs to run through our internal code-quality process before it appears on market place.

     

    Best,

    Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
Sign In or Register to comment.