The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Custom polarity options in Aylien sentiment analysis

m_okem_oke Member Posts: 11 Contributor I
edited November 2018 in Help

Is it possible to have custom polarity options in Aylien/RapidMiner's sentiment analysis, beyond the usual "positive, neutral, negative"?

 

Custom examples will be options like happy, sad, mild feeling etc...

Tagged:

Best Answer

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    Solution Accepted

    outside of RapidMiner?  Why would I do that?  :)

     

    Here you go:

     

    <?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
     <context>
     <input/>
     <output/>
     <macros/>
     </context>
     <operator activated="true" class="process" compatibility="7.6.001" expanded="true" name="Process">
     <process expanded="true">
     <operator activated="true" breakpoints="after" class="text:read_document" compatibility="7.5.000" expanded="true" height="68" name="Read Document" width="90" x="45" y="34">
     <parameter key="file" value="/Users/genzerconsulting/Desktop/Scott's comments from 2006.txt"/>
     <description align="center" color="transparent" colored="false" width="126">get Scott's Comments from 2006 (Extraction)</description>
     </operator>
     <operator activated="true" breakpoints="after" class="subprocess" compatibility="7.6.001" expanded="true" height="82" name="Subprocess" width="90" x="179" y="34">
     <process expanded="true">
     <operator activated="true" class="text:documents_to_data" compatibility="7.5.000" expanded="true" height="82" name="Documents to Data" width="90" x="45" y="34">
     <parameter key="text_attribute" value="comment"/>
     <parameter key="add_meta_information" value="false"/>
     </operator>
     <operator activated="true" class="split" compatibility="7.6.001" expanded="true" height="82" name="Split" width="90" x="179" y="34">
     <parameter key="split_pattern" value="\n"/>
     </operator>
     <operator activated="true" class="transpose" compatibility="7.6.001" expanded="true" height="82" name="Transpose" width="90" x="313" y="34"/>
     <operator activated="true" class="rename" compatibility="7.6.001" expanded="true" height="82" name="Rename" width="90" x="447" y="34">
     <parameter key="old_name" value="att_1"/>
     <parameter key="new_name" value="comment"/>
     <list key="rename_additional_attributes"/>
     </operator>
     <operator activated="true" class="trim" compatibility="7.6.001" expanded="true" height="82" name="Trim" width="90" x="581" y="34"/>
     <operator activated="true" class="filter_examples" compatibility="7.6.001" expanded="true" height="103" name="Filter Examples" width="90" x="715" y="34">
     <parameter key="parameter_expression" value="matches(prefix(comment,1),&quot;[A-Z]&quot;)"/>
     <list key="filters_list">
     <parameter key="filters_entry_key" value="comment.is_not_missing."/>
     </list>
     </operator>
     <operator activated="true" class="subprocess" compatibility="7.6.001" expanded="true" height="82" name="Subprocess (2)" width="90" x="849" y="34">
     <process expanded="true">
     <operator activated="true" class="generate_id" compatibility="7.6.001" expanded="true" height="82" name="Generate ID" width="90" x="45" y="34">
     <parameter key="create_nominal_ids" value="true"/>
     </operator>
     <operator activated="true" class="replace" compatibility="7.6.001" expanded="true" height="82" name="Replace" width="90" x="179" y="34">
     <parameter key="attribute_filter_type" value="single"/>
     <parameter key="attribute" value="id"/>
     <parameter key="include_special_attributes" value="true"/>
     <parameter key="replace_what" value="id_"/>
     </operator>
     <connect from_port="in 1" to_op="Generate ID" to_port="example set input"/>
     <connect from_op="Generate ID" from_port="example set output" to_op="Replace" to_port="example set input"/>
     <connect from_op="Replace" from_port="example set output" to_port="out 1"/>
     <portSpacing port="source_in 1" spacing="0"/>
     <portSpacing port="source_in 2" spacing="0"/>
     <portSpacing port="sink_out 1" spacing="0"/>
     <portSpacing port="sink_out 2" spacing="0"/>
     </process>
     <description align="center" color="transparent" colored="false" width="126">create IDs</description>
     </operator>
     <connect from_port="in 1" to_op="Documents to Data" to_port="documents 1"/>
     <connect from_op="Documents to Data" from_port="example set" to_op="Split" to_port="example set input"/>
     <connect from_op="Split" from_port="example set output" to_op="Transpose" to_port="example set input"/>
     <connect from_op="Transpose" from_port="example set output" to_op="Rename" to_port="example set input"/>
     <connect from_op="Rename" from_port="example set output" to_op="Trim" to_port="example set input"/>
     <connect from_op="Trim" from_port="example set output" to_op="Filter Examples" to_port="example set input"/>
     <connect from_op="Filter Examples" from_port="example set output" to_op="Subprocess (2)" to_port="in 1"/>
     <connect from_op="Subprocess (2)" from_port="out 1" to_port="out 1"/>
     <portSpacing port="source_in 1" spacing="0"/>
     <portSpacing port="source_in 2" spacing="0"/>
     <portSpacing port="sink_out 1" spacing="0"/>
     <portSpacing port="sink_out 2" spacing="0"/>
     </process>
     <description align="center" color="transparent" colored="false" width="126">clean them up (Transfer and Load)</description>
     </operator>
     <operator activated="true" class="filter_example_range" compatibility="7.6.001" expanded="true" height="82" name="Filter Example Range" width="90" x="313" y="34">
     <parameter key="first_example" value="1"/>
     <parameter key="last_example" value="3"/>
     </operator>
     <operator activated="true" breakpoints="after" class="subprocess" compatibility="7.6.001" expanded="true" height="82" name="Subprocess (8)" width="90" x="447" y="34">
     <process expanded="true">
     <operator activated="true" class="web:enrich_data_by_webservice" compatibility="7.3.000" expanded="true" height="68" name="Enrich Data by Webservice (9)" width="90" x="179" y="34">
     <parameter key="query_type" value="Indexed"/>
     <list key="string_machting_queries"/>
     <list key="regular_expression_queries">
     <parameter key="result" value=".*"/>
     </list>
     <list key="regular_region_queries">
     <parameter key="foo" value="document_tone.[-}][-}]"/>
     </list>
     <list key="xpath_queries"/>
     <list key="namespaces"/>
     <parameter key="assume_html" value="false"/>
     <list key="index_queries">
     <parameter key="result" value="0.999999"/>
     </list>
     <list key="jsonpath_queries">
     <parameter key="AngerScore" value="$.tones[?(@.tone_id=='anger')]"/>
     </list>
     <parameter key="request_method" value="POST"/>
     <parameter key="service_method" value="foo"/>
     <parameter key="body" value="body=&lt;%comment%&gt;"/>
     <parameter key="url" value="https://gateway.watsonplatform.net/tone-analyzer/api/v3/tone?version=2016-05-19"/>
     <list key="request_properties">
     <parameter key="username" value="username"/>
     <parameter key="password" value="password"/>
     <parameter key="Content-Type" value="text/plain"/>
     <parameter key="sentences" value="false"/>
     </list>
     <description align="center" color="transparent" colored="false" width="126">tone with POST</description>
     </operator>
     <operator activated="true" class="generate_attributes" compatibility="7.6.001" expanded="true" height="82" name="Generate Attributes (10)" width="90" x="313" y="34">
     <list key="function_descriptions">
     <parameter key="result" value="concat(result,&quot;}&quot;)"/>
     </list>
     <description align="center" color="transparent" colored="false" width="126">add stupid } to the end of the extract</description>
     </operator>
     <operator activated="true" class="subprocess" compatibility="7.6.001" expanded="true" height="82" name="Subprocess (12)" width="90" x="447" y="34">
     <process expanded="true">
     <operator activated="true" class="multiply" compatibility="7.6.001" expanded="true" height="103" name="Multiply (2)" width="90" x="45" y="34"/>
     <operator activated="true" class="loop_examples" compatibility="7.6.001" expanded="true" height="103" name="Loop Examples (4)" width="90" x="313" y="34">
     <process expanded="true">
     <operator activated="true" class="filter_example_range" compatibility="7.6.001" expanded="true" height="82" name="Filter Example Range (2)" width="90" x="45" y="34">
     <parameter key="first_example" value="%{example}"/>
     <parameter key="last_example" value="%{example}"/>
     </operator>
     <operator activated="true" class="text:data_to_documents" compatibility="7.5.000" expanded="true" height="68" name="Data to Documents (3)" width="90" x="380" y="34">
     <parameter key="select_attributes_and_weights" value="true"/>
     <list key="specify_weights">
     <parameter key="result" value="1.0"/>
     </list>
     </operator>
     <operator activated="true" class="text:combine_documents" compatibility="7.5.000" expanded="true" height="82" name="Combine Documents (3)" width="90" x="514" y="34"/>
     <operator activated="true" class="text:extract_information" compatibility="7.5.000" expanded="true" height="68" name="Extract Information (4)" width="90" x="648" y="34">
     <parameter key="query_type" value="JsonPath"/>
     <list key="string_machting_queries"/>
     <list key="regular_expression_queries"/>
     <list key="regular_region_queries"/>
     <list key="xpath_queries"/>
     <list key="namespaces"/>
     <list key="index_queries"/>
     <list key="jsonpath_queries">
     <parameter key="AngerScore" value="$.document_tone.tone_categories[0].tones[0].*"/>
     <parameter key="DisgustScore" value="$.document_tone.tone_categories[0].tones[1].*"/>
     <parameter key="FearScore" value="$.document_tone.tone_categories[0].tones[2].*"/>
     <parameter key="JoyScore" value="$.document_tone.tone_categories[0].tones[3].*"/>
     <parameter key="SadnessScore" value="$.document_tone.tone_categories[0].tones[4].*"/>
     <parameter key="AnalyticScore" value="$.document_tone.tone_categories[1].tones[0].*"/>
     <parameter key="ConfidenceScore" value="$.document_tone.tone_categories[1].tones[1].*"/>
     <parameter key="TentativeScore" value="$.document_tone.tone_categories[1].tones[2].*"/>
     <parameter key="OpennessScore" value="$.document_tone.tone_categories[2].tones[0].*"/>
     <parameter key="ConscientiousnessScore" value="$.document_tone.tone_categories[2].tones[1].*"/>
     <parameter key="ExtraversionScore" value="$.document_tone.tone_categories[2].tones[2].*"/>
     <parameter key="AgreeablenessScore" value="$.document_tone.tone_categories[2].tones[3].*"/>
     <parameter key="EmotionalRangeScore" value="$.document_tone.tone_categories[2].tones[4].*"/>
     </list>
     </operator>
     <operator activated="true" class="text:documents_to_data" compatibility="7.5.000" expanded="true" height="82" name="Documents to Data (4)" width="90" x="782" y="34">
     <parameter key="text_attribute" value="foo2"/>
     </operator>
     <operator activated="true" class="select_attributes" compatibility="7.6.001" expanded="true" height="82" name="Select Attributes (4)" width="90" x="916" y="34">
     <parameter key="attribute_filter_type" value="single"/>
     <parameter key="attribute" value="foo2"/>
     <parameter key="invert_selection" value="true"/>
     </operator>
     <connect from_port="example set" to_op="Filter Example Range (2)" to_port="example set input"/>
     <connect from_op="Filter Example Range (2)" from_port="example set output" to_op="Data to Documents (3)" to_port="example set"/>
     <connect from_op="Data to Documents (3)" from_port="documents" to_op="Combine Documents (3)" to_port="documents 1"/>
     <connect from_op="Combine Documents (3)" from_port="document" to_op="Extract Information (4)" to_port="document"/>
     <connect from_op="Extract Information (4)" from_port="document" to_op="Documents to Data (4)" to_port="documents 1"/>
     <connect from_op="Documents to Data (4)" from_port="example set" to_op="Select Attributes (4)" to_port="example set input"/>
     <connect from_op="Select Attributes (4)" from_port="example set output" to_port="output 1"/>
     <portSpacing port="source_example set" spacing="0"/>
     <portSpacing port="sink_example set" spacing="0"/>
     <portSpacing port="sink_output 1" spacing="0"/>
     <portSpacing port="sink_output 2" spacing="0"/>
     </process>
     </operator>
     <operator activated="true" class="append" compatibility="7.6.001" expanded="true" height="82" name="Append (4)" width="90" x="447" y="34"/>
     <operator activated="true" class="generate_id" compatibility="7.6.001" expanded="true" height="82" name="Generate ID (4)" width="90" x="313" y="238"/>
     <operator activated="true" class="generate_id" compatibility="7.6.001" expanded="true" height="82" name="Generate ID (5)" width="90" x="581" y="34"/>
     <operator activated="true" class="join" compatibility="7.6.001" expanded="true" height="82" name="Join (3)" width="90" x="782" y="187">
     <list key="key_attributes"/>
     </operator>
     <connect from_port="in 1" to_op="Multiply (2)" to_port="input"/>
     <connect from_op="Multiply (2)" from_port="output 1" to_op="Loop Examples (4)" to_port="example set"/>
     <connect from_op="Multiply (2)" from_port="output 2" to_op="Generate ID (4)" to_port="example set input"/>
     <connect from_op="Loop Examples (4)" from_port="output 1" to_op="Append (4)" to_port="example set 1"/>
     <connect from_op="Append (4)" from_port="merged set" to_op="Generate ID (5)" to_port="example set input"/>
     <connect from_op="Generate ID (4)" from_port="example set output" to_op="Join (3)" to_port="left"/>
     <connect from_op="Generate ID (5)" from_port="example set output" to_op="Join (3)" to_port="right"/>
     <connect from_op="Join (3)" from_port="join" to_port="out 1"/>
     <portSpacing port="source_in 1" spacing="0"/>
     <portSpacing port="source_in 2" spacing="0"/>
     <portSpacing port="sink_out 1" spacing="0"/>
     <portSpacing port="sink_out 2" spacing="0"/>
     </process>
     <description align="center" color="transparent" colored="false" width="126">process Json data</description>
     </operator>
     <operator activated="true" class="subprocess" compatibility="7.6.001" expanded="true" height="82" name="Subprocess (13)" width="90" x="581" y="34">
     <process expanded="true">
     <operator activated="true" class="parse_numbers" compatibility="7.6.001" expanded="true" height="82" name="Parse Numbers (4)" width="90" x="45" y="34">
     <parameter key="attribute_filter_type" value="regular_expression"/>
     <parameter key="regular_expression" value=".*Score"/>
     <description align="center" color="transparent" colored="false" width="126">convert Scores to numerical</description>
     </operator>
     <operator activated="true" class="select_attributes" compatibility="7.6.001" expanded="true" height="82" name="Select Attributes (7)" width="90" x="246" y="34">
     <parameter key="attribute_filter_type" value="subset"/>
     <parameter key="attribute" value="id"/>
     <parameter key="attributes" value="result|id"/>
     <parameter key="invert_selection" value="true"/>
     <parameter key="include_special_attributes" value="true"/>
     </operator>
     <connect from_port="in 1" to_op="Parse Numbers (4)" to_port="example set input"/>
     <connect from_op="Parse Numbers (4)" from_port="example set output" to_op="Select Attributes (7)" to_port="example set input"/>
     <connect from_op="Select Attributes (7)" from_port="example set output" to_port="out 1"/>
     <portSpacing port="source_in 1" spacing="0"/>
     <portSpacing port="source_in 2" spacing="0"/>
     <portSpacing port="sink_out 1" spacing="0"/>
     <portSpacing port="sink_out 2" spacing="0"/>
     </process>
     <description align="center" color="transparent" colored="false" width="126">clean up</description>
     </operator>
     <connect from_port="in 1" to_op="Enrich Data by Webservice (9)" to_port="Example Set"/>
     <connect from_op="Enrich Data by Webservice (9)" from_port="ExampleSet" to_op="Generate Attributes (10)" to_port="example set input"/>
     <connect from_op="Generate Attributes (10)" from_port="example set output" to_op="Subprocess (12)" to_port="in 1"/>
     <connect from_op="Subprocess (12)" from_port="out 1" to_op="Subprocess (13)" to_port="in 1"/>
     <connect from_op="Subprocess (13)" from_port="out 1" to_port="out 1"/>
     <portSpacing port="source_in 1" spacing="0"/>
     <portSpacing port="source_in 2" spacing="0"/>
     <portSpacing port="sink_out 1" spacing="0"/>
     <portSpacing port="sink_out 2" spacing="0"/>
     </process>
     <description align="center" color="transparent" colored="false" width="126">IBM Watson - get comment &amp;quot;tone&amp;quot; scores</description>
     </operator>
     <operator activated="true" class="order_attributes" compatibility="7.6.001" expanded="true" height="82" name="Reorder Attributes" width="90" x="581" y="34">
     <parameter key="attribute_ordering" value="comment"/>
     </operator>
     <connect from_op="Read Document" from_port="output" to_op="Subprocess" to_port="in 1"/>
     <connect from_op="Subprocess" from_port="out 1" to_op="Filter Example Range" to_port="example set input"/>
     <connect from_op="Filter Example Range" from_port="example set output" to_op="Subprocess (8)" to_port="in 1"/>
     <connect from_op="Subprocess (8)" from_port="out 1" to_op="Reorder Attributes" to_port="example set input"/>
     <connect from_op="Reorder Attributes" from_port="example set output" to_port="result 1"/>
     <portSpacing port="source_input 1" spacing="0"/>
     <portSpacing port="sink_result 1" spacing="0"/>
     <portSpacing port="sink_result 2" spacing="0"/>
     </process>
     </operator>
    </process>

Answers

  • Telcontar120Telcontar120 RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    I doubt it, as this is a third party tool designed for use for lots of different people with different use cases.

    You could either remap the existing polarities returned, or contact Aylien about developing a custom version for you.  Either that, or try building your own sentiment models in RapidMiner.

     

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    Hi @m_oke - I would look into IBM Watson's Tone Analyzer API.  I use it all the time and will give you what you are looking for.


    Scott

  • Telcontar120Telcontar120 RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    Nice, @sgenzer --do you have a sample RM process for that?  Or are you using it outside of RM?

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • Telcontar120Telcontar120 RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    Nice, I will check it out!

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • m_okem_oke Member Posts: 11 Contributor I

    I will definitely check this out and feedback, thank you @sgenzer

  • m_okem_oke Member Posts: 11 Contributor I

    @sgenzer I see that the document you used for the project is on your desktop.

    If there are no privacy issues, do you mind making the document publicly avaliable?

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    @m_oke haha.  There are no privacy issues as long as you don't kill yourself laughing too much.  The doc is a anonymously-cleansed version of report card comments I gave my students back in 2006.  :)  They are attached.

     

    Scott

     

Sign In or Register to comment.