The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Enrich Data by Webservice debugging

sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
edited November 2018 in Help
hi...is there any way to debug the Enrich Data by Webservice operator when it fails?  I would to see the JSONpath that it is sending out in a POST request, and the actually reply from the server with the error code.  I can do it in curl but not via the RapidMiner interface.

In a similar vein, is there a way to send a curl request directly from RM?

And by the way, when it fails on a GET request, the salmon-colored error box is enormous (due to the large URL) and runs off the left side of the screen.  It is almost impossible to click the "Got it" button.

Scott
Tagged:

Answers

  • BalazsBaranyBalazsBarany Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert Posts: 955 Unicorn
    Hi,
    sgenzer wrote:

    hi...is there any way to debug the Enrich Data by Webservice operator when it fails?  I would to see the JSONpath that it is sending out in a POST request, and the actually reply from the server with the error code.  I can do it in curl but not via the RapidMiner interface.
    If your web service is accessible over HTTP, you can easily dump the entire communication with Wireshark.
    If it needs HTTPS, you could try something like Fiddler <http://www.telerik.com/fiddler>.

    In a similar vein, is there a way to send a curl request directly from RM?
    You can use the Execute Program operator.
  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    hmm thanks.  I was hoping to tap into the logging features in RM rather than something outside of it.

    ah Execute Program.  :)  I should have thought of that!  Duh. Thanks.  How do I pass the information into Execute Program so that I can use curl?  macros?

    Scott
  • JEdwardJEdward RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 578 Unicorn
    It comes with handy output ports so you can use the result in RapidMiner. 

    See this example: 
    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <process version="7.1.000-BETA">
      <context>
        <input/>
        <output/>
        <macros/>
      </context>
      <operator activated="true" class="process" compatibility="6.0.002" expanded="true" name="Process">
        <parameter key="logverbosity" value="init"/>
        <parameter key="random_seed" value="2001"/>
        <parameter key="send_mail" value="never"/>
        <parameter key="notification_email" value=""/>
        <parameter key="process_duration_for_mail" value="30"/>
        <parameter key="encoding" value="SYSTEM"/>
        <parameter key="parallelize_main_process" value="false"/>
        <process expanded="true">
          <operator activated="true" class="productivity:execute_program" compatibility="7.1.000-BETA" expanded="true" height="103" name="Execute Program" width="90" x="112" y="85">
            <parameter key="command" value="cmd /c java -version"/>
            <parameter key="log_stdout" value="true"/>
            <parameter key="log_stderr" value="true"/>
            <list key="env_variables"/>
          </operator>
          <operator activated="true" class="text:read_document" compatibility="7.0.000" expanded="true" height="68" name="Read Document (2)" width="90" x="380" y="136">
            <parameter key="extract_text_only" value="true"/>
            <parameter key="use_file_extension_as_type" value="true"/>
            <parameter key="content_type" value="txt"/>
            <parameter key="encoding" value="SYSTEM"/>
          </operator>
          <operator activated="true" class="text:read_document" compatibility="7.0.000" expanded="true" height="68" name="Read Document" width="90" x="380" y="34">
            <parameter key="extract_text_only" value="true"/>
            <parameter key="use_file_extension_as_type" value="true"/>
            <parameter key="content_type" value="txt"/>
            <parameter key="encoding" value="SYSTEM"/>
          </operator>
          <connect from_op="Execute Program" from_port="out" to_op="Read Document" to_port="file"/>
          <connect from_op="Execute Program" from_port="err" to_op="Read Document (2)" to_port="file"/>
          <connect from_op="Read Document (2)" from_port="output" to_port="result 2"/>
          <connect from_op="Read Document" from_port="output" to_port="result 1"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>
          <portSpacing port="sink_result 3" spacing="0"/>
        </process>
      </operator>
    </process>
  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    ok yes that works.  Thank you.  It's still not solving my problem.  I am running this command:

    curl -X GET --header 'Accept: text/plain' --header 'accept: text/plain' 'https://watson-api-explorer.mybluemix.net/language-translation/api/v2/identify?text=je suis faim'

    if I run it in terminal, I get the correct response ("fr"), but in "Execute Program", I get an error message and it will not run.

    Scott

  • BalazsBaranyBalazsBarany Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert Posts: 955 Unicorn
    Hi!

    You can use "Read Document" to convert the standard output and standard error of your process into readable text.

    Example process: 

    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <process version="7.0.001">
      <context>
        <input/>
        <output/>
        <macros/>
      </context>
      <operator activated="true" class="process" compatibility="7.0.001" expanded="true" name="Process">
        <process expanded="true">
          <operator activated="true" class="productivity:execute_program" compatibility="7.0.001" expanded="true" height="103" name="Execute Program" width="90" x="112" y="34">
            <parameter key="command" value="ls /tmp"/>
            <list key="env_variables"/>
          </operator>
          <operator activated="true" class="text:read_document" compatibility="6.5.000" expanded="true" height="68" name="Read stdout" width="90" x="380" y="34"/>
          <operator activated="true" class="text:read_document" compatibility="6.5.000" expanded="true" height="68" name="Read stderr" width="90" x="514" y="85"/>
          <connect from_op="Execute Program" from_port="out" to_op="Read stdout" to_port="file"/>
          <connect from_op="Execute Program" from_port="err" to_op="Read stderr" to_port="file"/>
          <connect from_op="Read stdout" from_port="output" to_port="result 1"/>
          <connect from_op="Read stderr" from_port="output" to_port="result 2"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>
          <portSpacing port="sink_result 3" spacing="0"/>
        </process>
      </operator>
    </process>
    Do you get the error message from RapidMiner or on the error output from curl? Do you need authentication for accessing the web service?

    With curl -v you can get verbose output that shows the details of the communication with the server.
  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    thanks.  No I do not get any errors when I use curl in the Unix terminal shell.  Only in "Execute Program".  In fact I cannot even get that to work when using your example, finding the java version.
Sign In or Register to comment.