Logging Webservice Response
Hallo Community,
I have a question which is related to this post:
I am running a web service which outputs rather short JSON as a response to the calls. I get that it may not make sense that calls to web services are shown in the process scheduler, as per the post above. Nonetheless, I would still like to log the response that is sent. I know how to build this within the process that is used in the webservice, using the "Extract Log Value" and "Log" operators. However, it somehow feels a bit too complicated for the mere goal of logging a short JSON response. Hence, I am suspecting that I am missing something very simple and obvious here and wanted to ask how others have approached/solved this.
Thank you very much for any input.
Best Answer
-
JEdward
RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 578 
Unicorn
If you have a list of set values that you wish to store each time then perhaps store it as a Building Block so that you can use it in different processes easily. I presume you want execution time, date, etc?
Why not add a Write Database operator into the flow of your process which executes each time the process is run?
Here's a very quick example of a building block you might use: note that I have used Generate Data by User Specification rather than Extract Log Values. You might need Extract Log Values if you are extracting specific execution times for individual operators, but for process_start & time_now then you can just use the single operator as I have done.
<?xml version="1.0" encoding="UTF-8"?><process version="7.5.003">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="7.5.003" expanded="true" name="Process">
<process expanded="true">
<operator activated="true" class="handle_exception" compatibility="7.5.003" expanded="true" height="103" name="Handle Exception" width="90" x="246" y="85">
<parameter key="exception_macro" value="myProblems"/>
<process expanded="true">
<operator activated="true" class="generate_data_user_specification" compatibility="7.5.003" expanded="true" height="68" name="MyLog" width="90" x="112" y="289">
<list key="attribute_values">
<parameter key="process_name" value="%{process_name}"/>
<parameter key="time_now" value="date_now()"/>
<parameter key="process_start" value="%{process_start}"/>
<parameter key="status" value=""Success""/>
</list>
<list key="set_additional_roles"/>
</operator>
<operator activated="true" class="jdbc_connectors:write_database" compatibility="7.5.003" expanded="true" height="68" name="WriteLog" width="90" x="246" y="289">
<parameter key="table_name" value="myWebServiceLog"/>
</operator>
<operator activated="true" class="text:data_to_json" compatibility="7.5.000" expanded="true" height="82" name="Data To JSON" width="90" x="179" y="34"/>
<operator activated="true" class="text:combine_documents" compatibility="7.5.000" expanded="true" height="82" name="Combine Documents" width="90" x="313" y="34"/>
<connect from_port="in 1" to_op="Data To JSON" to_port="example set 1"/>
<connect from_op="MyLog" from_port="output" to_op="WriteLog" to_port="input"/>
<connect from_op="WriteLog" from_port="through" to_port="out 2"/>
<connect from_op="Data To JSON" from_port="documents" to_op="Combine Documents" to_port="documents 1"/>
<connect from_op="Combine Documents" from_port="document" to_port="out 1"/>
<portSpacing port="source_in 1" spacing="0"/>
<portSpacing port="source_in 2" spacing="0"/>
<portSpacing port="sink_out 1" spacing="0"/>
<portSpacing port="sink_out 2" spacing="0"/>
<portSpacing port="sink_out 3" spacing="0"/>
</process>
<process expanded="true">
<operator activated="true" class="generate_data_user_specification" compatibility="7.5.003" expanded="true" height="68" name="MyLog (Error)" width="90" x="112" y="289">
<list key="attribute_values">
<parameter key="process_name" value="%{process_name}"/>
<parameter key="time_now" value="date_now()"/>
<parameter key="process_start" value="%{process_start}"/>
<parameter key="status" value=""Failed : %{myProblems}""/>
</list>
<list key="set_additional_roles"/>
</operator>
<operator activated="true" class="jdbc_connectors:write_database" compatibility="7.5.003" expanded="true" height="68" name="WriteLog (Error)" width="90" x="246" y="289">
<parameter key="table_name" value="myWebServiceLog"/>
</operator>
<operator activated="true" class="text:create_document" compatibility="7.5.000" expanded="true" height="68" name="Error Message to User" width="90" x="45" y="136">
<parameter key="text" value="{Soz, totally failed becoz I gots %{myProblems} }"/>
</operator>
<connect from_op="MyLog (Error)" from_port="output" to_op="WriteLog (Error)" to_port="input"/>
<connect from_op="Error Message to User" from_port="output" to_port="out 1"/>
<portSpacing port="source_in 1" spacing="0"/>
<portSpacing port="source_in 2" spacing="0"/>
<portSpacing port="sink_out 1" spacing="0"/>
<portSpacing port="sink_out 2" spacing="0"/>
<portSpacing port="sink_out 3" spacing="0"/>
</process>
<description align="center" color="transparent" colored="false" width="126">Do Error Trapping</description>
</operator>
<connect from_port="input 1" to_op="Handle Exception" to_port="in 1"/>
<connect from_op="Handle Exception" from_port="out 1" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>2
Answers
Thank you. The quick and easy way I was looking for is your suggested "Write Database" route. This does exactly what I want and is implemented with basically no effort.
No problem.
If you also define a macro in your Context Menu and set it as '_ra_user ' with the value 'StudioUser' and put this macro into your generated log with the write database method, you'll find that when running on RM Server it will replace the value with the user running the webservice.
Really handy for that extra level of security so you can check which users might be abusing the calls.
To take it even further you can monitor that database on a regular basis with another process and when it reaches a certain number of calls you can trigger another process to stop the user access rights on the RapidMiner webservice. This way you can easily setup your own backend API charging method, all within RM. (license restrictions may apply depending on usage).
Note: you can do this at the start of each webservice call, but I feel the overhead adding a ReadDB at the start of each process when the user has already passed login is unnecessary.
Not sure how to get the IP address though. Anyone have tips on that one?