The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Log generation in Rapidminer Studio

ZahraSadeghiboga92ZahraSadeghiboga92 Member Posts: 6 Learner I
in Help
I have seen the execution log generated by AI Hub which reports individual steps of execution of a process.

INFO: Started operator: Process
Jan 13, 2022 7:35:55 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processStartedOperator
INFO: Started operator: Loop Files
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: Start: importing event log
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.extract.ExtractXLogOperator read
INFO: End: importing event log (0 sec)
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.logmanipulation.AddClassifierOperator doWork
INFO: Start: add classifier
Jan 13, 2022 7:35:55 PM org.rapidprom.operators.logmanipulation.AddClassifierOperator doWork

However, I need more verbose logging which can be used to chain together the sequence of events that is taking place in a complex process.

At this point, the only solution seems to be modifying Rapidminer Studio source code (github.com/rapidminer/rapidminer-studio) to emit such a process log. Can you please point out to me wherein the source code does it make sense to perform this?

Thanks in advance,

Answers

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist
    Hi,
    not sure if it helps, but the main process as a parameter called log verbosity. Maybe you just want to increase it?

    BR,
    Martin
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist
    Oh, and you can write into this log using the Write to Console operator.
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • ZahraSadeghiboga92ZahraSadeghiboga92 Member Posts: 6 Learner I
    edited January 2022
    Hello,

    I think that log level parameter only affects the operational log of the RapidMiner and not the execution log of the processes.

    To clarify, I should add, that the only information about an operator in the execution log is the start and end time. But there is no information that can help us chain together a sequence of related events when there is a parallel Loop Files operator in a process.

    For instance, the log will have several starts and end times for executing an operator without knowing which one is for which iteration of the loop.

    Inside "loop files"

    Moreover, I want to log the parameters that are passed to each operator.
    I have attached a sample of an existing execution log, and also what it should ideally look like:

    Actual execution log:
    INFO: Started operator : Process
    Jan 15, 2022 4:51:04 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processStartedOperator
    INFO: Started operator : Loop Files
    Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: Start: importing event log
    Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: Start: importing event log
    Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: Start: importing event log
    Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: End: importing event log (0 sec)
    Jan 15, 2022 4:51:04 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: End: importing event log (0 sec)
    Jan 15, 2022 4:51:05 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: End: importing event log (0 sec)
    Jan 15, 2022 4:51:05 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishdOperator
    INFO: Finished operator : Loop Files
    Jan 15, 2022 4:51:05 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishedOperator
    INFO: Finished operator : Process

    Expected execution log:
    INFO: Started operator : Process
    Jan 15, 2022 5:18:35 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processStartedOperator
    INFO: Started operator : Loop Files
    Jan 15, 2022 5:18:35 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: Start: importing event log second_file.xes
    Jan 15, 2022 5:18:35 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: Start: importing event log first_file.xes
    Jan 15, 2022 5:18:35 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: End: importing event log (0 sec) first_file.xes
    Jan 15, 2022 5:18:38 PM org.rapidprom.operators.extract.ExtractXLogOperator read
    INFO: End: importing event log (3 sec) second_file.xes
    Jan 15, 2022 5:18:38 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishedOperator
    INFO: Finished operator : Loop Files
    Jan 15, 2022 5:18:38 PM com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener processFinishedOperator
    INFO: Finished operator : Process

    My guess is that I will have to modify the source code of RapidMiner.
    Can you please point out to be the appropriate place in the source code where this can be done?


  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist
    Hi,
    why don't you use the normal Log operator for it? It can just log parameter and execution times of operators. Not sure why you  want to code something.

    Best,
    Martin
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • ZahraSadeghiboga92ZahraSadeghiboga92 Member Posts: 6 Learner I
    Hi
    I want to get execution log from any existing workflow, as is, without modifying it.

    So for example, some else might have provided a large complex workflow, and I want to get execution log from it. modifying it may not be a feasible option.

  • wadhvakartik22wadhvakartik22 Member Posts: 1 Learner I
    edited January 2022
    I think that log level parameter only affects the operational log of the RapidMiner and not the execution log of the processes.  mobdro
  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist
    @ZahraSadeghiboga92 even if we would add a log to every operator execution, you cannot just add code to the AIHub software. Those code pieces of course need to be signed by RM.
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • ZahraSadeghiboga92ZahraSadeghiboga92 Member Posts: 6 Learner I
    edited January 2022
    Hi,

    I probably could change the source code of RapidMiner Studio to get a log like what I said, right?

    Rapidminer has a large codebase. Can you please let me know how can I get some hints about which part of the code is the best place to get started with this?

    Thanks in advance
Sign In or Register to comment.