[Solved]Convert Graphical Design to Commands in CLI

aryan_hosseinza

Hi everybody,

I've been working with the GUI version of rapidminer up until now , but now I have to run the things on a remote linux system which only provides CLI and not a GUI ,

I'd like to know if there's any way to convert what the design in GUI (all those operators and settings and connections and settings) to arguments which can be executed in CLI,  

(e.g. one of the possible solutions that I think is to first create the whole process in the RapidMiner GUI and then feed the rmp file into rapidminer in command line , but how can I do that and I don't know if it is possible)


Best REgards,
Arian

aryan_hosseinza

I tried : java -Xmx6G -jar rapidminer/lib/rapidminer.jar -f //NewLocalRepository/Feature_Selection but nothing resulted ! 

17-Nov-2012 7:34:22 AM com.rapidminer.tools.ParameterService init
INFO: Reading configuration resource com/rapidminer/resources/rapidminerrc.
17-Nov-2012 7:34:22 AM com.rapidminer.tools.LaunchListener installListener
INFO: Listening for other instances on port 59254. Writing /home/arian/.RapidMiner5/socket.
java com.rapidminer.gui.RapidMinerGUI [processfile]

wessel

java -cp rapidminer.jar com.rapidminer.RapidMinerCommandLine //rapidminer_repository_ALIAS/path_to_process/process

aryan_hosseinza

How Can I set the repository address ? I mean  "//rapidminer_repository_ALIAS/path_to_process/process" shouldn't be set before executing this command ?

wessel

What error are you getting when using com.rapidminer.RapidMinerCommandLine?

As far as I recall, the -f thing is for a windows script, which uses com.rapidminer.RapidMinerCommandLine internally.

Best regards,

Wessel

wessel

$JAVA \
   -Xmx${MAX_JAVA_MEMORY}m \
   -XX:MaxPermSize=128m\
-cp ${CLASSPATH} \
-Drapidminer.home=${RAPIDMINER_HOME} \
-Drapidminer.operators.additional=${RAPIDMINER_OPERATORS_ADDITIONAL} \
${RAPIDMINER_JDBC_DRIVER_PARAMETER} \
com.rapidminer.RapidMinerCommandLine $


Note that there is no "-f" here.

aryan_hosseinza

I created a folder (RMRepository2) in folder /rapidminer and I put my files there , but when I run the following command , it doesn't execute the process


Command :
user@Server:~/rapidminer$ java -Xmx2G -jar lib/rapidminer.jar com.rapidminer.RapidMinerCommandLine //RMRepository/Feature_Selection_2.rmp

Result :
Nov 17, 2012 10:19:49 AM com.rapidminer.tools.ParameterService init
INFO: Reading configuration resource com/rapidminer/resources/rapidminerrc.
Nov 17, 2012 10:19:49 AM com.rapidminer.tools.LaunchListener installListener
INFO: Listening for other instances on port 38603. Writing /home/arian/.RapidMiner5/socket.
java com.rapidminer.gui.RapidMinerGUI [processfile]

Best Regards,
Arian

wessel

Create a script that generates some data and writes this to a file.
Make sure the script doesn't load anything from the repository.
If this script does run, there is a problem with setting the correct relative path, which is always difficult.

Best regards,

Wessel

aryan_hosseinza

I tried the following code which generates a dataset but it still doesn't work !

The remote system is a linux ubuntu , I copied the rapidminer to the home/user folder and created a Repository folder in /home/user/rapidminer , and then I copied the following files in to the repository :

massive_data_generator_feature_selection_.properties
massive_data_generator_feature_selection_.rmp

then, I executed the following command :
user@server:~/rapidminer$ java -Xmx2G -jar lib/rapidminer.jar com.rapidminer.RapidMinerCommandLine //RMRepository2/massive_data_generator_feature_selection_.rmp

result:
Nov 17, 2012 7:08:47 PM com.rapidminer.tools.ParameterService init
INFO: Reading configuration resource com/rapidminer/resources/rapidminerrc.
Nov 17, 2012 7:08:47 PM com.rapidminer.tools.LaunchListener installListener
INFO: Listening for other instances on port 51991. Writing /home/arian/.RapidMiner5/socket.
java com.rapidminer.gui.RapidMinerGUI [processfile]

Do you think the problem is with the relative path ?  how can I make it run ?

Regards,
Arian

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.2.008">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.2.008" expanded="true" name="Process">
    <process expanded="true" height="494" width="1016">
      <operator activated="true" class="generate_massive_data" compatibility="5.2.008" expanded="true" height="60" name="Generate Massive Data" width="90" x="112" y="75">
        <parameter key="number_examples" value="1000"/>
        <parameter key="number_attributes" value="100"/>
        <parameter key="sparse_fraction" value="0.95"/>
      </operator>
      <operator activated="true" class="nominal_to_binominal" compatibility="5.2.008" expanded="true" height="94" name="Nominal to Binominal" width="90" x="313" y="75">
        <parameter key="attribute_filter_type" value="single"/>
        <parameter key="attribute" value="label"/>
      </operator>
      <operator activated="true" class="weka:W-ReliefFAttributeEval" compatibility="5.1.001" expanded="true" height="76" name="W-ReliefFAttributeEval" width="90" x="447" y="75">
        <parameter key="sort_direction" value="descending"/>
      </operator>
      <operator activated="true" class="select_by_weights" compatibility="5.2.008" expanded="true" height="94" name="Select by Weights" width="90" x="648" y="75">
        <parameter key="weight_relation" value="top k"/>
        <parameter key="weight" value="0.0"/>
        <parameter key="k" value="25"/>
      </operator>
      <operator activated="true" class="x_validation" compatibility="5.2.008" expanded="true" height="112" name="Validation" width="90" x="782" y="75">
        <process expanded="true" height="649" width="522">
          <operator activated="true" class="naive_bayes" compatibility="5.2.008" expanded="true" height="76" name="Naive Bayes" width="90" x="216" y="30"/>
          <connect from_port="training" to_op="Naive Bayes" to_port="training set"/>
          <connect from_op="Naive Bayes" from_port="model" to_port="model"/>
          <portSpacing port="source_training" spacing="0"/>
          <portSpacing port="sink_model" spacing="0"/>
          <portSpacing port="sink_through 1" spacing="0"/>
        </process>
        <process expanded="true" height="649" width="522">
          <operator activated="true" class="apply_model" compatibility="5.2.008" expanded="true" height="76" name="Apply Model" width="90" x="45" y="30">
            <list key="application_parameters"/>
          </operator>
          <operator activated="true" class="performance_binominal_classification" compatibility="5.2.008" expanded="true" height="76" name="Performance" width="90" x="283" y="30">
            <parameter key="accuracy" value="false"/>
            <parameter key="AUC" value="true"/>
            <parameter key="f_measure" value="true"/>
            <parameter key="false_positive" value="true"/>
            <parameter key="false_negative" value="true"/>
            <parameter key="true_positive" value="true"/>
            <parameter key="true_negative" value="true"/>
          </operator>
          <connect from_port="model" to_op="Apply Model" to_port="model"/>
          <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
          <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
          <connect from_op="Performance" from_port="performance" to_port="averagable 1"/>
          <portSpacing port="source_model" spacing="0"/>
          <portSpacing port="source_test set" spacing="0"/>
          <portSpacing port="source_through 1" spacing="0"/>
          <portSpacing port="sink_averagable 1" spacing="0"/>
          <portSpacing port="sink_averagable 2" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="write_as_text" compatibility="5.2.008" expanded="true" height="76" name="Write as Text" width="90" x="916" y="75">
        <parameter key="result_file" value="/home/arian/resultttt.res"/>
      </operator>
      <connect from_op="Generate Massive Data" from_port="output" to_op="Nominal to Binominal" to_port="example set input"/>
      <connect from_op="Nominal to Binominal" from_port="example set output" to_op="W-ReliefFAttributeEval" to_port="example set"/>
      <connect from_op="W-ReliefFAttributeEval" from_port="weights" to_op="Select by Weights" to_port="weights"/>
      <connect from_op="W-ReliefFAttributeEval" from_port="example set" to_op="Select by Weights" to_port="example set input"/>
      <connect from_op="Select by Weights" from_port="example set output" to_op="Validation" to_port="training"/>
      <connect from_op="Validation" from_port="averagable 1" to_op="Write as Text" to_port="input 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
    </process>
  </operator>
</process>

wessel

No, there is no repository stuff here, so no problems with relative path, this should run.
I will try and upload to a remote Ubuntu machine, and see if I can run it.

wessel

I can run your process.

I used:

java -cp //home/wln320/RapidMiner5/lib/rapidminer.jar com.rapidminer.RapidMinerCommandLine -f //home/wln320/myrep/csvwrite.rmp


When you want to use relative paths, you should not use "-f" and also not add ".rmp" at the end of your process file.

wessel

[wln320@hidden myrep]$ java -cp //home/wln320/RapidMiner5/lib/rapidminer.jar com.rapidminer.RapidMinerCommandLine -f //home/wln320/myrep/csvwrite.rmp
RapidMiner version 5.2.008, Copyright (C) 2001-2012
RapidMiner comes with ABSOLUTELY NO WARRANTY; This is free software,
and you are welcome to redistribute it under certain conditions;
see license information in the file named LICENSE.
Nov 18, 2012 4:41:44 PM com.rapid_i.Launcher ensureRapidMinerHomeSet
INFO: Property rapidminer.home is not set. Guessing.
Nov 18, 2012 4:41:44 PM com.rapid_i.Launcher ensureRapidMinerHomeSet
INFO: Trying parent directory of '/home/wln320/RapidMiner5/lib/rapidminer.jar'...gotcha!
Nov 18, 2012 4:41:44 PM com.rapidminer.tools.ParameterService init
INFO: Reading configuration resource com/rapidminer/resources/rapidminerrc.
Nov 18, 2012 4:41:44 PM com.rapidminer.tools.I18N <clinit>
INFO: Set locale to en.
Nov 18, 2012 4:41:44 PM com.rapid_i.Launcher ensureRapidMinerHomeSet
INFO: rapidminer.home is '/home/wln320/RapidMiner5'.
Nov 18, 2012 4:41:45 PM com.rapidminer.parameter.ParameterTypePassword decryptPassword
WARNING: Password in XML file looks like unencrypted plain text.
Nov 18, 2012 4:41:46 PM com.rapidminer.tools.jdbc.JDBCProperties <init>
WARNING: Missing database driver class name for ODBC Bridge (e.g. Access)
Nov 18, 2012 4:41:46 PM com.rapidminer.tools.jdbc.JDBCProperties registerDrivers
INFO: JDBC driver ca.ingres.jdbc.IngresDriver not found. Probably the driver is not installed.
Nov 18, 2012 4:41:46 PM com.rapidminer.tools.jdbc.JDBCProperties registerDrivers
INFO: JDBC driver oracle.jdbc.driver.OracleDriver not found. Probably the driver is not installed.
Nov 18, 2012 4:41:46 PM com.rapidminer.tools.WrapperLoggingHandler log
INFO: No filename given for result file, using stdout for logging results!
Nov 18, 2012 4:41:46 PM com.rapidminer.Process run
INFO: Process /home/wln320/myrep/csvwrite.rmp starts
Nov 18, 2012 4:41:46 PM com.rapidminer.Process run
INFO: Process /home/wln320/myrep/csvwrite.rmp finished successfully after 0 s
Nov 18, 2012 4:41:46 PM com.rapidminer.RapidMinerCommandLine run
INFO: Process finished successfully

aryan_hosseinza

Thanks Wessel for your help,

Now I can run a very simple design , but for the design that I mentioned (or even a simpler one) it returns me an error :

user@Server:~/rapidminer/RMRepository$ java -cp //home/arian/rapidminer/lib/rapidminer.jar com.rapidminer.RapidMinerCommandLine -f //home/arian/rapidminer/RMRepository/testRemote.rmp
RapidMiner version 5.2.008, Copyright (C) 2001-2012
RapidMiner comes with ABSOLUTELY NO WARRANTY; This is free software,
and you are welcome to redistribute it under certain conditions;
see license information in the file named LICENSE.
Nov 18, 2012 5:06:32 PM com.rapid_i.Launcher ensureRapidMinerHomeSet
INFO: Property rapidminer.home is not set. Guessing.
Nov 18, 2012 5:06:32 PM com.rapid_i.Launcher ensureRapidMinerHomeSet
INFO: Trying parent directory of '/home/arian/rapidminer/lib/rapidminer.jar'...gotcha!
Nov 18, 2012 5:06:32 PM com.rapidminer.tools.ParameterService init
INFO: Reading configuration resource com/rapidminer/resources/rapidminerrc.
Nov 18, 2012 5:06:32 PM com.rapidminer.tools.I18N <clinit>
INFO: Set locale to en.
Nov 18, 2012 5:06:32 PM com.rapid_i.Launcher ensureRapidMinerHomeSet
INFO: rapidminer.home is '/home/arian/rapidminer'.
Nov 18, 2012 5:06:33 PM com.rapidminer.parameter.ParameterTypePassword decryptPassword
WARNING: Password in XML file looks like unencrypted plain text.
Nov 18, 2012 5:06:33 PM com.rapidminer.tools.jdbc.JDBCProperties <init>
WARNING: Missing database driver class name for ODBC Bridge (e.g. Access)
Nov 18, 2012 5:06:33 PM com.rapidminer.tools.jdbc.JDBCProperties registerDrivers
INFO: JDBC driver ca.ingres.jdbc.IngresDriver not found. Probably the driver is not installed.
Nov 18, 2012 5:06:33 PM com.rapidminer.tools.jdbc.JDBCProperties registerDrivers
INFO: JDBC driver oracle.jdbc.driver.OracleDriver not found. Probably the driver is not installed.
Nov 18, 2012 5:06:34 PM com.rapidminer.io.process.XMLImporter addMessage
INFO: <em class="error">The operator class 'weka:W-ReliefFAttributeEval' is unknown. Possibly you must install a plugin for operators of group 'weka'.</em>
Nov 18, 2012 5:06:34 PM com.rapidminer.io.process.XMLImporter addMessage
INFO: <em class="error">The input port <var>example set</var> is unknown at operator <var>W-ReliefFAttributeEval (3)</var>.</em>
Nov 18, 2012 5:06:34 PM com.rapidminer.io.process.XMLImporter addMessage
INFO: <em class="error">The output port <var>weights</var> is unknown at operator <var>W-ReliefFAttributeEval (3)</var>.</em>
Nov 18, 2012 5:06:34 PM com.rapidminer.tools.WrapperLoggingHandler log
INFO: No filename given for result file, using stdout for logging results!
Nov 18, 2012 5:06:34 PM com.rapidminer.Process run
INFO: Process /home/arian/rapidminer/RMRepository/testRemote.rmp starts
Nov 18, 2012 5:06:34 PM com.rapidminer.RapidMinerCommandLine run
SEVERE: Process failed: The dummy operator W-ReliefFAttributeEval (3) (replacing weka:W-ReliefFAttributeEval) cannot be executed.
com.rapidminer.operator.UserError: The dummy operator W-ReliefFAttributeEval (3) (replacing weka:W-ReliefFAttributeEval) cannot be executed.
at com.rapidminer.operator.DummyOperator.doWork(DummyOperator.java:88)
at com.rapidminer.operator.Operator.execute(Operator.java:834)
at com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:51)
at com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:711)
at com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:379)
at com.rapidminer.operator.Operator.execute(Operator.java:834)
at com.rapidminer.Process.run(Process.java:925)
at com.rapidminer.Process.run(Process.java:848)
at com.rapidminer.Process.run(Process.java:807)
at com.rapidminer.Process.run(Process.java:802)
at com.rapidminer.Process.run(Process.java:792)
at com.rapidminer.RapidMinerCommandLine.run(RapidMinerCommandLine.java:145)
at com.rapidminer.RapidMinerCommandLine.main(RapidMinerCommandLine.java:182)
Nov 18, 2012 5:06:34 PM com.rapidminer.RapidMinerCommandLine run
SEVERE: Here:          Process[1] (Process)
          subprocess 'Main Process'
            +- Generate Massive Data[1] (Generate Massive Data)
            +- Nominal to Binominal[1] (Nominal to Binominal)
      ==>  +- W-ReliefFAttributeEval (3)[1] (dummy)
Nov 18, 2012 5:06:34 PM com.rapidminer.RapidMinerCommandLine run
SEVERE: Process not successful

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.2.008">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.2.008" expanded="true" name="Process">
    <process expanded="true" height="631" width="949">
      <operator activated="true" class="generate_massive_data" compatibility="5.2.008" expanded="true" height="60" name="Generate Massive Data" width="90" x="45" y="75">
        <parameter key="number_examples" value="1000"/>
        <parameter key="number_attributes" value="100"/>
        <parameter key="sparse_fraction" value="0.95"/>
      </operator>
      <operator activated="true" class="nominal_to_binominal" compatibility="5.2.008" expanded="true" height="94" name="Nominal to Binominal" width="90" x="179" y="75">
        <parameter key="attribute_filter_type" value="single"/>
        <parameter key="attribute" value="label"/>
        <parameter key="include_special_attributes" value="true"/>
      </operator>
      <operator activated="true" class="weka:W-ReliefFAttributeEval" compatibility="5.1.001" expanded="true" height="76" name="W-ReliefFAttributeEval (3)" width="90" x="313" y="75">
        <parameter key="sort_direction" value="descending"/>
        <parameter key="K" value="21.0"/>
      </operator>
      <connect from_op="Generate Massive Data" from_port="output" to_op="Nominal to Binominal" to_port="example set input"/>
      <connect from_op="Nominal to Binominal" from_port="example set output" to_op="W-ReliefFAttributeEval (3)" to_port="example set"/>
      <connect from_op="W-ReliefFAttributeEval (3)" from_port="weights" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

It seems that the W-ReliefFAttributeEval doesn't work on the remote machine but it works on my own local machine, what could be the reason  ?


Thanks ,

Arian

aryan_hosseinza

And when I want to store a result in the repository  it gives me the following error :
...
...
SEVERE: Process failed: Cannot store data in repository at entry '//RMRepository/testRemoteStorage'. Reason: Requested repository RMRepository does not exist..
com.rapidminer.operator.UserError: Cannot store data in repository at entry '//RMRepository/testRemoteStorage'. Reason: Requested repository RMRepository does not exist..
...
...

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.2.008">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.2.008" expanded="true" name="Process">
    <process expanded="true" height="631" width="949">
      <operator activated="true" class="generate_massive_data" compatibility="5.2.008" expanded="true" height="60" name="Generate Massive Data" width="90" x="45" y="75">
        <parameter key="number_examples" value="1000"/>
        <parameter key="number_attributes" value="100"/>
        <parameter key="sparse_fraction" value="0.95"/>
      </operator>
      <operator activated="true" class="nominal_to_binominal" compatibility="5.2.008" expanded="true" height="94" name="Nominal to Binominal" width="90" x="179" y="75">
        <parameter key="attribute_filter_type" value="single"/>
        <parameter key="attribute" value="label"/>
        <parameter key="include_special_attributes" value="true"/>
      </operator>
      <operator activated="true" class="store" compatibility="5.2.008" expanded="true" height="60" name="Store" width="90" x="380" y="75">
        <parameter key="repository_entry" value="//RMRepository/testRemoteStorage"/>
      </operator>
      <connect from_op="Generate Massive Data" from_port="output" to_op="Nominal to Binominal" to_port="example set input"/>
      <connect from_op="Nominal to Binominal" from_port="example set output" to_op="Store" to_port="input"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
    </process>
  </operator>
</process>

wessel

@ Weka plugin
Do you have all the correct files uploaded?
W-.* is Weka stuff, so you need to have the Weka plugin installed.

@ Repository
For repository stuff you might have to change the repositories.xml in the .RapidMiner5 folder.
But I don't fully understand why it sometimes does and does not work.
You need to ask someone one the RapidMiner team.

Best regards,

Wessel

aryan_hosseinza

Hi Wessel,

1.What do you mean that I need to install Weka plugin ? I don't have it installed on my local machine , but it works fine , I guess that these are the libraries which were implemented in Weka and now RapidMiner is using them, isn't it the case ?

2.Do you know how I can change the xml file ? (Neither I find the RapidMiner5 folder)


Thanks ,
Arian

wessel

1. Really, you do have it installed on your local machine.
http://img1.uploadscreenshot.com/images/orig/11/32218380794-orig.jpg



2. Check out this post:
http://rapid-i.com/rapidforum/index.php/topic,2385.0.html

The repositories.xml is in your home folder.
You can manually edit it or copy it from your local machine.
Should contain entries like this:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<repositories>
  <localRepository>
    <file>/full/path/to/your/repository/Repository</file>
    <alias>Repository Name</alias>
  </localRepository>
</repositories>

aryan_hosseinza

Great , now it works , I had to copy .RapidMiner5 and set the repositories.xml file ,

but I've got a problem with the performance , do you know how can I make rapidminer (or maybe java) to use more of available cpu ?


Thanks ,
Arian