The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
[SOLVED] Error:Deviaion of performance was NAN
Contributor II
Hi
I used evolutionary optimization operator to optimize select by weights operator and nu-SVR at the same time. but in the log wiev, i encountered the error "PM WARNING: SimpleCriterion: Deviation of Performance was NaN!". How can i fix the problem? Please i am waiting for your help
My model xml are:
I used evolutionary optimization operator to optimize select by weights operator and nu-SVR at the same time. but in the log wiev, i encountered the error "PM WARNING: SimpleCriterion: Deviation of Performance was NaN!". How can i fix the problem? Please i am waiting for your help
My model xml are:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.2.008">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="5.2.008" expanded="true" name="Process">
<parameter key="parallelize_main_process" value="true"/>
<process expanded="true" height="400" width="480">
<operator activated="true" class="retrieve" compatibility="5.2.008" expanded="true" height="60" name="Retrieve" width="90" x="45" y="30">
<parameter key="repository_entry" value="//kenan/Tarih/20.11.2012/Clus3/ClusByKMeans(Fast)Euclidian"/>
</operator>
<operator activated="true" class="weight_by_relief" compatibility="5.2.008" expanded="true" height="76" name="Weight by Relief" width="90" x="180" y="30"/>
<operator activated="true" class="parallel:optimize_parameters_evolutionary_parallel" compatibility="5.1.000" expanded="true" height="148" name="Optimize Parameters (Evolutionary)" width="90" x="313" y="210">
<list key="parameters">
<parameter key="Select by Weights (2).k" value="[15;35]"/>
<parameter key="SVM (2).gamma" value="[0.0;0.01]"/>
<parameter key="SVM (2).nu" value="[0.0;0.5]"/>
<parameter key="SVM (2).C" value="[0.0;0.01]"/>
<parameter key="SVM (2).epsilon" value="[0.0;0.01]"/>
</list>
<parameter key="max_generations" value="90"/>
<parameter key="population_size" value="6"/>
<parameter key="crossover_prob" value="0.8"/>
<parameter key="number_of_threads" value="8"/>
<parameter key="parallelize_optimization_process" value="true"/>
<process expanded="true" height="400" width="632">
<operator activated="true" class="select_by_weights" compatibility="5.2.008" expanded="true" height="94" name="Select by Weights (2)" width="90" x="45" y="30">
<parameter key="weight_relation" value="top k"/>
<parameter key="k" value="16"/>
</operator>
<operator activated="true" class="split_validation" compatibility="5.1.002" expanded="true" height="112" name="Validation (2)" width="90" x="380" y="30">
<parameter key="split_ratio" value="0.9"/>
<parameter key="sampling_type" value="linear sampling"/>
<parameter key="parallelize_training" value="true"/>
<parameter key="parallelize_testing" value="true"/>
<process expanded="true" height="400" width="165">
<operator activated="true" class="support_vector_machine_libsvm" compatibility="5.2.008" expanded="true" height="76" name="SVM (2)" width="90" x="45" y="30">
<parameter key="svm_type" value="nu-SVR"/>
<parameter key="gamma" value="0.0018245416895055634"/>
<parameter key="C" value="0.00562070136566738"/>
<parameter key="nu" value="0.48424641085734477"/>
<parameter key="epsilon" value="0.009366372796859642"/>
<list key="class_weights"/>
</operator>
<connect from_port="training" to_op="SVM (2)" to_port="training set"/>
<connect from_op="SVM (2)" from_port="model" to_port="model"/>
<portSpacing port="source_training" spacing="0"/>
<portSpacing port="sink_model" spacing="0"/>
<portSpacing port="sink_through 1" spacing="0"/>
</process>
<process expanded="true" height="400" width="300">
<operator activated="true" class="apply_model" compatibility="5.2.008" expanded="true" height="76" name="Apply Model (2)" width="90" x="45" y="30">
<list key="application_parameters"/>
</operator>
<operator activated="true" class="performance_regression" compatibility="5.2.008" expanded="true" height="76" name="Performance (2)" width="90" x="180" y="30">
<parameter key="main_criterion" value="root_mean_squared_error"/>
<parameter key="absolute_error" value="true"/>
<parameter key="relative_error" value="true"/>
<parameter key="root_relative_squared_error" value="true"/>
<parameter key="squared_error" value="true"/>
<parameter key="squared_correlation" value="true"/>
<parameter key="spearman_rho" value="true"/>
<parameter key="kendall_tau" value="true"/>
</operator>
<connect from_port="model" to_op="Apply Model (2)" to_port="model"/>
<connect from_port="test set" to_op="Apply Model (2)" to_port="unlabelled data"/>
<connect from_op="Apply Model (2)" from_port="labelled data" to_op="Performance (2)" to_port="labelled data"/>
<connect from_op="Performance (2)" from_port="performance" to_port="averagable 1"/>
<portSpacing port="source_model" spacing="0"/>
<portSpacing port="source_test set" spacing="0"/>
<portSpacing port="source_through 1" spacing="0"/>
<portSpacing port="sink_averagable 1" spacing="0"/>
<portSpacing port="sink_averagable 2" spacing="0"/>
</process>
</operator>
<connect from_port="input 1" to_op="Select by Weights (2)" to_port="weights"/>
<connect from_port="input 2" to_op="Select by Weights (2)" to_port="example set input"/>
<connect from_op="Select by Weights (2)" from_port="example set output" to_op="Validation (2)" to_port="training"/>
<connect from_op="Select by Weights (2)" from_port="weights" to_port="result 3"/>
<connect from_op="Validation (2)" from_port="model" to_port="result 1"/>
<connect from_op="Validation (2)" from_port="training" to_port="result 2"/>
<connect from_op="Validation (2)" from_port="averagable 1" to_port="performance"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="source_input 2" spacing="0"/>
<portSpacing port="source_input 3" spacing="0"/>
<portSpacing port="sink_performance" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
<portSpacing port="sink_result 3" spacing="0"/>
<portSpacing port="sink_result 4" spacing="0"/>
</process>
</operator>
<operator activated="true" class="log" compatibility="5.2.008" expanded="true" height="76" name="Log" width="90" x="380" y="30">
<parameter key="filename" value="C:\Users\KenanB\Desktop\RapidminerLog\ParameterLog.txt"/>
<list key="log">
<parameter key="ApplyCount" value="operator.Optimize Parameters (Evolutionary).value.applycount"/>
<parameter key="RMSE" value="operator.Performance (2).value.root_mean_squared_error"/>
<parameter key="K" value="operator.Select by Weights (2).parameter.k"/>
<parameter key="C" value="operator.SVM (2).parameter.C"/>
<parameter key="NU" value="operator.SVM (2).parameter.nu"/>
<parameter key="Gamma" value="operator.SVM (2).parameter.gamma"/>
<parameter key="Epsilon" value="operator.SVM (2).parameter.epsilon"/>
</list>
</operator>
<connect from_op="Retrieve" from_port="output" to_op="Weight by Relief" to_port="example set"/>
<connect from_op="Weight by Relief" from_port="weights" to_op="Optimize Parameters (Evolutionary)" to_port="input 1"/>
<connect from_op="Weight by Relief" from_port="example set" to_op="Optimize Parameters (Evolutionary)" to_port="input 2"/>
<connect from_op="Optimize Parameters (Evolutionary)" from_port="performance" to_port="result 2"/>
<connect from_op="Optimize Parameters (Evolutionary)" from_port="parameter" to_op="Log" to_port="through 1"/>
<connect from_op="Optimize Parameters (Evolutionary)" from_port="result 1" to_port="result 3"/>
<connect from_op="Optimize Parameters (Evolutionary)" from_port="result 2" to_port="result 4"/>
<connect from_op="Log" from_port="through 1" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
<portSpacing port="sink_result 3" spacing="0"/>
<portSpacing port="sink_result 4" spacing="0"/>
<portSpacing port="sink_result 5" spacing="0"/>
</process>
</operator>
</process>
0
Answers
as written elsewhere the evolutionary optimization operators have some hiccups at times. Please consider using the grid parameter optimization instead. Your process setup looks fine!
Good luck,
Marius
Thank for your reply. As you said there are really hiccups in evolutionary optimization. I clustered the data into three clusters. For two clusters this process works very vell. I watched log when processing, there was only two or three times i saw deviation was nan error. The process worked well and i got the results. For last cluster i got this error message. when i was watching log, sometimes i saw a message like "compatibility level of split validation was changed to 5.0.000". I think the cause of error was compatibility problem. Do you have any idea about compatibility problem? I need evolutionary optimization because the range for optimizing is too high. Grid optimization will be very slow.
Thanks again for your concern
I don't know if it will solve your problems, but I have some remarks on your process setup:
- The Split Validation is very dependent on the split point and much less robust than the X-Validation. You selected linear sampling, which means that your training set will always be the first 90 percent of the examples. If your data is sorted in any way, that will be a problem.
- Your Split Validation is set to compatibility level 5.1.002. Try to increase it to the current version of RapidMiner.
- Please inspect the problematic cluster manually. Do both training and test set contain reasonable examples?
Concerning the log, please post the errors with some lines before and behind. You should also try to start RapidMiner from the command line and see if you see further messages in the console.
~Marius
Thank you for your recommendations. I have renewed the model. I added the log operator after the validation. now it is working. Before running, ı worked it in debug mode. This way solved my problem but i don't think it is a robust solution. I am looking forward to develop the optmization operator from rapidminer team. I think it is best part of rapidminer. Thanks again for your instructions. My new model xml is below:
Beh, written that I just saw that you are still using the Split Validation, not the Cross Validation, so in this case it does not matter which value you are using, nevertheless I submit my writings as a hint for the next time
Best regards,
Marius