The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Any ways of calculating coefficient of var and standard error in Linear Reg for cross validation?

binaytamrakarbinaytamrakar Member Posts: 5 Contributor II
edited November 2018 in Help

I am using Rapid miner for Linear regression analysis.

 

I think there are some important analysis missing in rapidminer, For eg, There is relative Mean square error but no Standard error of Regression. From my knowledge, I know that RMSE takes N in denominator whereas standard error takes (N-2) in the denominator.

Also, I am looking for Coefficient of Variation for the model (100*std error/ mean of dependent var.) analysis while comparing the performance of one model on the other.

 

I can do these calculations manually, but when I am using 10- fold cross validation, which takes the avg of all the 10 performances, I cannot calculate for all of the folds and take average. 

 

Could someone explain me, if we can calculate standard error of regression and coefficient of variation for the model in the model using rapidminer?

Tagged:

Answers

  • IngoRMIngoRM Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 RM Founder

    Hi,

     

    You can use the operator X-Prediction to get the predictions for all examples in a cross validated fashion.  Then you can use Generate Attributes on the original label and the prediction for the calculation of the error followed by Aggregate and another division (probably easierst via Generate Macro) to calculate the performances you want.  This would only lead to the microaverage of the cross validation though, not the macro average and standard deviations.

     

    Another option is to create an extension which offers those performance metrics in case you know some Java.

     

    Cheers,

    Ingo

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist

    Hi,

     

    another way to do this is to generate the performance value by hand (using e.g. Aggregate) and then Data to Performance to get a performance vector which can be used everywhere in Rapidminer - for example in cross validation. You can put this construct into a process which can be used anywhere using excute process.

     
    ~Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • binaytamrakarbinaytamrakar Member Posts: 5 Contributor II

    Dear Ingo,

     

    Very much well appreciated for your answer. Yes, following this procedure will yield me only the mikro average, but I prefer macro average, which is taken across the folds. Is there other option, if I want to calculate macro average?

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist

    Dear Binaytamrakar,

     

    you can use a log operator to log the individual performances and then use Log to Data to get the table for all folds.

     

    ~Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
Sign In or Register to comment.