"Normalized Absolute Error"

islem_h

Hi everyone

I applied different models (Random Forest, SVR, GLM...) to make predictions of a numerical target variable. In order to assess their performances, I chose RMSE and NAE (Normalized Absolute Error) from the options suggested by RapidMiner's operator "Performance".

I got (slightly) different rankings using the 2 metrics. What could be a reason for that?
It would be very helpful if anyone could provide me with a reference (documentation) about the Normalized Absolute Error (NAE). I couldn't interpret the results (1,3 and 0,62 and 0,59) based only on Rapidminer's definition of NAE.

Thank you a lot in advance!

varunm1

Hi @islem_h

Can you go through the below link where these performance metrics are explained with an example? There can be slight variation as in NAE we are normalizing which depends on minimum and maximum values. I think the better comparison would be with Normalized RMSE(not sure RM has this) and NAE. If possible please try to post the dataset and XML process so that we can understand better and inform you.

 https://www.researchgate.net/publication/327646833_Quality_of_the_predictions_mean_absolute_error_accuracy_and_coverage

islem_h

Hi @varunm1

The paper of the link you attached doent contain any information about NAE. 
It briefly mentions NRMSE and NMAE but no interpretation or so.

Thank you though

varunm1

Hi @islem_h

Based on RapidMiner document, I observe that NAE is also considering the average which is similar to NMAE. Lets see if @IngoRM could suggest anything about this

IngoRM

Sorry guys, a bit short on time today but maybe the source helps here:

https://github.com/rapidminer/rapidminer-studio/blob/master/src/main/java/com/rapidminer/operator/performance/NormalizedAbsoluteError.java

The class doc states:

"Normalized absolute error is the total absolute error normalized by the error simply predicting the average of the actual values."

Hope this helps,
Ingo