The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
Confidence interval calculation on performance
Contributor IIHello,
I see that for many classification performance metric, RapidMiner provides an estimation which I interpret as an interval of confidence around the value provided. However I have failed to find the exact calculation that is performed, in particular for the kappa value.
To be clear, my question seems similar to the one of @taghaddo from last December (see post there: https://community.rapidminer.com/discussion/54694/error-range-of-classifier), which as far as I can tell, hadn't been answered to. Would anyone be able to clarify this point? Maybe by posting a snippet of the actual source code for that calculation as it seems to be sometimes done (e.g. for the kappa calculation there https://community.rapidminer.com/discussion/54909/regarding-kappa-value-in-cross-validation)?
Many thanks,
François
Tagged:
0
Best Answer
-
lionelderkrikor
RapidMiner Certified Analyst, Member Posts: 1,195 
Unicorn
Hi François,
It is because RapidMiner is using, in this particular case, a Cross Validation (See the Help section of this operator).
With this technique, RM is training and testing k models (according to the number of folds you set)
Thus it obtains k performances. Then RM calculates the average and the standard deviation of this k performances.
So the displayed values are : mean +/- standard deviation
Regards,
Lionel7
Answers