Specificity, sensitivity and AUC measures via RM v9.1

CindyDK

For non-binominal/polynominal label target data, what would be the best/recommended cross-validation testing data performance measures/operators for Specificity, sensitivity, and AUC measures in RM v9.1?

Telcontar120

ROC / AUC and its related measures are designed for binominal label classification problems only.
The best way to approach if you have multiple classes is to encode with a one-vs-all-others approach.  So if you have possible label values {1,2,3} then encode that as 1 vs not 1, and generate the ROC.  You can repeat for the other classes as needed, store the performances, and average them as well if you want.

Telcontar120

Sensitivity is the true positive rate, and specificity is the true negative rate.  Both are also available in the Performance (Binominal classification) operator in RapidMiner so you should be able to get them in the same way you get the AUC above, by using the 1 vs all others comparison method for a multi-class problem. 
Here's a basic description of the concepts in case you need a refresher:   https://en.wikipedia.org/wiki/Sensitivity_and_specificity

CindyDK

Thanks for the sharing. It seems to utilize the loop to cover the number of the classes (Re: 1 vs all others). Any other thoughts about "specificity, sensitivity" measures?