The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
how i can describe the quality of my model using confidence level value?
Contributor I
if I have a classification model to detect X from dataset contains X and Y labels.
1: what I can say about the model if the confidence level of Y prediction= 100% and for X=0 (most likely = Y?
2: what I can say about the model if the confidence level of X prediction=70% and for Y=30 (most likely = X?
1: what I can say about the model if the confidence level of Y prediction= 100% and for X=0 (most likely = Y?
2: what I can say about the model if the confidence level of X prediction=70% and for Y=30 (most likely = X?
Tagged:
0
Best Answer
-
Telcontar120
RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 
Unicorn
The confidence level is essentially the score. In general, the higher the confidence, the more likely that the model is going to be correct in its prediction. However, this is something that is true globally but not necessarily for any individual observation, in which case the prediction is always either correct or not correct.
You probably don't want to measure the quality of your model based on the confidence scores alone, but rather by using one of the performance metrics that are included. If you run the cross-validation tutorial, for example, you can see that for a classification problem, you can look at the confusion matrix and measure the quality of the score using measures like accuracy, precision, and recall. Or you could use the AUC (a personal favorite) which is slightly more complicated but tells you how good a job your score does overall separating and rank ordering the two classes.5
