The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

"Compare model performance graphs"

dupolasdupolas Member Posts: 4 Contributor I
edited June 2019 in Help
Hi there,

I was wondering if it is possible to run multiple classification models on the same dataset and compare the different performance curves.

For example I run a decision tree, neural net and logistic regression on a training and test set and would like to compare training and test performance.(eg. sq. avg error)

is this possible with Rapidminer?  I know it is with SAS and it is pretty usefull for me

Thanks a lot...

I googled for this and could not see any tutorials/posts on it..

Regards,

Geoffrey
Tagged:

Answers

  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Hi,

    I suppose you already know how to evaluate a single model in RapidMiner, e.g. with the X-Validation. The X-Validation puts out a performance vector, which should contain the root mean squared error. This value is only a scalar value, so I don't know from which data you want to create curves. Of course you can compare the values from different models, and if you extract the performance values, you also can write them into a single example set for easier comparison.

    For classification problems we can create ROC comparison charts.

    For the performance curves, please give us a more detailed description of what you want to do. Please read the post linked in my signature on how to write good questions.

    Happy Mining!
    ~Marius
  • dupolasdupolas Member Posts: 4 Contributor I
    Marius

    Thanks for the hints.... Still struggling with some of the concepts being new to data mining .

    Yes I was looking for ROC curves for classification problems.

    However conceptually trying this on the iris data set didn't work out because the label needs to be binomial instead of multiclass. So will have to manipulate my data set a bit to get the 'one vs the rest' labels. Some practical learning to be done there :)

    Regards

    Geoffrey
  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Fine. if you get stuck anywhere just post a message here!
  • AprileAprile Member Posts: 3 Contributor I
    Hi, I have the same question too and I've tried this process. But I'm not sure if it right in theory but it actually present the Comparing ROC for my 3 models. Would you mind helping me discuss about the process, if I did anything wrong.
  • AprileAprile Member Posts: 3 Contributor I
    https://community.rapidminer.com/discussion/59501/how-to-compare-classification-models-in-the-same-time-with-compare-their-rocs-this-is-my-process#latest
Sign In or Register to comment.