The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

AUC Unknown

emteeemtee Member Posts: 3 Contributor I
edited March 2020 in Help

Hi!

 

I'm doing predictive modeling using logistic regression with batch x validation. Everything is working fine I guess except for the AUC value. It shows Unknown as shown in the attached image. I also attached a screenshot of my process for your reference. Thank you!

Tagged:

Answers

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist

    Hi Emtee,

     

    kind of funny. I yesterday had an internal talk about Batch X-Val :).


    Are you sure that each fold of X-Val has both classes in it? Otherwise the AUC gets NaN, which might explain it. To check this you can either use breakpoints or use a store with something like ../data/%{a} as a path. %{a} will be replaced with the executions count of this operator (1 in the first iteration, 2 in the second etc). You have then k results in your repo and can check it.

     

    ~Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • ozgeozyazarozgeozyazar Member Posts: 21 Maven
    Hi @mschmitz !

    I also has this kind of output and I do not understand your solution advice. Could you please add more details ?

    By the way, I am doing classification with decision tree with AdaBoost and using cross validation.

    Best Regards,

    Özge
  • Telcontar120Telcontar120 RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn
    If you have a small sample or a very imbalanced class, you could end up with a sample that does not contain both classes, in which case the AUC is not defined.  @mschmitz is saying you should check your samples and ensure that is not the case.   You can usually prevent this if you are using automatic sampling but if you are using other other sampling options it does not control for this.

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • ozgeozyazarozgeozyazar Member Posts: 21 Maven
    Hi @Telcontar120

    thanks for explanation ☺️
Sign In or Register to comment.