Bug report : error in Logistic Regression in AutoModel with FS enabled

lionelderkrikor

Hi,

I wanted report a bug raised by AutoModel when logistic Regression is used as model with Feature Selection enabled :  



The bug seems linked to the feature Selection (because without FS, no error is occuring).
So to reproduce this error,  submit the data in attached file, to AutoModel by enabling Feature Selection and Feature Generation.

Regards,

Lionel 

varunm1

Hello @lionelderkrikor

When I run all algorithms in AM it is showing the error for LR & NB, but when I only run LR in AM it is not.

I got results (.rmp attached), I only used FS and also tried both FS and generation. 

Now coming to the error, I observe there are two errors here, one is for Naive Bayes and the other is for Logistic regression. One reason I remember is related to the problem with H2O based model operators. I remember @IngoRM informing that H2O is removing constant columns automatically, due to this the model is not getting any input attribute to train during feature selection as seen in Naive Bayes error below.

       Cannot execute log reg calibration learning: Error while training the H2O model: Illegal argument(s) for GLM model: ERRR on field: _train: Training data must have at least 2 features (incl. response).     

I suspect something similar is happening with Logistic regression as well. I tried to debug the LR alone don't know why it is not throwing an error. @IngoRM might help us here.

lionelderkrikor

Thanks for your message, Varun.

I suspect something similar is happening with Logistic regression as well.

Yes, I agree with you : I have the same intuition as you.

One reason I remember is related to the problem with H2O based model operators

Yes, I have "reopen" the dedicated thread for this error. (the error with Naive Bayes). In deed i allow myself to reopen this other thread
because now the dataset is "normal"  :  there are not rare values in the label and not rare values in the nominal (regular) attributes.
So as the conclusion this dataset is not "problematic"from my point of view...

 due to this the model is not getting any input attribute to train during feature selection as seen in Naive Bayes error below

Yes, if it is the case, I think that these particular cases have to be handled by an "HANDLE EXCEPTION" or a "IF/ELSE" statement in order to not produce an error that deprives the user of result for a given model (in this case NB). 

Regards,


Lionel

varunm1

Yes, if it is the case, I think that these particular cases have to be handled by an "HANDLE EXCEPTION" or a "IF/ELSE" statement in order to not produce an error that deprives the user of result for a given model (in this case NB). 

I am thinking something similar to " ignore errors" option in optimize selections operator, that will help in resolving this, same as your suggestion

sgenzer

hi yes I have a hunch that this is the same H20 issue as before but I will push it along. Thx @lionelderkrikor

svmayor

How do you turn off the Feature Selection so that it is not enabled in Auto Model?

sgenzer

hi @svmayor can you please post a new question? This thread is dedicated to a particular bug report. Thank you.

svmayor

@sgenzer This is in relation to the bug. It was previously stated that if you turn off Feature Selection enabled then the bug goes away. I have since found where to turn this off and the bug still occurs.

sgenzer

ok thank you for the feedback.

varunm1

Hello @svmayor

The feature selection is default off in automodel. If you are looking for a way to restrict (On/Off) Auto model from using "Automatic Feature Engineering" there is an option. If you are facing this error without feature selection, please attach your process from automodel and data, so that we will look into it.



If this is not what you are looking for, please inform us with a detailed request in a new post.