W-Logistic only predicts 0 or 1

Username

Maybe I'm doing something wrong but the model learned with W-Logistic always returns 1 or 0 but no other values in between. I looked at the WEKA source code to check if this is correct but unfortunately the WEKA code is not as good as the RM code, so I'm not sure where the problem is.

Maybe the reason is that the label attribute of the learning example set is binominal and the predicted label attribute of the learned regression model must be numerical. The predicted label gets "cloned" from the label of the learning example set (getLabel() in PredictionModel). As far I understand the WEKA code the classifyInstance(Instance instance) method checks if the class of the instance is nominal or not.

TobiasMalbrecht

Hi,

well, since logistic regression is actually an approach to model a binary/binominal classification problem, I do not really see why a logistic regression model should predict any other values than 0 and 1 or the two classes, respectively. As for the confidences, they are of course returned by both Weka and RM logistic learners.

Regards,
Tobias

Username

Im actually interested in the values of exp(bx)/(1+exp(bx)). How can I calculate this value from the confidence of the prediction by the W-Logistic operator? Any hints are appreciated .

IngoRM

Hi,

you could apply a "FeatureGeneration" operator after applying the model. With this operator you can construct the desired feature.

Hope that helps. Cheers,
Ingo

jacoblyles

I am actually interested in this same problem. According to my understanding of logistic regression, the two class model returns a probability between 0.0 and 1.0 that indicates the conditional likelihood that a particular example will be in a particular class. Presumably, the LogisticRegression operator uses this probability internally to predict labels for the example set by using some sort of cut-off.

I would like to figure out what method it is using to predict labels and recover the class probabilities for the example set.

steffen

Hello

no problem: After the learning of a model and the usage of the operator "ModelApplier" a new attribute is added to the exampleset called confidence(y). This confidence is representing the conditioned likelihood p(y|x).

What you call "cutoff" is called "Threshold" - in RapidMiner. The default threshold is 0.5. The applied rule in the case of binary classification problem is:
Let y={0,1}

if(confidence(1) > threshold) 
       then 1
else 
       0

To set another threshold use the operators in the group Postprocessing.
The application of a threshold modifies or addes another attribute called prediction(...).

hope that was helpful

regards,

Steffen

brianbaker

I am trying to grab a model's confidence value as described in this post like this:

    <operator name="ModelApplier (3)" class="ModelApplier" breakpoints="after">
        <parameter key="keep_model"	value="true"/>
        <list key="application_parameters">
        </list>
        <parameter key="create_view"	value="true"/>
    </operator>
    <operator name="failConfidence" class="ConditionedFeatureGeneration" breakpoints="after">
        <parameter key="attribute_name"	value="fail1"/>
        <parameter key="value_type"	value="integer"/>
        <list key="values">
          <parameter key="confidence(F)"	value="prediction = F"/>
        </list>
        <parameter key="default_value"	value="0"/>
    </operator>

No attribute is created.  It is a binary label, F and P.  If I put in a constant instead of confidence(F) it works as expected.  I have tried several variants of confidence(F) -  confidence_F, confidence, F, confidence(1), but none work.

What is the naming convention for the confidence values?  This post mentions confidence(y), but I am not getting any success following that lead.

Thanks!

land

Hi,
there's  a simple way to get to know the attribute names. Simply insert a break point after the model applier using the context menu. Then the intermediate result will be shown and you might check which attributes are present in the data set. You then could simply mark the attributes needed for construction in the meta data view by clicking on the table cell and press Ctrl + C for copying the value into the clipboard and then pasting it in the parameter field. After adapting the parameter, you might press the button for proceeding with the process, using the new parameter values.

And again this will be much simpler in RapidMiner 5

Greetings,
  Sebastian

brianbaker

hmmm... I've already tried that.

I noticed a type error when it tries to create the feature that I missed yesterday.
[Error] failConfidence: values have to be numerical for numerical attributes: no feature is generated

I change the derived variable type to real, to match the confidence type, and then to numeric, as a catch all.  Both gave the same error.

Will RapinMiner 5 allow individual attribute typing?

Thanks for your help!

land

Hi,
what do you mean by individual attribute typing? If you mean, that you can set your own, user defined attribute types, then no, this will not work. No operator would know how to handle your user types. If you mean, that you can set the type individually per attribute, then the answer is, that this already works with RapidMiner 4.6, by using an AttributePreprocessing operator chain.

The problems I don't understand, because I don't know what you are doing. The small snippet of the process isn't enough to catch the complete background...

Greetings,
  Sebastian