Multi label performance operator

ruhaila

Hi RapidMiner Community,

I need some help.

I am conducting an experiment for a paper I am writing. I want to compare a baseline experiment on Multi Target Prediction based on the following description.

"most straight-forward approach to solving MTP problems consists of constructing one model M for every target independently, and to concatenate the predictions of these models into the sought multi-target prediction. In the multi-label classification community, this model M for every target independently, and to concatenate the predictions of these models into the sought multi-target prediction."

I have put together operators to model 2 targets independently and all is working until to the point of the Multi Label Performance operator. I have set it to "auto detect label and prediction attributes", but it is not able to read the prediction attributes that I have set using the Set Roles operator before it. I have attached the workflow and the Set Role operator's parameters.

Anybody can suggest a workaround this? Would really appreciate it.

Regards

tftemme

Hi @ruhaila

The role of the prediction attributes has to be prediction_<label_attribute_name>.
So for your example prediction_anxiety-severity-lvl

Hopes this solves the problem. If not, please post your whole process (as xml file) here, best with a sample data set, if possible. This would the easiest way to find the problems you have.

By the way, do you know that there is also the operator Multi Label Modeling, to create the collection of models for your multi target prediction problem. Please have a look into the help text and the tutorial process to see how it can be configured.

Best regards,
Fabian

ruhaila

Thanks.
Your suggestion works.
I am aware of the Multi Label Modeling, however, I am doing an experiment that can be described by the following,

"most straight-forward approach to solving MTP problems consists of constructing one model M for every target independently, and to concatenate the predictions of these models into the sought multi-target prediction. In the multi-label classification community, this model M for every target independently, and to concatenate the predictions of these models into the sought multi-target prediction."

That is to be my baseline.

Additionally, is there a specific format for "confidence attributes"? Multi Label Performance is asking for this since my target variable is nominal. The error stated that the confidence attribute does not exist. I tried different possible formats,

confidence(anxiety-severity-lvl = Moderate)
confidence_anxiety-severity-lvl = Moderate
anxiety-severity-lvl = Moderate

but to no success.

ruhaila

Found the answer in this online documentation. 

https://docs.rapidminer.com/latest/studio/operators/modeling/predictive/ensembles/multi_label_model_learner.html

It would be good if Rapidminer could update this info in the Multi Label Performance operator's Help note. 

Thank you and regards. 

tftemme

Hi @ruhaila,

Good to hear that it helped. I honestly don't see a difference between your experiment and the Multi Label Modeling operator. Cause you select the target variables in the operator and the operator will built one model M for every target independently. When you use the Multi Label Model in Apply Model, it will generate prediction and confidence attributes for each target variable and concatenate them.

Yes you are right, the description of the expected confidence roles is not described in the Multi Label Performance operator, will see that we change this. The reason was, the operator were designed to be used together with the Multi Label Modeling operator (as I described above), so it was forgotten. Thanks for the feedback.

Best regards,
Fabian