"Decision Tree With Missing Values --

dragoljub

Hi Guys,

I am running into some strange results. It looks like the decision tree algorithm works quite nicely with missing attributes, however when I try to apply the model on the training data set the results are much worse. Is the apply model operator capable of classifying samples with  missing values, or is the decision tree algorithm just unrealisticly optimistic with missing values.

Anyone apply decision tree with missing values?  ???

Thanks,
-Gagi

dan_agape

Hi,

Not sure how missing values are handled by the RM's decision trees implemented algorithm, or by the model application. I wander if this is done via the usual technique of using surrogate attributes in a node, whenever the attribute to be tested has a missing value. CART is doing that, and RM documentation says the implemented algorithm is similar to C4.5 or CART (perhaps because of the splitting criteria, at least).

As a concrete solution, I suggest you try the following. When having missing values in the dataset, I usually do not leave them to be handled by the learner (although it is possible for many algorithms), but I just apply preprocessing techniques that are suitable to the data, before the main learning:

- attributes having too many missing values, let's say over 40-50%, may be discarded (as less useful, and may imply a lot of imprecision if used in learning and model application)

- the remaining missing values are refilled by:
--> using Impute Missing Values operator (excellent & more advanced feature that I prefer, but to take with caution as it is under development according to the documentation); so far I saw it works well and leads to good results when used with the k Nearest Neighbour as inner learner
--> using a more trivial method via computing the average or mode (among other options) on the known values of the attribute (use Replace Missing Values operator)

Dan

dragoljub

Thanks Dan,

I have also explored imputation however, it would be great to know how RM actually handles missing values. If anyone can chime in about this it would be helpful.

Also, it would be great if we had a List of operators that support/handle missing values.

Thanks,
-Gagi

land

Hi Gagi,
actually this list is built in. There is the capability "can handle missing values" and you can filter the operators accordingly if you use the "new operators" dialog.
The decision tree does not handle missing values in a very smart fashion, surrogate splits are still missing. We are aware of this since at least two years, but didn't find the time to reimplement the tree algorithm. I have already a few new ideas about the implementation including parallelization and missing value handling, but actually there's no time to really write the code. As you might have seen I even had to lower my activity here in the forum.
In fact I wouldn't be to sad if either someone would come up with an improved version of the code or would pay us for doing so. I personally would prefer getting payed for improving RapidMiner instead for drawing slides...

Greetings,
  Sebastian