"Decision Tree for telemarketing sales, control group shows HIGHER response!"

IvanGV

We are astonished...

We made two fairly well performing, robust decision trees to use for telemarketing fixed-term deposits sales. For modelling, as a classifier I used telemarketing sales for one model and open market sales (no direct stimulation on behalf of the company) for the other one.

We selected the best cluster for each model and 'went live', later checking results with control groups left untouched.

The results tell us that, for both models' best clusters, leaving the clients alone results in greater sales than applying a telemarketing action on them to get the ones that wouldn't buy on their own.

So apparently, after telemarketing came in...

A) Some of the clients that were thinking about getting a fixed-term deposit decided it was not such a good idea.

The clients that according to the decision tree could have bought one were:

1. Interested, but also spooked by the offer (¿?).
OR...
2. Not interesed, not good model.

I'm trying as hard as I can to blame the model but I find no logical explanation that could take it down. Maybe it isn't good for clustering clients for this application, and that explains B)2., BUT, it doesn't explain A).

Am I missing something?

What do you think?

haddock

Hi there!

And welcome to RapidMiner. With respect it is almost impossible to make any sensible comments on your venture, as there is not enough information on either your data or your methods; probably your best bet, as this is a commercial venture, is to get consultancy from RM http://rapid-i.com/content/view/3/76/lang,en/.

Just my two cents!

steffen

I can only second haddock, but here some additional thoughts from me ...

Assuming your results are statistically significant ...

So you have built two separate models. But what about building a combined model on both datasets to select whose who would not buy on their own and simultaneously have a high probability to buy when contacted ?  You can easily imagine a table with four fields, representing buy=yes/know and contact=yes/know.

There is whole area devoted to this type of model building called "Uplift Modelling" (see e.g. http://en.wikipedia.org/wiki/Uplift_modelling).

regards,

steffen

IvanGV

Thank you for input steffen!

I have found this...

---
For uplift modeling, ideally we should have similar looking customer segments, who have been targeted as well as not, and the responses could be used to identify the effect of campaign on these segments. To develop uplift models, two representative samples are drawn from the population to form the Test (targeted by the campaign) and Control group (not targeted by the campaign).

From the positive respondents for the campaign on the Test group, we will develop a predictive model to predict the propensity of purchase, which will be denoted by

pT = probability ( purchase | Campaign)
Another predictive model would be developed on the control group population to determine the propensity of customers to purchase and this is denoted as

pU = probability ( purchase | no Campaign)
Based on the ouput for these models, we can determine the uplift in propensity for any customer if he is included in a campaign. Based on the magnitude of incremental lift in purchase propensity, customers could be targeted for the campaign.
--- from http://www.latentview.com/uplift-modeling.html

steffen

Hello IvanGV

Thank you for the link.

greetings,

steffen