Tree to Rules

Madcap

Hi, I am doing a dissertation on data mining techniques, I am coming to the end of writing about decision trees and have included a little on rule extraction. 
I just wondered what benefits rule induction might offer for businesses. The most I could find is that it may for some people be simpler to understand however I am guessing there has to be more?
If someone could point me in the write direction that would be great.
Thanks for any help
-Madcap

IngoRM

Hi @Madcap

I personally the understandability aspect is very similar.  In fact, the representation of trees can sometimes be even a bit more compact than rule sets.  I think the main advantage of rules is that many systems for decision management (https://en.wikipedia.org/wiki/Decision_management) are often rule-based and so it may look easier to users if the output of models is already rule-based.  But in all fairness, most of those systems are powerful enough to also invoke external services which for example create a (set of) scores which are then used in the rules of those systems.

Just my 2c,
Ingo

kypexin

Hi @Madcap

Ingo has explained it well, but just want to add my 5 cents with one concrete case here, as the question topic is really close to my expertise. 

There are quite a few certain industries / domains where now days the trend is that any machine learning based solution does not fully replace rule engine. I work in fraud detection domain (mostly finance and e-commerce) for quite long and there are really big fraud prevention providers which still maintain and utilize business rule systems along with more advanced ML-based solutions. I see no tendency for it to change in the nearest future. 

The reason is that these approaches do not compete but organically add up to each other. Often it is really faster and more easy to quickly add a new rule rather than to update models in production environment (which is almost always longer). Also, approaches like 'rule induction' might also contribute to quick creation of a ruleset based on model results. Too many businesses still have really complicated legacy rule systems and it might be a matter of years to fully get rid of them (in case there's such an intention). And many business models need some tools for very quick reaction, this is where rule systems are still strong.

Aside of that, many small companies just don't have enough expertise and IT power for implementing advanced solutions (and they just might not need to do it). This said, design a simple rule system is faster and cheaper than building a model hosting ecosystem (or hard-coding decision trees )

I hope this example helps  

rfuentealba

Hello,

I have something to add to the already excellent answers by @IngoRM and @kypexin:

Every rule-based decision management system I've seen lack the ability to work together with databases, or if they have it, it is pretty rudimentary. All the data preparation comes before the invocation of a decision manager because they are conceptualized as part of SOA (Service Oriented Architecture).

As @kypexin says, rule-based systems and data science systems are complements to each other. I can give you an example of a fintech customer I work with: they have both a data science suite (not RapidMiner, but they are strongly considering it, so yay!) and a rule based decision manager. Once every month, they take their database, upload it to their data science suite and study new behaviors. They then go to the rule designer, make a few adjustments to put their changes to production.

The data science suite helps them dealing with imbalanced data and other phenomena because the decision manager doesn't have the features required to do it, but the decision manager helps in controlling every aspect of the decisions being made by generating rules in natural language. Plus, it does not need to be operated by a data science practitioner.

I'm told that most fintech companies here operate under this scheme, and some smaller ones don't even make use of a data science suite. Instead, they hire a third-party company that makes this kind of studies on demand.

Now, how to deal with legacy rule systems is up to the kinds of features rule systems have. The most common one is incremental versioning, but I've found a few (painful) ones that don't even have incremental versioning and rely upon request rewriting to rebuild past decisions.

I hope this helps,

Rodrigo.

Madcap

Hi guys, thanks very much for the insights really helpful, especially with the examples.
That gives me a much better idea and saves me from just writing, "well they are a bit easier to understand..."!!

Thanks again
-Madcap