Tutorial (5): Exercise 2

Using Entropy to Construct a Decision Tree

The dataset:

District	House Type	Income	Previous Customer	Outcome
Suburban	Detached	High	No	Nothing
Suburban	Detached	High	Yes	Nothing
Rural	Detached	High	No	Responded
Urban	Semi-detached	High	No	Responded
Urban	Semi-detached	Low	No	Responded
Urban	Semi-detached	Low	Yes	Nothing
Rural	Semi-detached	Low	Yes	Responded
Suburban	Terrace	High	No	Nothing
Suburban	Semi-detached	Low	No	Responded
Urban	Terrace	Low	No	Responded
Suburban	Terrace	Low	Yes	Responded
Rural	Terrace	High	Yes	Responded
Rural	Detached	Low	No	Responded
Urban	Terrace	High	Yes	Nothing

The Decision Tree: Interactively build it

• Click on the root node below and start building the tree.
• Non leaf nodes can be "pruned" once they have been chosen (by clicking on the node and selecting "prune node completely")
• The ratios on the branches indicate how well the chosen attribute at a node splits the remaining data based on the target attribute ('outcome').
• Click on any nodes to hilight the rows in the data table that the rule down to that node covers.
• At each node, the entropy of the data at that point in the tree will be given.
• Information gain (entropy reduction) is then specified for each attribute.
• Reducing entropy to zero is a way of building a decision tree here ( as the ID3 algorithm does).
  When no more nodes can be expanded, the tree has classified all the training data.

root node