Decision Trees Tutorial 5: Exercise 2


Using Entropy to Construct a Decision Tree

The dataset:

District House Type Income Previous Customer Outcome
Suburban Detached High No Nothing
Suburban Detached High Yes Nothing
Rural Detached High No Responded
Urban Semi-detached High No Responded
Urban Semi-detached Low No Responded
Urban Semi-detached Low Yes Nothing
Rural Semi-detached Low Yes Responded
Suburban Terrace High No Nothing
Suburban Semi-detached Low No Responded
Urban Terrace Low No Responded
Suburban Terrace Low Yes Responded
Rural Terrace High Yes Responded
Rural Detached Low No Responded
Urban Terrace High Yes Nothing

The Decision Tree: Interactively build it

  • Click on the root node below and start building the tree.
  • Non leaf nodes can be "pruned" once they have been chosen (by clicking on the node and selecting "prune node completely")
  • The ratios on the branches indicate how well the chosen attribute at a node splits the remaining data based on the target attribute (‘outcome’).
  • Click on any nodes to hilight the rows in the data table that the rule down to that node covers.
  • At each node, the entropy of the data at that point in the tree will be given.
  • Information gain (entropy reduction) is then specified for each attribute.
  • Reducing entropy to zero is a way of building a decision tree here ( as the ID3 algorithm does).
    When no more nodes can be expanded, the tree has classified all the training data.
root node

6 Responses to Decision Trees Tutorial 5: Exercise 2

  1. Anonymous says:

    when the gain on “suburban” node is calculated.. income gave us the highest gain.. but y we are using home type as a new split??
    is this because we want to include every attribute or what??
    pleas reply

    • admin says:

      Yes we want all the data to be filtered by the tree and so we keep adding as many attributes as is necessary to cover all the instances of data.

  2. Anonymous says:

    Math for calculating entropy on subsequent nodes would be very helpful. Seeing the math/numbers is always better than words!

    Thanks though, great page!

  3. Anonymous says:

    that's the best tutorialĀ i ever seen !!!

  4. German Hernandez says:

    What is the name of the sofware that you used?
    Can i get it?

  5. michaelN says:

    I wrote the software on the site. It’s written in javascript and css… just the basic ID3 plus some more… it would be difficult to use it for your own data because of the way I wrote it. Check out the software page on this site if you are interested in proper software. thanks!

Leave a Reply to Anonymous Cancel reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>