Topic 2: Basic Terminology of Decision Trees
This topic forms the foundation of everything that follows in Decision Trees. Once you understand this terminology, advanced topics such as splitting criteria, tree pruning, feature importance, Random Forests, and Gradient Boosting become much easier to understand.
Why Do We Need This Terminology?
Every field has its own vocabulary.
For example:
- Biology uses terms such as cell, tissue, and organ.
- Computer Networks use terms like router, switch, and gateway.
- Decision Trees have their own terminology that describes the structure of the tree and how predictions are made.
Throughout the remaining topics, we'll repeatedly use these terms. Therefore, it's essential to become familiar with them now.
Topics Covered
In this lesson, we'll study the following terms:
- Root Node
- Decision Node
- Leaf (Terminal) Node
- Parent Node
- Child Node
- Branch
- Subtree
- Depth of a Tree
- Height of a Tree
- Levels
Example Decision Tree
We'll use the same Decision Tree throughout this topic so that each term becomes easy to understand.
This tree predicts whether a customer will buy a product.
Every prediction starts at the top of the tree and follows one path until a final decision is reached.
1. Root Node
Definition
The Root Node is the topmost node of a Decision Tree. It is the starting point for every prediction made by the model.
In Our Example
This question is the Root Node because it is the first question every customer must answer.
Why Is the Root Node Important?
Every prediction begins at the Root Node.
Suppose a customer has the following details:
- Age = 35 years
- Income = £120,000
- Married = Yes
The algorithm does not immediately examine the customer's income or marital status.
Instead, it always begins by asking:
Only after answering this question does the algorithm move to the next appropriate branch of the tree.
Characteristics of the Root Node
- There is exactly one Root Node in every Decision Tree.
- The Root Node has no parent.
- Every prediction always starts from the Root Node.
- It usually contains the most informative feature selected by the learning algorithm.
How Is the Root Node Chosen?
The Root Node is not selected randomly.
During training, the Decision Tree algorithm evaluates all available features and chooses the one that produces the best separation of the training data.
In later topics, we'll discover how algorithms such as ID3, C4.5, and CART mathematically decide which feature deserves to become the Root Node using concepts such as Entropy, Information Gain, and Gini Impurity.
Part 1 Summary
- A Decision Tree has its own structural vocabulary.
- We'll use one example tree throughout this topic.
- The Root Node is the topmost node and the starting point of every prediction.
- Every Decision Tree has exactly one Root Node.
- The Root Node usually contains the feature that best separates the training data.
2. Decision Node
Definition
A Decision Node is any node in a Decision Tree that asks a question and divides the data into two or more branches.
Every time the tree reaches a Decision Node, it evaluates a condition based on one of the input features and decides which branch should be followed next.
Decision Nodes in Our Example Tree
Consider the Decision Tree introduced earlier.
The following nodes ask questions and therefore are Decision Nodes:
- Age ≥ 30?
- Income ≥ £100k?
- Married?
Function of a Decision Node
The primary purpose of a Decision Node is to partition (split) the dataset into smaller and more homogeneous groups.
For example, suppose the tree asks:
This single question divides all customers into two groups:
Group 1
Customers with an income greater than or equal to £100,000.
Group 2
Customers with an income below £100,000.
This repeated splitting of data is the core mechanism behind every Decision Tree.
3. Leaf Node (Terminal Node)
Definition
A Leaf Node is a node that does not split any further.
Instead of asking another question, it stores the final prediction made by the Decision Tree.
Leaf Nodes in Our Example Tree
The highlighted predictions are the Leaf Nodes.
- Don't Buy
- Buy
- Buy
- Don't Buy
Why Are They Called Terminal Nodes?
The word terminal means ending point.
Once the algorithm reaches a Leaf Node, the prediction process stops. No further questions are asked.
Example
Suppose a customer has:
- Age = 25 years
The prediction path becomes:
Since the tree has reached a Leaf Node, the prediction is complete.
Leaf Nodes in Classification Trees
In a Classification Tree, each Leaf Node contains a class label.
Typical examples include:
Leaf Nodes in Regression Trees
In a Regression Tree, the Leaf Node stores a numerical value instead of a category.
Predicted House Price = £450,000
Therefore, the contents of a Leaf Node depend on whether the Decision Tree is solving a classification problem or a regression problem.
Decision Node vs. Leaf Node
| Decision Node | Leaf Node |
|---|---|
| Asks a question. | Stores the final prediction. |
| Splits the dataset. | Does not split further. |
| Has one or more child nodes. | Has no children. |
| Represents a decision. | Represents the final outcome. |
Part 2 Summary
- A Decision Node asks a question and splits the dataset into smaller groups.
- The repeated splitting of data is the core mechanism of every Decision Tree.
- A Leaf (Terminal) Node stores the final prediction and does not split further.
- Classification Trees store class labels in Leaf Nodes, while Regression Trees store numerical values.
- The prediction process always ends at a Leaf Node.
4. Parent Node
Definition
A Parent Node is any node that has one or more child nodes.
Whenever a node branches into other nodes, it automatically becomes the parent of those nodes.
Parent Relationship in Our Example
Consider the following portion of our Decision Tree:
Here, Age ≥ 30? is the Parent Node because it has two children:
- Don't Buy
- Income ≥ £100k?
Another Example
Now consider another part of the same tree:
Here, Income ≥ £100k? is also a Parent Node because it has two children:
- Buy
- Married?
Important Observation
A node is called a Parent Node only because it has child nodes.
Whether a node is a parent has nothing to do with its position in the tree. It depends entirely on whether it has descendants.
Every internal decision node is usually a Parent Node because it continues the prediction process by creating one or more branches.
Can a Node Be Both a Parent and a Child?
Yes.
This is one of the most important concepts to understand.
A node can simultaneously be:
- a child of another node, and
- a parent of additional nodes.
Example
Here, Income ≥ £100k? plays two different roles:
- It is a child of Age ≥ 30?.
- It is a parent of Buy and Married?.
5. Child Node
Definition
A Child Node is any node that descends from a Parent Node.
Every node except the Root Node has exactly one parent.
Child Relationship in Our Example
Both Buy and Married? are Child Nodes because they originate from the Parent Node Income ≥ £100k?.
Family Tree Analogy
One of the easiest ways to remember these terms is to compare a Decision Tree with a family tree.
| Family Tree | Decision Tree |
|---|---|
| Grandparent | Root Node |
| Parent | Decision Node |
| Child | Leaf Node (or another Decision Node) |
Just like people can be both someone's child and someone else's parent, an internal node in a Decision Tree can simultaneously be a Child Node and a Parent Node.
Quick Practice
Consider the following simplified tree:
Can you identify the different node types?
- Weather? → Root Node and Parent Node
- Don't Play → Child Node and Leaf Node
- Play → Child Node and Leaf Node
Part 3 Summary
- A Parent Node has one or more child nodes.
- A Child Node descends from a Parent Node.
- Every node except the Root Node has exactly one parent.
- An internal Decision Node can be both a Parent Node and a Child Node at the same time.
- The family tree analogy is an excellent way to remember these relationships.
6. Branch
Definition
A Branch is the connection between a Parent Node and one of its Child Nodes.
Every branch represents the outcome of a decision.
Once a question is answered, the algorithm follows the corresponding branch to continue making the prediction.
Branches in Our Example Tree
Consider the Root Node:
Here, there are two branches:
- Left Branch → Answer = No
- Right Branch → Answer = Yes
How Should We Interpret a Branch?
A branch represents a decision path.
For example, suppose the algorithm follows this branch:
This means:
Every branch narrows down the dataset by applying one additional rule.
A Complete Decision Path
Multiple branches together form a Decision Path.
This complete sequence of branches means:
7. Subtree
Definition
A Subtree is any portion of a Decision Tree that itself forms a complete tree.
In other words, if you select any node together with all of its descendants, the resulting structure is called a Subtree.
Example of a Subtree
Consider the following part of our Decision Tree:
This entire structure is a Subtree.
Notice that it has its own root (Income ≥ £100k?), branches, internal nodes and leaf nodes.
Why Are Subtrees Important?
Subtrees play a major role in many Decision Tree algorithms and advanced machine learning techniques.
| Technique | How Subtrees Are Used |
|---|---|
| Tree Pruning | Removes unnecessary subtrees to reduce overfitting. |
| Random Forest | Builds many independent trees, each containing numerous subtrees. |
| Gradient Boosting | Sequentially grows trees and improves predictions by learning from previous trees. |
Visualising a Subtree
Imagine removing everything above the highlighted node. The remaining structure is still a valid Decision Tree.
Real-Life Analogy
Think of a company organisation chart.
Chief Executive Officer (CEO)
├── Sales Department
├── Finance Department
└── Engineering Department
If we focus only on the Engineering Department together with all its teams, we obtain a smaller organisation chart.
That smaller organisation chart is analogous to a Subtree in a Decision Tree.
Part 4 Summary
- A Branch connects a Parent Node to one of its Child Nodes.
- Every branch represents the outcome of a decision.
- Several branches together form a Decision Path.
- A Subtree is any node together with all of its descendants.
- Subtrees are fundamental to tree pruning, Random Forests and Gradient Boosting algorithms.
8. Depth of a Tree
Definition
The depth of a node is the number of edges between the Root Node and that particular node.
In simple terms, depth tells us how far a node is from the Root Node.
We count edges (connections), not nodes.
Example
Consider the following simplified Decision Tree.
Let's determine the depth of each node.
Counting the Edges
Start from the Root Node and count the number of connections required to reach each node.
Root Node
Number of edges from the root: 0
Income ≥ £100k?
Number of edges: 1
Married?
Number of edges: 2
Depth of Individual Nodes
| Node | Number of Edges from Root | Depth |
|---|---|---|
| Age ≥ 30? | 0 | 0 |
| Don't Buy | 1 | 1 |
| Income ≥ £100k? | 1 | 1 |
| Buy | 2 | 2 |
| Married? | 2 | 2 |
| Don't Buy | 3 | 3 |
Depth of the Entire Tree
The depth of a Decision Tree is defined as the maximum depth among all of its nodes.
In other words, find the deepest Leaf Node and count how many edges separate it from the Root Node.
Suppose the deepest Leaf Node has a depth of 3.
Why Is Tree Depth Important?
Tree depth directly affects how complex a Decision Tree becomes.
| Tree Depth | Interpretation |
|---|---|
| Small Depth | Simpler model that is easier to understand but may underfit the data. |
| Large Depth | More powerful model that may capture complex patterns but can overfit the training data. |
Choosing an appropriate tree depth is therefore an important part of building an effective Decision Tree model.
Common Beginner Mistakes
- Counting nodes instead of edges.
- Assuming the Root Node has a depth of 1. It always has a depth of 0.
- Confusing the depth of a single node with the depth of the entire tree.
Interview Tip
What is the depth of the Root Node?
Answer:The Root Node always has a depth of 0 because there are no edges between the Root Node and itself.
Part 5 Summary
- The depth of a node is the number of edges between the Root Node and that node.
- The Root Node always has a depth of 0.
- The depth of a tree is the maximum depth among all of its nodes.
- Tree depth influences model complexity and the risk of overfitting.
- Always count edges, not nodes.
9. Height of a Tree
Definition
The height of a node is the number of edges on the longest path from that node to any Leaf Node.
Unlike Depth, which measures the distance from the Root Node downward, Height measures the distance from a node downward to its deepest descendant.
Height is always measured downward towards the Leaf Nodes.
Example
Consider the following Decision Tree.
Let's calculate the height of each node.
Calculating Height
Leaf Node
A Leaf Node has no descendants.
Therefore,
Income ≥ £100k?
The longest path from this node reaches Married? and then the final Don't Buy leaf.
Age ≥ 30?
This is the Root Node.
The longest path to a Leaf Node contains 3 edges.
Height of Individual Nodes
| Node | Height |
|---|---|
| Don't Buy (Leaf) | 0 |
| Buy (Leaf) | 0 |
| Married? | 1 |
| Income ≥ £100k? | 2 |
| Age ≥ 30? (Root) | 3 |
Height of the Entire Tree
The height of a Decision Tree is simply the height of its Root Node.
If the Root Node has a height of 3, then the entire Decision Tree also has a height of 3.
Depth vs Height
Beginners often confuse these two terms because both involve counting edges.
| Depth | Height |
|---|---|
| Measured from the Root Node. | Measured towards the Leaf Nodes. |
| Counts edges from Root to the current node. | Counts edges from the current node to the deepest leaf. |
| Root Node always has depth 0. | Every Leaf Node always has height 0. |
10. Levels
Definition
A Level groups together all nodes having the same Depth.
Therefore:
Example
Nodes at Each Level
| Level | Nodes |
|---|---|
| 0 | Age ≥ 30? |
| 1 | Don't Buy, Income ≥ £100k? |
| 2 | Buy, Married? |
| 3 | Don't Buy |
Part 6 Summary
- Height measures the distance from a node to its deepest Leaf Node.
- The height of every Leaf Node is 0.
- The height of the entire tree equals the height of the Root Node.
- Depth is measured from the Root, whereas Height is measured towards the deepest Leaf.
- A Level contains all nodes having the same depth.
Putting Everything Together
We have now learned all the important structural terms used in a Decision Tree.
Let's combine everything into a single labelled tree so that you can clearly see how every concept fits together.
Fully Labelled Decision Tree
Identifying Every Component
| Component | Example in Our Tree |
|---|---|
| Root Node | Age ≥ 30? |
| Decision Nodes | Income ≥ £100k?, Married? |
| Leaf Nodes | Buy, Don't Buy |
| Parent Node | Income ≥ £100k? |
| Child Nodes | Buy, Married? |
| Branches | Yes / No connections between nodes |
| Subtree | Entire tree beginning at "Income ≥ £100k?" |
What is a Decision Path?
A Decision Path is the complete sequence of branches followed from the Root Node to a Leaf Node.
Every prediction produced by a Decision Tree follows exactly one decision path.
Age ≥ 30?
│
Yes
│
Income ≥ £100k?
│
Yes
│
Buy
This entire route is one complete Decision Path.
Prediction Walkthrough
Suppose a customer has the following information:
- Age = 40 years
- Income = £120,000
- Married = Yes
Step-by-Step Prediction
- Start at the Root Node.
-
Ask:
Age ≥ 30?
Answer: Yes - Move along the Yes Branch to Income ≥ £100k?
-
Ask:
Income ≥ £100k?
Answer: Yes - Follow the Yes Branch to the Leaf Node.
-
Final Prediction:
Buy
Another Example
Consider another customer.
- Age = 24 years
- Income = £90,000
- Married = No
Prediction:
Notice that once the tree reaches a Leaf Node, no additional questions are asked.
Overall Prediction Flow
Start ↓ Root Node ↓ Decision Node ↓ Decision Node ↓ ... ↓ Leaf Node ↓ Prediction
Every prediction generated by a Decision Tree follows exactly this workflow.
One-Minute Revision
| Term | Quick Memory Trick |
|---|---|
| Root Node | Starting point |
| Decision Node | Asks a question |
| Leaf Node | Final prediction |
| Branch | Connection between nodes |
| Subtree | A smaller tree inside the main tree |
| Depth | Distance from the Root Node |
| Height | Distance to the deepest Leaf Node |
| Level | All nodes with the same depth |
Frequently Asked Interview Questions
Interviewers frequently ask these fundamental questions to check whether you understand the basic structure and terminology of a Decision Tree. These concepts are essential before moving on to more advanced topics such as splitting criteria, pruning and ensemble learning.
Question 1
Answer
A Decision Node asks a question about one of the input features and splits the dataset into two or more branches.
A Leaf Node, also called a Terminal Node, does not ask any further questions. Instead, it stores the final prediction produced by the model.
| Decision Node | Leaf Node |
|---|---|
| Asks a question | Gives the final prediction |
| Splits the data | Does not split further |
Question 2
Answer
Yes.
Internal Decision Nodes usually play both roles simultaneously.
- They are the Child Node of the node above them.
- They are also the Parent Node of the nodes below them.
In this example, Income ≥ £100k? is:
- Child of Age ≥ 30?
- Parent of Married?
Question 3
Answer
The Root Node always has a depth of 0.
This is because there are zero edges between the Root Node and itself.
Root → Depth = 0
Question 4
Answer
A Subtree is any node together with all of its descendants.
Every subtree is itself a valid Decision Tree containing its own:
- Root Node
- Branches
- Decision Nodes
- Leaf Nodes
Quick-Fire Interview Round
| Question | Expected Answer |
|---|---|
| Where does every prediction start? | Root Node |
| Which node stores the final prediction? | Leaf Node |
| What connects two nodes? | Branch |
| What is the depth of the Root Node? | 0 |
| What is the height of every Leaf Node? | 0 |
| What forms a Decision Path? | Sequence of branches from Root to Leaf |
