Decision Trees: Interpretable by Default
A decision tree splits data by asking yes/no questions about features, creating a flowchart that maps inputs to predictions.
Neural networks are powerful, but try explaining to a compliance officer why your model denied a loan. "The 47th neuron in the third hidden layer activated strongly" won't cut it. Decision trees give you something neural networks can't: a clear explanation for every prediction.
Here's the identity bridge — a decision tree is nested if/else statements. If you've ever written conditional rendering logic in a component, you've already built a decision tree.
Learning Objectives
- ○Understand how decision trees split data using feature thresholds
- ○Calculate Gini impurity to measure split quality
- ○Build a simple decision tree from scratch in TypeScript
- ○Recognize overfitting as the tree equivalent of over-nested components
Trees Are Just Nested If/Else
This is the rare "identity bridge" — the frontend concept and the ML concept are the same thing.
Frontend
Nested if/else
if (age > 30) { if (income > 50k) { return 'approve'; } }Machine Learning
Decision tree
tree.split('age', 30).split('income', 50000).predict()// This React component IS a decision tree
function PricingTier({ user }: { user: User }) {
if (user.plan === 'enterprise') {
if (user.seats > 100) return <EnterprisePlus />;
return <Enterprise />;
}
if (user.monthlySpend > 50) return <Pro />;
return <Free />;
}
// The ML equivalent learns these rules from data
// instead of you hard-coding them
interface TreeNode {
feature?: string;
threshold?: number;
left?: TreeNode; // feature <= threshold
right?: TreeNode; // feature > threshold
prediction?: number; // leaf node
}
const learnedTree: TreeNode = {
feature: 'plan_level',
threshold: 2, // enterprise = 3, pro = 2, free = 1
left: {
feature: 'monthly_spend',
threshold: 50,
left: { prediction: 0 }, // free tier
right: { prediction: 1 }, // pro tier
},
right: {
feature: 'seats',
threshold: 100,
left: { prediction: 2 }, // enterprise
right: { prediction: 3 }, // enterprise plus
},
};Gini Impurity: How Good Is a Split?
When the tree decides where to split, it needs a way to measure quality. Gini impurity answers: "If I randomly picked two items from this group, how likely is it they'd be different classes?"
// Gini impurity: 0 = perfectly pure, 0.5 = maximum impurity (binary)
function giniImpurity(labels: number[]): number {
const total = labels.length;
if (total === 0) return 0;
const counts = new Map<number, number>();
for (const label of labels) {
counts.set(label, (counts.get(label) ?? 0) + 1);
}
let impurity = 1;
for (const count of counts.values()) {
const p = count / total;
impurity -= p * p;
}
return impurity;
}
// Pure group: all same class → gini = 0
console.log(giniImpurity([1, 1, 1, 1])); // 0
// Perfectly mixed → gini = 0.5
console.log(giniImpurity([0, 1, 0, 1])); // 0.5
// Mostly one class → low gini
console.log(giniImpurity([1, 1, 1, 0])); // 0.375Finding the Best Split
The tree tries every possible feature and threshold, picks the split with the lowest weighted Gini impurity in the resulting groups.
interface DataPoint {
features: Record<string, number>;
label: number;
}
function findBestSplit(data: DataPoint[], featureNames: string[]) {
let bestGini = Infinity;
let bestFeature = '';
let bestThreshold = 0;
for (const feature of featureNames) {
// Get unique values for this feature, sorted
const values = [...new Set(data.map(d => d.features[feature]))].sort((a, b) => a - b);
// Try each midpoint as a threshold
for (let i = 0; i < values.length - 1; i++) {
const threshold = (values[i] + values[i + 1]) / 2;
const left = data.filter(d => d.features[feature] <= threshold);
const right = data.filter(d => d.features[feature] > threshold);
// Weighted Gini impurity
const weightedGini =
(left.length / data.length) * giniImpurity(left.map(d => d.label)) +
(right.length / data.length) * giniImpurity(right.map(d => d.label));
if (weightedGini < bestGini) {
bestGini = weightedGini;
bestFeature = feature;
bestThreshold = threshold;
}
}
}
return { feature: bestFeature, threshold: bestThreshold, gini: bestGini };
}Overfitting: The Component Over-Nesting Problem
A decision tree with no depth limit will create a perfect rule for every single training example — and fail on new data. This is like creating a deeply nested component tree where each component handles exactly one edge case. It works, but it's brittle.
Set a maxDepth to prevent this, just like you'd refactor deeply nested components.
Challenge
Build a decision tree that learns splits from data.
Exercise
Build a Decision Tree
Implement the giniImpurity function that measures how mixed the class labels are in a group. Then implement findBestSplit, which tries every feature and threshold to find the split that minimizes weighted Gini impurity. The giniImpurity of a pure group should be 0 and a perfectly mixed binary group should be 0.5.
Key Takeaways
- ✓Decision trees are literally nested if/else — the identity bridge
- ✓Gini impurity measures how mixed the classes are (0 = pure, 0.5 = max)
- ✓Trees find the best feature and threshold to split on at each node
- ✓Max depth prevents overfitting — same principle as avoiding deep component nesting