Course Wiki

Decision Trees

The if-else style model behind random forests: learning split rules from data.

Module 6 opens ensemble learning with decision trees: models that partition the feature space with learned split rules, choosing splits that best separate the target. A single tree overfits easily — it can memorize the training set — so the module treats it mainly as a building block.

The practical work trains trees with scikit-learn, reads feature importance from the fitted model, and then combines many trees two ways: bagging into random forests, and boosting with XGBoost. Hyperparameter tuning (max depth, min samples per leaf) against a validation set decides how deep the trees grow.

Taught in

• Machine Learning Zoomcamp

  • Module 1: Introduction to Machine Learning
    • Model Selection Process
  • Module 5: Deploying Machine Learning Models
    • Summary
  • Module 6: Decision Trees and Ensemble Learning
    • Credit risk scoring project
    • Decision trees
    • Decision tree learning algorithm
    • Decision trees parameter tuning
    • Ensemble learning and random forest
    • Gradient boosting and XGBoost
    • XGBoost parameter tuning
    • Selecting the best model
    • Summary ## Related concepts

• Gradient Boosting

• Logistic Regression

• Classification Metrics

• Neural Networks