Course Wiki

Gradient Boosting

Training trees sequentially on errors — the XGBoost technique ML Zoomcamp teaches for tabular prediction.

Gradient boosting trains an ensemble one tree at a time, with each new tree predicting what the ensemble so far gets wrong. Module 6 uses XGBoost for the practical work: monitoring training and validation error during boosting, tuning the learning rate, tree depth, and number of estimators, and reading feature importance from the final ensemble.

The module positions boosted trees as the strongest general-purpose model for tabular data in the course, and the capstone projects commonly use them. The evaluation discipline is the same as the rest of the course: tune against validation, report once on test.

Taught in

• Machine Learning Zoomcamp

  • Module 1: Introduction to Machine Learning
    • Setting up the Environment
  • Module 5: Deploying Machine Learning Models
    • Summary
  • Module 6: Decision Trees and Ensemble Learning
    • Gradient boosting and XGBoost
    • XGBoost parameter tuning
    • Selecting the best model
    • Summary
  • Module 8: Neural Networks and Deep Learning
    • Transfer learning
  • Module 9: Serverless Deep Learning
    • Explore more

• LLM Zoomcamp

  • Module 1: Agentic RAG
    • Search
  • Module 7: End-to-End Project
    • Evaluating Retrieval ## Related concepts

• Decision Trees

• Classification Metrics

• Regularization

• Linear Regression