Course Wiki

Regularization

Constraining model weights so redundant or noisy features cannot destabilize predictions.

In Machine Learning Zoomcamp, regularization means adding a penalty on the size of the weights to the training objective. When features carry similar information (for example, duplicate car specifications), unregularized linear regression can push weights to large opposite values that cancel out — the model fits the training data but generalizes badly.

The course shows ridge-style weight control implemented by hand in the normal-equation solution, and the same idea carries into logistic regression in the classification module. The regularization parameter is tuned against the validation set, which is the module’s repeated lesson: model settings are selected on held-out data, never on the test set.

Taught in

• Machine Learning Zoomcamp

  • Module 2: Machine Learning for Regression
    • Car price prediction project
    • Regularization
    • Tuning the model
    • Car price prediction project summary
  • Module 3: Machine Learning for Classification
    • Explore more
  • Module 6: Decision Trees and Ensemble Learning
    • XGBoost parameter tuning
  • Module 8: Neural Networks and Deep Learning
    • Fashion classification
    • Regularization and dropout ## Related concepts

• Linear Regression

• Logistic Regression

• Classification Metrics

• Gradient Boosting