Course Wiki

Training linear regression: Normal equation — Machine Learning Zoomcamp Module 2

Obtaining predictions as close as possible to target values requires the calculation of weights from the general LR equation. The feature matrix does not have an inverse because it is not square, so it is required to obtain an approximate...

Machine Learning Zoomcamp › Module 2: Machine Learning for Regression › Training linear regression: Normal equation

Notes

Obtaining predictions as close as possible to $y$ target values requires the calculation of weights from the general LR equation. The feature matrix does not have an inverse because it is not square, so it is required to obtain an approximate solution, which can be obtained using the Gram matrix (multiplication of feature matrix ($X$) and its transpose ($X^T$)). The vector of weights or coefficients $w$ obtained with this formula is the closest possible solution to the LR system.

Normal Equation:

$w$ = $(X^TX)^{-1}X^Ty$

Where:

$X^TX$ is the Gram Matrix

The entire code of this project is available in this jupyter notebook.

• Notes from Peter Ernicke

Key concepts

• Linear Regression
• Jupyter Notebooks

Related notes

• mlz-m01-introduction-to-numpy
• mlz-m01-introduction-to-pandas
• mlz-m01-linear-algebra-refresher
• mlz-m01-setting-up-the-environment
• mlz-m02-baseline-model-for-car-price-prediction-project
• mlz-m02-car-price-prediction-project
• mlz-m02-car-price-prediction-project-summary
• mlz-m02-categorical-variables
• mlz-m02-computing-rmse-on-validation-data
• mlz-m02-data-preparation
• mlz-m02-exploratory-data-analysis
• mlz-m02-explore-more
• mlz-m02-feature-engineering
• mlz-m02-linear-regression
• mlz-m02-linear-regression-vector-form
• mlz-m02-regularization
• mlz-m02-root-mean-squared-error-rmse
• mlz-m02-setting-up-the-validation-framework
• mlz-m02-tuning-the-model
• mlz-m02-using-the-model
• mlz-m03-data-preparation
• mlz-m03-eda
• mlz-m03-explore-more
• mlz-m03-feature-importance-churn-rate-and-risk-ratio
• mlz-m03-feature-importance-correlation
• mlz-m03-feature-importance-mutual-information
• mlz-m03-logistic-regression
• mlz-m03-model-interpretation
• mlz-m03-one-hot-encoding
• mlz-m03-setting-up-the-validation-framework
• mlz-m03-training-logistic-regression-with-scikit-learn
• mlz-m04-accuracy-and-dummy-model
• mlz-m04-confusion-table
• mlz-m04-cross-validation
• mlz-m04-roc-auc
• mlz-m04-roc-curves
• mlz-m04-summary
• mlz-m06-gradient-boosting-and-xgboost
• mlz-m06-xgboost-parameter-tuning
• mlz-m08-installation-of-tensorflow
• mlz-m10-creating-a-pre-processing-service

Sources

• Video
• Lesson file