Course Wiki

Feature importance: Mutual information — Machine Learning Zoomcamp Module 3

Mutual information is a concept from information theory, which measures how much we can learn about one variable if we know the value of another. In this project, we can think of this as how much do we learn about churn if we have the...

Machine Learning Zoomcamp › Module 3: Machine Learning for Classification › Feature importance: Mutual information

Notes

Mutual information is a concept from information theory, which measures how much we can learn about one variable if we know the value of another. In this project, we can think of this as how much do we learn about churn if we have the information from a particular feature. So, it is a measure of the importance of a categorical variable.

Classes, functions, and methods:

• mutual_info_score(x, y) - Scikit-Learn class for calculating the mutual information between the x target variable and y feature.
• df[x].apply(y) - apply a y function to the x series of the df dataframe.
• df.sort_values(ascending=False).to_frame(name='x') - sort values in an ascending order and called the column as x.

The entire code of this project is available in this jupyter notebook.

• Notes from Peter Ernicke

Key concepts

• Pandas
• scikit-learn
• Jupyter Notebooks

Related notes

• mlz-m01-introduction-to-numpy
• mlz-m01-introduction-to-pandas
• mlz-m01-linear-algebra-refresher
• mlz-m01-setting-up-the-environment
• mlz-m01-summary
• mlz-m02-baseline-model-for-car-price-prediction-project
• mlz-m02-categorical-variables
• mlz-m02-computing-rmse-on-validation-data
• mlz-m02-data-preparation
• mlz-m02-exploratory-data-analysis
• mlz-m02-explore-more
• mlz-m02-feature-engineering
• mlz-m02-linear-regression
• mlz-m02-linear-regression-vector-form
• mlz-m02-regularization
• mlz-m02-root-mean-squared-error-rmse
• mlz-m02-setting-up-the-validation-framework
• mlz-m02-training-linear-regression-normal-equation
• mlz-m02-tuning-the-model
• mlz-m02-using-the-model
• mlz-m03-churn-prediction-project
• mlz-m03-data-preparation
• mlz-m03-eda
• mlz-m03-explore-more
• mlz-m03-feature-importance-churn-rate-and-risk-ratio
• mlz-m03-feature-importance-correlation
• mlz-m03-logistic-regression
• mlz-m03-model-interpretation
• mlz-m03-one-hot-encoding
• mlz-m03-setting-up-the-validation-framework
• mlz-m03-summary
• mlz-m03-training-logistic-regression-with-scikit-learn
• mlz-m03-using-the-model
• mlz-m04-accuracy-and-dummy-model
• mlz-m04-confusion-table
• mlz-m04-cross-validation
• mlz-m04-roc-auc
• mlz-m04-roc-curves
• mlz-m04-summary
• mlz-m05-python-virtual-environment-pipenv
• mlz-m06-data-cleaning-and-preparation
• mlz-m06-decision-tree-learning-algorithm
• mlz-m06-decision-trees
• mlz-m06-decision-trees-parameter-tuning
• mlz-m06-ensemble-learning-and-random-forest
• mlz-m06-gradient-boosting-and-xgboost
• mlz-m06-xgboost-parameter-tuning
• mlz-m08-installation-of-tensorflow
• mlz-m09-explore-more
• mlz-m10-creating-a-pre-processing-service

Sources

• Video
• Lesson file