Course Wiki

Data cleaning and preparation — Machine Learning Zoomcamp Module 6

In this section we clean and prepare the [dataset](https://github.com/gastonstat/CreditScoring/raw/master/CreditScoring.csv) for the model which involves the following steps:

Machine Learning Zoomcamp › Module 6: Decision Trees and Ensemble Learning › Data cleaning and preparation

Notes

In this section we clean and prepare the dataset for the model which involves the following steps:

• Download the data from the given link.
• Reformat categorical columns (status, home, marital, records, and job) by mapping with appropriate values.
• Replace the maximum value of income, assests, and debt columns with NaNs.
• Replace the NaNs in the dataframe with 0 (will be shown in the next lesson).
• Extract only those rows in the column status who are either ok or default as value.
• Split the data in a two-step process which finally leads to the distribution of 60% train, 20% validation, and 20% test sets with random seed to 11.
• Prepare target variable status by converting it from categorical to binary, where 0 represents ok and 1 represents default.
• Finally delete the target variable from the train/val/test dataframe.

Add notes from the video (PRs are welcome)

• Notes from Peter Ernicke
• Notes from Peter Ernicke

Key concepts

• Pandas

Related notes

• mlz-m01-introduction-to-pandas
• mlz-m01-setting-up-the-environment
• mlz-m01-summary
• mlz-m02-categorical-variables
• mlz-m02-data-preparation
• mlz-m02-exploratory-data-analysis
• mlz-m02-setting-up-the-validation-framework
• mlz-m03-data-preparation
• mlz-m03-eda
• mlz-m03-feature-importance-churn-rate-and-risk-ratio
• mlz-m03-feature-importance-correlation
• mlz-m03-feature-importance-mutual-information
• mlz-m03-setting-up-the-validation-framework
• mlz-m06-credit-risk-scoring-project
• mlz-m06-decision-tree-learning-algorithm
• mlz-m06-decision-trees
• mlz-m06-decision-trees-parameter-tuning
• mlz-m06-ensemble-learning-and-random-forest
• mlz-m06-explore-more
• mlz-m06-gradient-boosting-and-xgboost
• mlz-m06-selecting-the-best-model
• mlz-m06-summary
• mlz-m06-xgboost-parameter-tuning

Sources

• Video
• Lesson file