Module 12: Model Optimization and Validation

By now you can build several kinds of model. This module is about building them well. You will learn why checking your test set repeatedly quietly corrupts it, and how cross-validation gives you an honest estimate without burning your final holdout. From there you will tune hyperparameters systematically rather than by trial and error, and finish with feature engineering — the encoding, scaling, and construction work that often improves a model more than switching algorithms ever would.

Learning Objectives

By the end of this module, you will:

  • Explain how repeated test-set evaluation leads to contamination and optimistic estimates
  • Implement k-fold cross-validation with cross_val_score() and cross_validate()
  • Compare models using cross-validation scores rather than test-set performance
  • Explain the bias–variance tradeoff and identify underfitting versus overfitting
  • Use GridSearchCV to search hyperparameters systematically, and compare grid, random, and Bayesian search
  • Encode categorical variables and scale numeric features appropriately
  • Create polynomial, interaction, and domain-driven features, and handle missing data deliberately
  • Assemble preprocessing and modeling into scikit-learn pipelines that prevent data leakage

Module Resources

Lecture

Resource Link
📊 Week 12 Slides View slides

Chapters & Notebooks

Each chapter has a companion Colab notebook with all the code from the chapter — ready to run, no installation required.

Chapter Topic Colab Notebook
28 Cross-Validation Open in Colab
29 Hyperparameter Tuning Open in Colab
30 Feature Engineering Open in Colab

Lab

Resource Link
🧪 Lab 12: Model Tuning Open in Colab

Reference

Resource Link
📋 Module 12 Cheat Sheet View cheat sheet