Welcome to Week 14!

  • Quick overview of today’s plan:

    • Wrapping up the term: What you’ve learned & where to go next
    • Course reflection & feedback discussion
    • Final project launch & expectations

Wrapping Up the Term

Your Learning Journey

Modules 1-6: Foundations

  • Python fundamentals
  • Jupyter notebooks & data structures
  • Pandas for data manipulation
  • Importing & cleaning data
  • Data aggregation & joins
  • Visualization (from simple to interactive)
  • Control flow & functions

Modules 7-13: Advanced Skills

  • Machine learning basics
  • Stakeholder management
  • Supervised vs. unsupervised learning
  • Regression vs. classification modeling
  • ML workflow considerations

Important

You started from scratch and built a comprehensive data science toolkit!

What You’ve Mastered: Supervised Learning

Regression Models

  • Linear regression
  • Decision trees for regression
  • Random forests for regression

Classification Models

  • Logistic regression
  • Decision trees for classification
  • Random forests for classification

Model Workflow

  • Train/test splits
  • Cross-validation
  • Hyperparameter tuning
  • Feature engineering
  • Model evaluation metrics
  • Avoiding overfitting

The 80/20 Rule

These classical ML methods solve 80% of real-world business problems. You’ve learned the workhorse algorithms of professional data science!

What You’ve Mastered: Unsupervised Learning

Clustering

  • K-means algorithm
  • Elbow method for k selection
  • Cluster profiling
  • Business applications

Dimensionality Reduction

  • Principal Component Analysis (PCA)
  • Variance explained
  • Feature compression
  • Visualization in 2D/3D
  • As a feature engineering tool

Important

These unsupervised techniques help you find hidden patterns without labeled data. Very valuable for business applications like customer segmentation and understanding complex datasets.

Real World Datasets

You’ve applied these skills across diverse domains:

Datasets You’ve Worked With

  • Housing Data – Predicting home prices, exploring market trends
  • Retail Shopping Data – Customer behavior analysis, purchase patterns
  • College Admissions – Classification modeling, decision factors
  • Flight Data – Delay analysis, operational patterns
  • Health Data – Medical outcomes, patient characteristics
  • And more! – Each bringing unique challenges and insights

What This Means

  • You’ve seen ML applied to real business problems, not just toy datasets
  • Each domain required different approaches and business understanding
  • You’ve practiced translating technical results into actionable insights
  • These examples mirror what you’ll encounter in professional data science roles

Transferable Skills

The ability to work with diverse datasets from different domains is exactly what employers value—you can adapt your skills to any industry!

Beyond What We Covered: Modern ML

This week’s readings introduce you to the broader ML landscape:

Gradient Boosting

  • XGBoost, LightGBM, CatBoost
  • Sequential ensemble learning
  • Often wins Kaggle competitions
  • Requires careful tuning

Neural Networks & Deep Learning

  • Foundation of modern AI
  • CNNs for computer vision
  • RNNs/Transformers for NLP
  • Generative AI (GPT, DALL-E)
  • Foundation models

Exposure, Not Mastery

Chapter 33 gives you a guided tour of modern ML so you understand what’s possible and how it connects to what you’ve learned. This will prepare you to dive deeper when your career requires it.

The ML Algorithm Landscape

How different algorithms relate to each other:

%%{init: {'theme':'base', 'themeVariables': { 'fontSize':'14px'}}}%%
graph LR
    A[Machine Learning] --> B[Supervised Learning]
    A --> C[Unsupervised Learning]

    B --> D[Regression]
    B --> E[Classification]

    D --> F[Linear Regression]
    D --> G[Decision Trees]

    E --> H[Logistic Regression]
    E --> I[Decision Trees]

    G --> J[Random Forest]
    G --> K[Gradient Boosting]

    I --> J
    I --> K

    F -.->|weights| L[Deep Learning]
    H -.->|activation| L
    K -.->|backprop| L

    C --> M[Clustering]
    C --> N[Dimension Reduction]

    M --> O[K-Means]
    N --> P[PCA]

    style A fill:#e1f5dd
    style B fill:#e1f5dd
    style C fill:#e1f5dd
    style D fill:#e1f5dd
    style E fill:#e1f5dd
    style M fill:#e1f5dd
    style N fill:#e1f5dd

    style F fill:#fff4cc
    style H fill:#fff4cc
    style G fill:#fff4cc
    style I fill:#fff4cc
    style J fill:#fff4cc
    style O fill:#fff4cc
    style P fill:#fff4cc

    style K fill:#d4e6f1
    style L fill:#d4e6f1

Note

Big concepts (Green) | Algorithms you’ve learned & practiced (Yellow) | Algorithms introduced this week (Blue)

Where to Go From Here: Chapter 34 Roadmap

Eight major directions for continued learning:

  1. Deep Learning & Neural Networks (images, text, audio)
  2. Natural Language Processing (text analysis, chatbots)
  3. Computer Vision (image recognition, object detection)
  4. MLOps & Deployment (production systems)
  1. Time Series Forecasting (sales, demand, trends)
  2. Reinforcement Learning (game AI, robotics)
  3. Responsible AI & Ethics (fairness, interpretability)
  4. AutoML & Advanced Feature Engineering

You Don’t Need to Master Everything!

Pick one area that excites you and dive deep. Chapter 34 provides learning paths, resources, and career guidance for each direction.

Career Paths in Data Science

Different roles require different skill blends:

Role Primary Focus Your Readiness
Data Analyst Business insights, visualization ✅ Strong foundation
Data Scientist Predictive modeling, analytics ✅ Ready to start
ML Engineer Model deployment, production systems 🟡 Need MLOps skills
Data Engineer Data pipelines, infrastructure 🟡 Need SQL/Spark
AI Engineer LLM integration, AI applications 🟡 Need foundation model skills
Research Scientist Algorithm development, research 🔴 Need advanced math/PhD

You’re positioned well for Data Analyst and Data Scientist roles!

You’re immediately qualified for analyst roles and junior data scientist positions. With focused additional learning, they can grow into specialized roles like ML Engineer or AI Engineer.

Building Your Portfolio

The best way to demonstrate your skills:

What Makes a Strong Portfolio Project?

  1. End-to-End Execution (problem → data → model → insights)
  2. Real Problem (not Titanic or Iris datasets!)
  3. Clear Documentation (README, comments, results)
  4. Technical Depth (proper evaluation, handles real-world messiness)
  5. Professional Presentation (organized GitHub repo)

Where to Host:

  • GitHub (essential!)
  • Personal blog/website
  • Kaggle competitions
  • LinkedIn project posts
  • YouTube demos

Quality Over Quantity

Three well-executed projects beat ten rushed projects. Show depth and follow-through! Get inspiration: https://github.com/topics/data-science-portfolio

Resources for Continued Learning

Curated recommendations from Chapter 34:

Online Courses

  • Fast.ai: Deep Learning for Coders (free)
  • Andrew Ng: ML & Deep Learning Specializations
  • Google’s MLOps Course (free)
  • Hugging Face NLP Course (free)

Books

  • “Hands-On ML” by Aurélien Géron
  • “Deep Learning with Python” by François Chollet
  • “Forecasting” by Hyndman & Athanasopoulos (free)

Communities

  • Kaggle (competitions, datasets, notebooks)
  • Stack Overflow (Q&A)
  • Reddit: r/MachineLearning, r/datascience
  • Papers with Code (research + implementations)

Practice Platforms

  • Kaggle competitions
  • DrivenData (social impact)
  • Personal projects on GitHub

Key Takeaways: Looking Forward

What You Know

  • Python fundamentals ✅
  • Data wrangling with pandas ✅
  • Visualization best practices ✅
  • Classical ML algorithms ✅
  • Proper evaluation workflow ✅
  • Feature engineering ✅
  • Model interpretation ✅

Growth Mindset

  • Algorithm choice matters less than you think
  • Feature engineering often beats algorithm swapping
  • Start simple, add complexity only when needed
  • The best model is the simplest one that meets requirements
  • You can’t know everything—adopt just-in-time learning

The 80/20 Rule Revisited

The classical methods you’ve mastered solve 80% of business problems. Deep expertise in fundamentals beats surface knowledge of every new technique.

Course Reflection & Feedback

Let’s Talk About This Course

Your feedback helps improve BANA 4080 for future students

Discussion Prompts

What went well?

  • Which topics or modules were most valuable?
  • What teaching methods helped you learn best?
  • Which assignments were most effective?
  • What did this course excel at that other courses lack?

What could be improved?

  • Which topics felt rushed or unclear?
  • What would you change about the pace?
  • What additional topics would you have liked?
  • What did this course lack that other courses have excelled at?

Open floor for any other thoughts!

Formal Course Evaluation Reminder

End-of-Term Survey

There is a formal course evaluation survey posted on Canvas

  • Called “Anthology Course Evaluations”
  • Please complete by end of week
  • Your feedback is anonymous and extremely valuable
  • Helps me improve the course structure, content, and delivery
  • Influences departmental decisions about course offerings

Please take 10-15 minutes to complete it thoughtfully!

Your honest feedback—both positive and constructive—directly shapes how this course evolves for future students.

Final Project Launch

Final Project Overview

Customer Churn Prediction at Regork Telecom: You are a data scientist at Regork Telecom tasked with predicting customer churn to help the company save millions in retention costs.

Three-Part Structure

  • Part 1 (35 pts): Exploratory Data Analysis
  • Part 2 (35 pts): Structured Predictive Modeling
  • Part 3 (30 pts): Open-Ended Modeling Competition

Key Details

  • Total Points: 100
  • Format: Single Jupyter Notebook + Quiz
  • Dataset: 5,999 training customers + 1,000 test customers
  • Target: Predict Status (Current vs. Left)

Individual Assignment

This is an individual project. You may discuss concepts with classmates, but all code and analysis must be your own work.

The Dataset

Two files available on Canvas:

1. customer_retention_training.csv (5,999 customers)

Clean Features (Signal)

  • Demographics: Age, Partner, Dependents
  • Account: Tenure, Contract, Monthly Charges
  • Services: Internet, Phone, Tech Support
  • Usage: App logins, Email opens, Page views

Noisy Features (Potential Noise)

  • RandomIDHash (random hash)
  • InternalCodeFlag (internal system flag)
  • SurveyVersion (A/B test)
  • LegacySystemScore (old metric)
  • PromoCategory (unclear)

2. test_features.csv (1,000 customers, no labels)

  • Used in Part 3 for competition predictions

Part of the Challenge!

The data engineering team dumped everything into this file. Part of your job is distinguishing valuable features from noise.

Part 1: Exploratory Data Analysis (35 pts)

Goal:

Understand the customer base and identify patterns between churn and customer features/behaviors

What You’ll Do:

  • Calculate key statistics (churn rates, means by groups)
  • Compare churned vs. stayed customers across features
  • Identify features that appear to have a strong relationship with churn (both categorical and numeric)
  • Check data quality (missing values)
  • Distinguish signal from noise

Deliverables:

  • 13 Quantitative Questions (calculations, statistics)
  • 4 Conceptual/Essay Questions including:
    • Feature correlation essay (6 pts): Identify top 2 features with quantitative evidence
    • Limitations essay (6 pts): Discuss dataset/analysis limitations

This Sets Up Everything Else

Your EDA insights directly inform which features to keep/drop in Parts 2 and 3!

Part 2: Structured Predictive Modeling (35 pts)

Goal:

Demonstrate you can build and evaluate models following a strict, reproducible workflow

Required Workflow (No Shortcuts!):

  1. Train/test split (70/30, stratified, random_state=42)
  2. Preprocessing pipeline (imputation + scaling/encoding)
  3. 5-fold stratified cross-validation
  4. Hyperparameter tuning with GridSearchCV (scoring='f1')
  5. Build three models: Logistic Regression, Decision Tree, Random Forest
  6. Evaluate with Accuracy, Precision, Recall, F1 Score
  7. Analyze feature importance (Permutation Importance)

Key Constraint: Must use ALL columns to demonstrate handling noisy data

Industry Best Practices

This part forces you to follow professional ML workflow standards—reproducibility, proper evaluation, and avoiding data leakage.

Part 2 Continued: Question Types

Part 2 has ~18 questions across multiple formats:

Modeling Output Questions (~13 questions)

  • Multiple choice (e.g., “Why do we encode the target variable?”)
  • Numeric answers (e.g., “What is the test set F1 score?”)
  • True/False (e.g., “Is there evidence of overfitting?”)
  • Short answer (e.g., “List the top 5 important features”)

These test your understanding of:

  • Workflow steps (train/test split, pipelines)
  • Cross-validation results
  • Model performance metrics
  • Feature importance analysis

Essay Questions (2 essays, 5 pts each)

Essay 1: Feature Interpretation

  • Explain top 2 features from permutation importance
  • Translate technical findings into business meaning
  • Provide actionable recommendations

Essay 2: Model Recommendation

  • Compare the three models
  • Discuss metrics (F1, Precision, Recall)
  • Consider generalization and overfitting
  • Make a clear recommendation with justification

Communication Matters!

The essays test your ability to explain technical results to business stakeholders (like a CEO). Avoid jargon and focus on actionable insights.

Part 3: Open-Ended Modeling Competition (30 pts)

Goal:

Build the best possible churn prediction model—guardrails are OFF!

What Changes in Part 3?

  • ❌ No workflow restrictions
  • ❌ No required algorithms
  • ❌ Don’t have to use all features
  • ✅ Remove noise based on your EDA
  • ✅ Try advanced algorithms (XGBoost, LightGBM)
  • ✅ Engineer new features
  • ✅ Ensemble multiple models
  • ✅ Optimize for ROC AUC (not F1!)

Grading: The Curve

\[\text{Score} = 25 + 5 \times \frac{AUC_i - AUC_{avg}}{AUC_{best} - AUC_{avg}}\]

  • Baseline: Class average = 25/30 pts
  • Ceiling: Best in class = 30/30 pts
  • Floor: Minimum = 15/30 pts

Deliverable: predictions.csv

Generate probability predictions for 1,000 test customers and submit as CSV file.

Part 3: Strategies for Success

How to beat the baseline and maximize your score:

1. Remove Noise

  • Drop features your EDA showed were irrelevant
  • This is the easiest performance boost!

2. Try Other Models

  • Regularized regression (i.e. sklearn.linear_model.Ridge)
  • XGBoost, LightGBM, CatBoost
  • Use libraries: xgboost, lightgbm

3. Tune Hyperparameters

  • Investigate more hyperparameter settings
  • More iterations = better tuning
  • Monitor cross-validation scores

4. Ensemble Models

  • Average predictions from multiple models
  • Example: 50% Random Forest + 50% XGBoost
  • Often beats any single model

Don’t Overfit!

If your training score is 0.99 but CV score is 0.75, you’re overfitting. Trust your cross-validation score!

Part 3: Submission

What you’ll submit:

Single CSV file containing predicted values, which I will compare to a holdout set.

  1. Use your final model to make predictions on the test_features.csv (1,000 customers, no labels).
  2. Save these predictions to a CSV file
    • Filename: predictions.csv
    • Format: 1,001 lines (header + 1,000 predictions)
    • Columns: id (1-1000), prediction (probability 0.0-1.0)
    • No index column! Use index=False when saving

Example:

id,prediction
1,0.8234
2,0.1456
...
1000,0.2341

Tip

Helper code provided for generating predictions

Use this template to create your predictions.csv file:

# 1. Retrain your best model on FULL training data (no split)
final_model = your_best_pipeline.fit(X, y)

# 2. Load test data
test_df = pd.read_csv('test_features.csv')

# 3. Generate PROBABILITY predictions (not class predictions!)
# Use .predict_proba()[:, 1] to get probability of "Left" (Class 1)
pred_probs = final_model.predict_proba(test_df)[:, 1]

# 4. Create submission DataFrame
submission = pd.DataFrame({
    'id': range(1, 1001),
    'prediction': pred_probs
})

# 5. Save to CSV (CRITICAL: index=False)
submission.to_csv('predictions.csv', index=False)

print("Success! predictions.csv is ready to upload.")

Common Mistake: Using .predict() instead of .predict_proba()

Part 3 requires probabilities (0.0 to 1.0), not binary predictions (0 or 1)!

Submission Requirements

You will submit TWO files via the Final Project Quiz on Canvas:

  1. Jupyter Notebook
    • Filename: lastname_firstname_final_project.ipynb
    • Must contain all three parts with clear section headers
    • All cells must be executed (outputs visible)
    • Code must run top-to-bottom without errors
    • Up to 10 points deducted for poor organization

Tip

Complete your entire notebook before opening the quiz to avoid timing out!

  1. Predictions File (Part 3)
    • Filename: predictions.csv

Timeline & Recommendations

Recommended workflow for success:

Phase Task Time Estimate
1 Complete Part 1 (EDA) thoroughly 1-2 hours
2 Complete Part 2 (Structured Modeling) 1-2 hours
3 Part 3 experiments + final tuning 1-n hours
4 Final review, submission prep <1 hour

Start Early!

Students who rush Part 1 struggle in Parts 2 and 3. Take time to understand the data deeply—it pays off!

Office Hours & Resources:

  • Office hours: Let’s discuss
  • Discussion forum for clarification questions
  • Canvas has complete project writeup and datasets

Grading Rubric Summary

Component Points Key Success Criteria
Part 1: EDA 35 • Accurate calculations
• Essays cite specific numbers
• Business-focused language
Part 2: Modeling 35 • Follows exact workflow
• Proper evaluation metrics
• Clear model recommendation
Part 3: Competition 30 • Curved based on ROC AUC
• Valid predictions file
• Documented experiments
Notebook Quality -10 (penalty) • Clear section headers
• All cells executed
• Runs top-to-bottom

Two Common Pitfalls to Avoid

  1. Skipping EDA: Part 1 insights are crucial for Parts 2 & 3
  2. Poor Documentation: If we can’t understand your notebook, we can’t give full credit

Take time to organize, comment, and explain your work!

Questions & Final Thoughts

What Makes a Great Final Project?

Beyond just getting a good grade—build something portfolio-worthy:

Technical Excellence

  • Thorough, insightful EDA
  • Proper ML workflow (no data leakage!)
  • Experiments documented clearly
  • Multiple approaches tried in Part 3
  • Code is clean and commented

Professional Documentation

  • Clear section headers and markdown
  • Visualizations that communicate insights
  • Essays written for business audience
  • Organized notebook explaining your approach
  • Can be shown to potential employers!

Portfolio Opportunity

If you document this project well, you can add it to your GitHub portfolio and discuss it in job interviews!

Connection to Your Career

This project mirrors real-world data science work:

Skills You’re Demonstrating

  • Exploratory data analysis
  • Distinguishing signal from noise
  • Building reproducible ML pipelines
  • Proper model evaluation
  • Hyperparameter tuning
  • Business communication
  • Competitive performance optimization

How This Helps Your Career

  • Interviews: Concrete project to discuss
  • Portfolio: Showcase end-to-end ML workflow
  • Skills: Everything employers look for
  • Confidence: You can handle real projects

This Is What Data Scientists Do!

Customer churn prediction is a real, high-value business problem. Companies pay data scientists to solve exactly this type of challenge.

Final Project Q&A

Common questions—let’s address them now:

Clarification Topics

  • Datasets
  • Part 1: EDA approach and essay expectations
  • Part 2: Exact workflow steps and pipeline setup
  • Part 3: Allowed techniques and submission format
  • Grading: How the curve works and rubric details
  • Timeline: When to start and how to allocate time
  • Resources: Where to get help if stuck

What questions do you have?

Start Early, Ask Questions, Document Well

These three habits will ensure your success on the final project!

Looking Ahead


This Week:

  • Start Part 1 (EDA) immediately
  • Review Chapters 33-34 for broader ML context
  • No additional homework—focus on final project
  • Course evaluation survey (please complete!)

Next Week (Week 15):

  • Final project due via Canvas Quiz (Wednesday, Dec 10th)
  • Complete all three parts + notebook + predictions file


Good luck on your final projects—I’m excited to see what you build!