Questions from Week 8?

• Midterm project feedback?
• Machine learning concepts and project planning?
• Anything confusing in the quiz or class lab?
• Time to ask!

Why Relationships Matter

In business, we rarely care about a single number in isolation. Leaders ask questions like:

• Does increasing marketing spend actually increase sales?
• Are higher salaries associated with better employee retention?
• Do customers in certain regions spend more per transaction?
• Which factors drive customer satisfaction scores?

Today’s Goal: Learn to measure and model these relationships quantitatively.

Think-Pair-Share: Business Relationships

Then we’ll take a few responses…

What is Correlation?

Team Challenge: Guess the Correlation!

Challenge Results & Key Insights

Congratulations to our winning team! 🏆

Key Takeaways from the Challenge:

• Visual patterns help us estimate correlation strength
• Business intuition often aligns with statistical relationships
• Perfect correlations (±1.0) are rare in real business data
• Direction matters: Positive vs negative relationships tell different stories

Next: How do we move from measuring relationships to making predictions?

Let’s Practice: Grocery Chain Data

Your turn! Let’s analyze some real grocery chain data.

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np

# Example dataset: advertising spend vs. weekly sales
data = pd.DataFrame({
    "ad_spend": [400, 500, 600, 700, 800, 900, 1000, 1100, 1200, 1300, 1400, 1500, 1600, 1700, 1800, 1900, 2000, 2100, 2200, 2300],
    "weekly_sales": [4200, 4400, 4100, 4800, 5600, 5200, 4900, 5500, 5300, 5900, 5700, 6300, 6900, 6200, 5800, 6600, 7100, 6800, 7300, 7800]
})

# Visualize the relationship
plt.figure(figsize=(8, 5))
plt.scatter(data["ad_spend"], data["weekly_sales"], alpha=0.7, s=60)
plt.xlabel("Advertising Spend ($)")
plt.ylabel("Weekly Sales")
plt.title("Ad Spend vs. Weekly Sales")
plt.grid(True, alpha=0.3)
plt.show()

Question for you: What do you think the correlation is? Write down your guess!

Computing Correlation

Let’s see how close your guess was!

# Compute the correlation coefficient
correlation = data["ad_spend"].corr(data["weekly_sales"])
print(f"Correlation coefficient: {correlation:.3f}")

# Or we can see the full correlation matrix
print("\nFull correlation matrix:")
data.corr()

Correlation coefficient: 0.941

Full correlation matrix:

Interpretation:

• Strong positive correlation (~0.9)
• As advertising spend increases, sales tend to increase
• Remember: Correlation ≠ causation!

From Correlation to Prediction

We found a strong correlation (~0.9) between advertising and sales.

But now the business question is: Can we use advertising spend to predict future sales?

Question: If you could draw a line to represent this relationship, what would it look like?

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np

# Visualize the relationship
plt.figure(figsize=(8, 5))
plt.scatter(data["ad_spend"], data["weekly_sales"], alpha=0.7, s=60)
plt.xlabel("Advertising Spend ($)")
plt.ylabel("Weekly Sales")
plt.title("Ad Spend vs. Weekly Sales")
plt.grid(True, alpha=0.3)
plt.show()

The Linear Regression Formula

Remember from high school? The equation of a line:

\[ y = mx + b \]

Fitting Our Regression Model

from sklearn.linear_model import LinearRegression

# Prepare the data
X = data[["ad_spend"]]  # Feature matrix (note the double brackets)
y = data["weekly_sales"]  # Target variable

# Fit the model
model = LinearRegression()
model.fit(X, y)

# Extract the results
intercept = model.intercept_
slope = model.coef_[0]

print(f"Intercept (β₀): ${intercept:.0f}")
print(f"Slope (β₁): ${slope:.2f}")

Intercept (β₀): $3552
Slope (β₁): $1.68

Our fitted equation: \[ \text{Weekly Sales} = 3552 + 1.68 \times \text{Ad Spend} \]

Interpretation: For every $1 increase in advertising, we expect weekly sales to increase by $1.68!

Visualizing Our Predictions

# Create the regression line visualization
plt.figure(figsize=(10, 6))
plt.scatter(data["ad_spend"], data["weekly_sales"], alpha=0.7, s=60, label="Actual data")

# Add the fitted regression line
plt.plot(data["ad_spend"], model.predict(X), color="red", linewidth=3, label="Regression line")

# Show specific predictions
pred_x = [1500, 2000]
for x_val in pred_x:
    pred_y = model.predict([[x_val]])[0]
    plt.scatter(x_val, pred_y, color="orange", s=120, zorder=5)
    plt.annotate(f"${x_val} → ${pred_y:.0f}", 
                xy=(x_val, pred_y), xytext=(5, -15), 
                textcoords='offset points', fontsize=10)

plt.xlabel("Advertising Spend ($)")
plt.ylabel("Weekly Sales")
plt.title("Linear Regression: Predictions on the Line")
plt.legend()
plt.grid(True, alpha=0.3)
plt.show()

Key insight: Every prediction falls exactly on our regression line!

Making Business Predictions

Business value: Now we can forecast sales for any advertising budget!

# Make predictions for different scenarios
scenarios = pd.DataFrame({
    'ad_spend': [1500, 2500, 3000]
})

predictions = model.predict(scenarios)

print("Business Predictions:")
for spend, pred in zip(scenarios['ad_spend'], predictions):
    print(f"Spend ${spend:,} → Predicted Sales: ${pred:,.0f}")
    
# Manual calculation to verify
print(f"\nManual check for $1,500:")
manual_pred = 1.68 * 1500 + 3552
print(f"1.68 × 1500 + 3552 = ${manual_pred:.0f}")

Business Predictions:
Spend $1,500 → Predicted Sales: $6,072
Spend $2,500 → Predicted Sales: $7,752
Spend $3,000 → Predicted Sales: $8,592

Manual check for $1,500:
1.68 × 1500 + 3552 = $6072

Critical Question

We now have a model that can make predictions…

• Advertising spend of $1,500 → $6,072 in sales
• Advertising spend of $2,500 → $7,752 in sales
  
• Advertising spend of $3,000 → $8,592 in sales

How would you assess prediction quality?

Key Evaluation Metrics

R² (R-squared): “What percentage of the variation does my model explain?”

• Range: 0 to 1 (higher is better)
• Example: R² = 0.85 means the model explains 85% of sales variation

RMSE (Root Mean Squared Error): “How far off are my predictions, on average?”

• Same units as your outcome (dollars, customers, etc.)
• Example: RMSE = $347 means predictions are typically off by $347

Computing Metrics with Scikit-Learn

Let’s see how to calculate these evaluation metrics using our advertising model:

from sklearn.metrics import r2_score, root_mean_squared_error

# Make predictions on our data
predictions = model.predict(X)

# Calculate evaluation metrics
r2 = r2_score(y, predictions)
rmse = root_mean_squared_error(y, predictions)

print(f"R² Score: {r2:.3f}")
print(f"RMSE: ${rmse:.0f}")

R² Score: 0.886
RMSE: $347

What do these numbers tell us about our model’s performance?

The Problem: Training Data Evaluation

Splitting Our Data

Let’s split our advertising data into training and test sets:

from sklearn.model_selection import train_test_split

# Split data: 70% training, 30% testing
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=30
)

# Check the sizes
print(f"Total data points: {len(X)}")
print(f"Training set: {len(X_train)} points ({len(X_train)/len(X):.1%})")
print(f"Test set: {len(X_test)} points ({len(X_test)/len(X):.1%})")

Total data points: 20
Training set: 14 points (70.0%)
Test set: 6 points (30.0%)

Important: random_state=30 ensures reproducible results!

Training and Evaluation

Now let’s train on training data and evaluate on both sets:

# Train on training data only
model_honest = LinearRegression()
model_honest.fit(X_train, y_train)

# Calculate RMSE on both sets
from sklearn.metrics import root_mean_squared_error
train_rmse = root_mean_squared_error(y_train, model_honest.predict(X_train))
test_rmse = root_mean_squared_error(y_test, model_honest.predict(X_test))

print(f"Training RMSE: ${train_rmse:.0f}")
print(f"Testing RMSE: ${test_rmse:.0f}")
print(f"Difference: ${test_rmse - train_rmse:.0f}")

Training RMSE: $330
Testing RMSE: $403
Difference: $73

Business Insight: The difference tells us how well our model will generalize to future data!

Notice: Test RMSE ($403) > Training RMSE ($330) - this is normal! Our model performs slightly worse on new data because it was optimized for the training set. However, the larger this difference becomes the more concern we should have on whether we have a good model or not.

Complete Workflow: Start to Finish

Let’s see the complete machine learning workflow using a fresh dataset:

# Step 1: Import a dataset
import pandas as pd
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score, root_mean_squared_error

# Load the classic Advertising dataset
advertising = pd.read_csv("../data/Advertising.csv")
print("Dataset shape:", advertising.shape)
advertising.head()

Dataset shape: (200, 4)

Step 1 complete: We have our data loaded and ready!

Step 2: Split the Data

# Step 2: Split data into features (X) and target (y), then train/test
X = advertising[['TV', 'radio', 'newspaper']]  # Multiple predictors
y = advertising['sales']  # Target variable

# Split into train/test sets
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

print(f"Training set: {len(X_train)} samples")
print(f"Test set: {len(X_test)} samples")

Training set: 160 samples
Test set: 40 samples

Step 2 complete: Data is properly split for honest evaluation!

Step 3: Train the Model

# Step 3: Train our regression model
model = LinearRegression()
model.fit(X_train, y_train)

# Look at our model coefficients
print("Model coefficients:")
for feature, coef in zip(X.columns, model.coef_):
    print(f"  {feature}: {coef:.3f}")
print(f"Intercept: {model.intercept_:.3f}")

Model coefficients:
  TV: 0.045
  radio: 0.189
  newspaper: 0.003
Intercept: 2.979

Step 3 complete: Model is trained and ready to make predictions!

Step 4: Evaluate Performance

# Step 4: Evaluate model performance
train_predictions = model.predict(X_train)
test_predictions = model.predict(X_test)

# Calculate metrics
train_r2 = r2_score(y_train, train_predictions)
test_r2 = r2_score(y_test, test_predictions)
train_rmse = root_mean_squared_error(y_train, train_predictions)
test_rmse = root_mean_squared_error(y_test, test_predictions)

print("Performance Summary:")
print(f"Training R²: {train_r2:.3f} | Training RMSE: {train_rmse:.2f}")
print(f"Test R²: {test_r2:.3f} | Test RMSE: {test_rmse:.2f}")

Performance Summary:
Training R²: 0.896 | Training RMSE: 1.64
Test R²: 0.899 | Test RMSE: 1.78

Complete workflow achieved! 🎉 We can now trust our model’s performance estimates.

Additional Concepts You’ll Learn

We covered the core concepts, but there’s more to explore in your readings: