Welcome to Week 5

Three things today:

  1. Your Visualization Toolkit — Pandas, Seaborn, Matplotlib, and Bokeh, and when to reach for each
  2. EDA Framework — how to think through an analysis, not just run code
  3. The Midterm Project — the scenario, deliverables, and Thursday’s lab

Today is about orientation, not memorization. The companion notebook has every example we see — plus the ones we skip.

📓 Follow Along in the Notebook

Week 5 Lecture Notebook

Open the companion notebook in Colab now. Everything we demo today is in there, runnable — along with cheat sheets for all four libraries and extra practice we won’t get to in class.

Open In Colab

Questions from Week 4?

  • Manipulating & cleaning DataFrames?
  • Aggregating data with groupby?
  • Joining datasets to bring in context?
  • Anything from the quiz or lab?


Anything still fuzzy from last week — now is the time.

Part 1: Your Visualization Toolkit

How Quickly Can You Spot Insights?

What can you tell from this table?

Week Produce Dairy Frozen Deli
1 11,500 12,100 7,320 4,100
2 12,800 11,400 7,180 4,080
3 12,200 10,800 1,780 4,120
4 13,900 10,100 7,410 4,090
5 13,400 9,800 7,260 4,110
6 15,100 9,200 7,350 4,085
7 14,800 8,700 7,420 4,115
8 16,400 8,100 7,190 4,095

How Quickly Can You Spot Insights?

What can you tell from this plot?

Different Visualization Needs

Scenario 1

You just pulled the Complete Journey data and need to quickly check whether basket spend looks reasonable before writing any analysis code.

Scenario 2

You’re presenting weekly revenue trends to the VP of Marketing in a board meeting. The chart will appear in a formal slide deck.

Scenario 3

The promotions team wants a tool to interactively explore coupon redemption rates by store, product category, and week.

Same data. Three very different jobs. No single library is best at all three.

Match the Tool to the Goal

We should match how we visualize to the goal we have in mind.


Purpose Tool Why It Wins Here
Quick EDA Pandas .plot() One line, straight off a DataFrame
Statistical comparison Seaborn Handles grouping, ordering, and stats for you
Polished reporting Matplotlib Total control over every element
Interactive exploration Bokeh Renders to the browser — zoom, hover, filter

Tip

These are not four competitors. Pandas and Seaborn are both built on top of Matplotlib — so they stack.

Pandas: Speed

Pandas — When Speed Beats Polish

  • Your first reach when you need a fast answer — minimal code, tied directly to DataFrames
  • Call .plot() on any Series or DataFrame and pass kind=
  • Built on Matplotlib, so anything you learn later still applies
  • Not meant for final deliverables

Tip

If you can answer your question in one line, Pandas .plot() is the right tool. Rule of thumb: if you’re typing a third line of formatting, you may want to switch libraries.

Pandas in One Line

category_totals.plot(
    kind='barh', x='department', y='sales_value',
    figsize=(10, 3.5), legend=False,
    title='Total revenue by department (top 10)', xlabel='Total sales ($)'
)
plt.show()

Swap kind= to change the chart: 'hist', 'box', 'line', 'bar', 'scatter', 'barh'.

Seaborn: Statistical Grouping

Seaborn — When You’re Comparing Groups

  • Built on Matplotlib — cleaner syntax for common statistical chart types
  • Shines when you need grouping, ordering, and distributions with shape
  • One function call handles what Matplotlib needs 10–15 lines to produce


Pandas Seaborn
.plot(kind='hist') — basic bin counts histplot() — bins + KDE curve in one call
.groupby().mean().plot.bar() — manual aggregation boxplot() / barplot() — handles stats automatically
No built-in group ordering order= parameter on every chart

Every Seaborn function follows one pattern: sns.function(data=df, x=…, y=…, hue=…, order=…)

Seaborn in One Call

fig, ax = plt.subplots(figsize=(10, 3.8))
sns.boxplot(
    data=basket_demo,
    x='income',
    y='basket_spend',
    order=income_order,   # explicit ordering — critical for income brackets
    ax=ax,
    showfliers=False
)
ax.set_xticklabels(ax.get_xticklabels(), rotation=45, ha='right', fontsize=8)
ax.set_xlabel('Income bracket')
ax.set_ylabel('Basket spend ($)')
plt.tight_layout()
plt.show()

Matplotlib: Control

Matplotlib — When the Chart Must Stand Alone

  • The most widely used Python plotting library — and the foundation for Seaborn and Pandas
  • Full customization: labels, annotations, tick formatting, styles, multi-panel layouts, export
  • Because Pandas and Seaborn return Matplotlib objects, you can start fast and polish later

Tip

Use Matplotlib when the chart needs to stand on its own — in a report, a slide deck, or a published figure. This is the library your project report will live on.

The Figure / Axes Hierarchy

Everything in Matplotlib is an object you can reference and modify:

fig, ax = plt.subplots(figsize=(10, 2.5))
# fig = the overall canvas
# ax  = the actual plot area (axes, ticks, lines, labels)
fig.set_facecolor('#eaf4fb')
ax.set_facecolor('#d0eaf8')
ax.set_title('fig (light blue canvas) contains ax (darker blue plot area)')
plt.tight_layout()
plt.show()

Once you have fig and ax, you control everything — including plots Seaborn or Pandas drew for you.

Before: A Quick Pandas Plot

Fast, but it would not survive an executive meeting:

weekly_sales.plot(kind='line', x='week', y='total_sales', figsize=(10, 3.0), legend=False)
plt.show()

After: Executive-Ready Matplotlib

fig, ax = plt.subplots(figsize=(10, 3.5))
ax.plot(weekly_sales['week'], weekly_sales['total_sales'], linewidth=2)

# --- readable formatting ---
ax.set_title('Weekly total sales — note holiday spike in late December', pad=10)
ax.set_xlabel('Week')
ax.yaxis.set_major_formatter(mtick.StrMethodFormatter('${x:,.0f}'))
ax.grid(True, alpha=0.3)
for spine in ['top', 'right']:
    ax.spines[spine].set_visible(False)

# --- annotate the insight ---
peak_idx  = weekly_sales['total_sales'].idxmax()
peak_week = weekly_sales.loc[peak_idx, 'week']
peak_val  = weekly_sales.loc[peak_idx, 'total_sales']
ax.annotate(
    f'Holiday spike\n${peak_val:,.0f}',
    xy=(peak_week, peak_val),
    xytext=(peak_week - pd.Timedelta(weeks=8), peak_val * 0.97),
    arrowprops=dict(arrowstyle='->', lw=1.2),
    fontsize=9
)
plt.tight_layout()
plt.show()

Caution

Matplotlib is powerful but verbose. There is a massive example gallery, and AI tools are very good at adapting Matplotlib code. The notebook walks this build step by step.

After: Executive-Ready Matplotlib

Bokeh: Interactivity

Bokeh — When Others Need to Explore

  • Renders HTML/JavaScript — charts live in a browser, not just a notebook
  • Users can zoom, pan, hover, and filter without writing new code
  • Best for dashboards, stakeholder exploration tools, and shareable deliverables
Loading BokehJS ...

Interactive Weekly Sales Explorer

source = ColumnDataSource(weekly_sales)

p = figure(
    title='Total Weekly Sales — hover, zoom, and pan to explore',
    x_axis_type='datetime',
    width=750, height=350,
    tools='pan,wheel_zoom,box_zoom,reset,save'
)

p.line('week', 'total_sales', source=source,
       line_width=2, color='steelblue')

hover = HoverTool(tooltips=[
    ('Week',  '@week{%F}'),
    ('Sales', '@total_sales{$0,0}')
], formatters={'@week': 'datetime'})
p.add_tools(hover)

p.xaxis.axis_label = 'Week'
p.yaxis.axis_label = 'Total Sales ($)'

show(p)

Tip

Run this in the companion notebook to see the interactive version — it won’t fully render in a static slide.

Exercise 1 — Run & Compare 💻

Open the Exercise 1 section of the companion notebook. Every cell is ready to run — you don’t write any code today.

Run them top to bottom. Same Complete Journey data, drawn four ways:

  • Pandas — a one-line distribution, then a category comparison
  • Seaborn — that same comparison, now split by group
  • Matplotlib — the executive-ready version
  • Bokeh — the interactive version

Answer these two as you go:

  1. Which tool got to a usable chart with the least code?
  2. Which output would you actually put in front of the CEO?

5 minutes — run, look, compare.

Where You Actually Learn These 📚

Today was a tour. The readings are where you learn to drive.

Chapter What it teaches you Where it shows up in the project
13 — Pandas .plot() for fast exploratory charts How you’ll explore your own data
14 — Seaborn / Matplotlib / Bokeh statistical charts, full control, interactivity How you’ll build report-quality figures
15 — EDA a complete case study, start to finish The model for your entire report

20 of the project’s 75 points ride on your charts

EDA (10 pts) and Findings (10 pts) are graded on whether your visuals reveal something non-obvious and are clearly narrated. Code & Craft and Delivery lean on them too.

Nobody produces professional-looking figures on the first attempt. The reps come from working through these chapters and applying them to your own data.

Part 2: How to Think About EDA

You Already Have Most of What EDA Needs

You learned Which lets you
Subsetting & filtering (Wk 3) isolate the rows a question is actually about
groupby & aggregation (Wk 4) compute statistics at any level — household, basket, department, week
Joins (Wk 4) pull in the context that makes a comparison possible
Visualization (this week) see the shape, catch what’s wrong, show the finding

Aggregation tells you what the numbers are. Visualization tells you what they mean and helps you communicate them. Together, that is most of an EDA.

Important

What’s left is understanding how to ask a good question of our data.

That’s what the rest of Part 2 is about.

EDA Is a Conversation With Your Data

Exploratory Data Analysis is not a checklist of charts to run.

It is a sequence of questions — each answer raising the next question.

Data dump (avoid)

  • Plot every variable
  • Report every mean and median
  • Move on

Real EDA (aim for)

  • Start with a sharp question
  • Let findings drive the next step
  • Stop when you can tell a story

Important

The goal is not to show that you looked at the data. The goal is to find something worth saying.

A Framework for EDA

flowchart LR
    A["1️⃣ Question"] --> B["2️⃣ Structure"]
    B --> C["3️⃣ Distributions"]
    C --> D["4️⃣ Segmentation"]
    D --> E["5️⃣ Story"]
    style A fill:#2c3e50,color:#fff
    style B fill:#2980b9,color:#fff
    style C fill:#27ae60,color:#fff
    style D fill:#e67e22,color:#fff
    style E fill:#8e44ad,color:#fff


Step Question to ask
Question What do I actually want to know? Is it specific enough to know when I’ve answered it?
Structure What shape is the data? What do I need to join? Where are the missing values?
Distributions What does the outcome variable look like across all households? Any outliers?
Segmentation Does the pattern differ across groups? Where does the interesting variation live?
Story What changed? What surprised me? What should the decision-maker do next?

Step 1: Start With a Sharp Question

The question determines everything that follows.

Caution

Too broad: “How do customers shop?”

This can’t be answered — no unit of analysis, no metric, no comparison to make.

Tip

Sharp: “Do households that redeem coupons have larger basket spend than those that don’t — and does this differ across income brackets?”

This names: the unit (basket), the metric (spend), the comparison (coupon vs. no coupon), and the segmentation (income).

A sharp EDA question tells you which tables to join, which columns to compute, and what a result would look like.

Note

We are going to answer this exact question over the next four slides — one step of the framework at a time.

Step 2: Structure — What Do I Actually Have?

Our question compares baskets, but transactions gives us 1.5 million item rows. So the first move is structural: collapse to one row per basket, flag which ones used a coupon, then attach household demographics for the income half of the question.

Every one of those steps can quietly lose rows — so count what survives:

Baskets:                    155,848
  ... that used a coupon:   9,827  (6.3%)
Households:                 2,469

After joining demographics: 81,817 baskets  (52% kept)
  ... households:           801

Warning

The join cost us half the data. Only 801 households have demographics. Any income comparison we make describes those households — not all shoppers. Notice this now, not in your conclusion.

Step 3: Distributions — Which Summary Can I Trust?

Before comparing any groups, look at the outcome variable alone: what does a typical basket even look like?

The mean sits $12 above the median — a handful of enormous baskets drag it to the right.

Report the median — and say why you did

With skew this strong, the mean describes a basket almost nobody actually has. The median is what a typical trip looks like, and it barely moves when a few $300 stock-ups show up.

Put that reasoning in your own report — one sentence is enough: “We compare medians because basket spend is right-skewed.” It tells your reader the choice was deliberate rather than a default, and defending a judgment call like this is exactly what the EDA portion of the rubric rewards.

Step 4: Segmentation — Does the Pattern Hold?

And across income brackets? The lift shows up in every bracket — 100% to 290%, with no clean upward or downward pattern.

Note

Let’s look, rather than assert.

Step 4 (cont.): Does It Differ by Income?

Orange clears blue in every single bracket. Both series drift up with income — so the real question is whether the gap widens.

Note

It doesn’t. As a ratio the lift bounces from roughly 100% up to 290% with no clean direction. Richer households spend more per trip, coupon or not — but the coupon effect itself is not an income story. The second half of our question resolves to “no,” and ruling something out is progress, not failure.

Warning

And notice the shaded brackets: those medians rest on 5–11 households. That is the Step 2 join loss surfacing exactly where you’d be most tempted to read a trend.

Step 5: Build the Story — and the “So What” Test

A 3.3x gap is tempting. Push one question further before you recommend anything.

Spend per item is identical. Coupon baskets are not pricier — they are simply bigger trips.

What We Can — and Cannot — Say

The “So What” Test

So what? Coupon redemption marks the weekly stock-up trip, not a change in what people buy.

Caution

The recommendation we cannot make: “Send more coupons to grow basket size.”

Nothing here shows coupons caused bigger baskets. Shoppers planning a big trip may be the ones who bother to bring a coupon.


Next question: do the same households shop differently with and without a coupon? That comparison could support a causal claim. This one can’t.

Exercise 2 — What’s the Next Question? 🤔

We concluded: coupon baskets are 3.3x larger, but spend per item is identical — coupons mark stock-up trips.

Your VP of Promotions reads this and asks: “So should we mail more coupons?”

Name 3 follow-up analyses you would run before answering, and what each would tell you.

Discuss with your neighbor for 3 minutes.

Part 3: The Midterm Project

The Regork Scenario

You are a data scientist at Regork, a national grocery chain.

Your manager has asked you to identify a potential growth opportunity — an area where the company could invest future resources to increase revenue or profits.

Your task:

  1. Define a clear business question that investigates a specific growth opportunity
  2. Analyze the Complete Journey data to answer it
  3. Communicate your findings through:
    • A written analytic report (Jupyter Notebook → HTML)
    • A 3-minute recorded presentation for the CEO

Tip

Groups: 2–4 students. Join one of the pre-defined groups in Canvas → People → Mid-Term Project. You may not create your own group outside that list.

What a Sharp Business Question Looks Like

You just watched one get answered

“Do households that redeem coupons have larger basket spend than those that don’t — and does this differ across income brackets?”

That question survived contact with the data because it named four things up front:

It named Which told us
A unit — the basket what one row of our table had to be
A metric — spend what to compute, and that skew mattered
A comparison — coupon vs. not what the chart at Step 4 had to show
A segmentation — income what to check next, even though it came back empty

Notice it was answerable in four steps — and we knew we were done. That is the bar.

Three More Sharp Questions


Private label

“Do households with kids buy a higher share of private-label products than households without — and does that gap differ by department?”

Unit: household · Metric: private-label share of spend · Compare: kids vs. none · Segment: department

Trip behavior

“Do households that shop twice a week or more spend more over the year than occasional shoppers — or do they spread the same annual spend across more trips?”

Unit: household · Metric: annual spend & trip count · Compare: frequent vs. occasional · Segment: income

Promotions

“Do products featured in the weekly mailer sell more during the featured week — and does any lift carry into the week after?”

Unit: product-week · Metric: units sold · Compare: featured vs. not · Segment: department

Note

Notice the second one has the same shape as our coupon finding: “more spend” and “more trips” are different answers, and the question is built so you can tell them apart. Design that distinction in from the start — don’t discover it at the end.

What Strong Analysis Looks Like

A question worth asking — specific, answerable, and tied to a decision someone actually makes. You saw the standard in Part 2.

A named decision-maker — who acts on this, and what do they do differently on Monday? The VP of Promotions needs a recommendation, not a summary of what you computed.

Reasoning you can follow — structure, then distributions, then segmentation. Each step raises the next question. Show the reasoning, not just the outputs.

Honesty about limits — say what your analysis does not establish. The strongest projects name the recommendation they can’t yet make, and what would be needed to make it.

Tip

Every chart earns its place by making one point. Label the axes, use a readable scale, and narrate what the reader should see. Plots without interpretation are decoration.

Don’t guess at the bar — go look at it

The Canvas project page includes real report and presentation submissions from previous classes. They are the clearest picture you will get of what “strong” actually means.

Before you write anything, read two of them end to end and ask: what did they investigate, how did they narrate each chart, and where did they choose to stop? An hour spent there will do more for your grade than an extra hour of code.

Deliverables

A. Written Technical Report

  • Jupyter Notebook rendered to HTML
  • Must include: data preparation, business question, EDA, narrative findings, recommendations
  • Show relevant code; suppress warnings and unnecessary/distracting output
  • File naming: YYYY_BANA4080_groupXX_finalproject.html

B. Recorded Stakeholder Presentation

  • 3 minutes maximum — no code, executive audience
  • Covers: business problem, key findings, one clear recommendation
  • Recorded via Zoom or Kaltura and submitted through Canvas

Note

One group member submits both deliverables together. Submit early — large video files can take time to upload.

What Gets Graded — and Where to Find the Rubric

75 points total: written report (40) + recorded presentation (35).

Report — 40 pts

Section Pts
Introduction 5
Libraries 5
EDA 10
Summary 5
Code & Craft 15

Presentation — 35 pts

Section Pts
Introduction 5
Findings 10
Summary 5
Delivery 15

Canvas → Assignments → Mid-Term Project

Full description, the complete rubric, and example submissions from a previous course. Reading the rubric before you start is one of the highest-leverage things you can do.

Tip

EDA (10 pts) is where projects separate. One non-obvious pattern explained well beats five polished charts that say nothing new.

Thursday’s Lab: Project Ideation

In Thursday’s lab you will:

  1. Form your group (2–4 students) and join a Canvas group
  2. Brainstorm three distinct project ideas, each with a business problem, decision-maker, data plan, and success criteria
  3. Choose your most compelling idea and do a 2–3 minute stand-up pitch to the class

Tips for a Strong Pitch

  • Lead with the question, not the dataset
  • Name a specific decision-maker and what they would do with your findings
  • Describe what a meaningful finding would look like — not just “explore the data”

After lab: refine your idea into a formal proposal (the homework due this week).

Q&A 🙋

Open floor for questions on:

  • Today’s visualization tools or the exercise
  • The EDA framework and how it connects to the project
  • Project scope, deliverables, or the proposal
  • Anything from the reading or labs so far