Questions from Week 2?

Let’s take a few minutes to address any questions or concerns from last week before diving into new material.

Try This!

Your Task: Find the average number of seats on aircraft manufactured by Embraer in 2004 or later

Let’s see what people found…

The Data Journey: From Disk to Detective Work

Python stores data in memory for fast analysis, but first we need to get it there!

Your Detective Toolkit: Pandas 🐼

Meet Pandas: Python’s most powerful tool for working with spreadsheet-like data

Let’s Import Our First Dataset!

Step 1: Import pandas and load the Ames housing data

import pandas as pd

# Load the real estate data straight from the course repo
url = "https://raw.githubusercontent.com/bradleyboehmke/uc-bana-7025/main/data/ames_raw.csv"
ames = pd.read_csv(url)

# Absolute path — full address from the root
pd.read_csv("/Users/jane/project/data/ames_raw.csv")

# Relative path — directions from where you are
pd.read_csv("../data/ames_raw.csv")

my_project/
├── notebooks/
│   └── analysis.ipynb  ← You are here
└── data/
    └── ames_raw.csv    ← Your data

Importing Data in Google Colab ☁️

The challenge: In Colab, your files aren’t on your local machine — they’re on a cloud VM.

Option 1: Load directly from a URL — what we just did

Because we read from a URL, that import ran the same way on your laptop and in Colab. This is the approach we’ll use most this term.

Option 2: Upload a local file — for data that isn’t online

from google.colab import files

# Opens a file picker — select a file from your computer
uploaded = files.upload()

Then read the uploaded file just like any local file:

ames = pd.read_csv('ames_raw.csv')

Size It Up Yourself

Pandas gives you a whole toolkit for getting to know a new dataset — run each of these in your notebook and read what comes back:

ames.shape       # How big is it?
ames.columns     # What are the columns called?
ames.dtypes      # What type is each column?
ames.head()      # What do the first rows look like?
ames.info()      # Types and missing values, all at once
ames.describe()  # Summary statistics for numeric columns

Attributes vs Methods: Your Detective Tools

Pop Quiz: Do you notice a difference between these commands?

ames.shape      # No parentheses
ames.columns    # No parentheses  
ames.dtypes     # No parentheses

ames.head()     # With parentheses
ames.info()     # With parentheses
ames.describe() # With parentheses

Why Inspect Before You Analyze

Those six commands take about ten seconds. Skipping them costs hours.

The Answer Revealed!

Same column, three different requests — look at how the output shape changes:

ames['SalePrice'].head(3)

0    215000
1    105000
2    172000
Name: SalePrice, dtype: int64

ames[['SalePrice']].head(3)

ames[['SalePrice', 'Year Built']].head(3)

The Two Dimensions of Subsetting

When analyzing data, you often want just a subset of your dataset:

Data Mystery #1 Solutions 🎯

# Check dataset dimensions
print(f"Dataset shape: {ames.shape}")
print(f"Number of houses: {ames.shape[0]}")

Dataset shape: (2930, 82)
Number of houses: 2930

# Find maximum sale price
highest_price = ames['SalePrice'].max()
print(f"Highest sale price: ${highest_price:,}")

Highest sale price: $755,000

# Calculate average year built
avg_year = ames['Year Built'].mean()
print(f"Average year built: {avg_year:.0f}")

Average year built: 1971

# Count the distinct values in the column
n_hoods = ames['Neighborhood'].nunique()
print(f"Unique neighborhoods: {n_hoods}")

Unique neighborhoods: 28

Filtering Rows: The Magic of Conditions

The Process: Ask a yes/no question about each row

# Which houses sold for more than $200,000?
expensive_houses = ames['SalePrice'] > 200000
expensive_houses

0        True
1       False
2       False
3        True
4       False
        ...  
2925    False
2926    False
2927    False
2928    False
2929    False
Name: SalePrice, Length: 2930, dtype: bool

# Keep only the True rows
filtered_ames = ames[expensive_houses]
print(f"Original dataset: {ames.shape[0]} houses")
print(f"Expensive houses: {filtered_ames.shape[0]} houses")
filtered_ames.head()

Original dataset: 2930 houses
Expensive houses: 857 houses

Common Detective Mistakes 🚨

Avoid these rookie errors:

ames.head     # Returns function, not data
ames.head()   # ✅ Returns first 5 rows

ames['saleprice']  # ❌ KeyError!
ames['SalePrice']  # ✅ Correct spelling

ames[(SalePrice > 200000) and (ames['Year Built'] > 2000)]  # ❌
ames[(ames['SalePrice'] > 200000) & (ames['Year Built'] > 2000)]    # ✅

ames['SalePrice']    # Series
ames[['SalePrice']]  # DataFrame

ames[Year Built > 2000]        # ❌ NameError
ames[ames['Year Built'] > 2000] # ✅ Correct

Real-World Detective Work Challenge!

Back to our original challenge: Let’s solve it the Python way!

Your Task: Find the average number of seats on aircraft manufactured by Embraer in 2004 or later

# Load the planes dataset
planes_url = "https://raw.githubusercontent.com/bradleyboehmke/uc-bana-7025/main/data/planes.csv"
planes = pd.read_csv(planes_url)

# Take a quick look at the data structure
planes.head()

Now it’s your turn! Write Python code to: 1. Filter for Embraer aircraft built in 2004 or later 2. Calculate the average number of seats

Hint: Remember to use .loc[] for filtering and .mean() for averages!

Compare this experience to your earlier spreadsheet work…

The Python Solution 🎯

Complete Solution: Here’s how to solve it step by step

# Step 1: Filter for Embraer aircraft built in 2004 or later
embraer_recent = planes.loc[
    (planes['manufacturer'] == 'EMBRAER') & (planes['year'] >= 2004)
]

print(f"Found {embraer_recent.shape[0]} matching aircraft")

# Step 2: Calculate the average number of seats
avg_seats = embraer_recent['seats'].mean()

print(f"Average seats on Embraer aircraft (2004+): {avg_seats:.1f}")

# Bonus: Let's see the range too
print(f"Seat range: {embraer_recent['seats'].min()} to {embraer_recent['seats'].max()}")

Found 128 matching aircraft
Average seats on Embraer aircraft (2004+): 33.7
Seat range: 20 to 55

Any Final Questions? 🙋‍♀️

About today’s concepts:

• Data importing or file paths?
• DataFrame vs Series confusion?
• Filtering or selection techniques?
• About Thursday’s lab?
• Questions from last week’s homework assignment?
• Upcoming assignments?