DataFrames

What you will be able to do

• read_csv into a DataFrame and use head, shape, dtypes
• Filter rows with boolean masks
• Fill or drop simple missing values

Why this matters for data work

pandas is the default table-in-memory tool for Python data work. DE jobs profile, filter, and aggregate before SQL or Spark; AI feature checks often start the same way.

Concepts

DataFrame — labeled columns, index of rows.

Series — one column.

read_csv — load delimited files. dtypes — inferred types; check them.

Boolean mask — df[df["quantity"] > 1].

NA — missing data: isna, fillna, dropna.

Practice on your machine

cd ~/dcubes/python-lab
source .venv/bin/activate
python -m pip install pandas
mkdir -p data/raw data/out
cat > data/raw/orders-frames.csv << 'EOF'
order_id,sku,quantity,unit_price,store
ORD-1,SKU-100,2,9.99,STR-01
ORD-2,SKU-101,1,15.00,STR-02
ORD-3,SKU-100,,9.99,STR-01
ORD-4,SKU-102,3,4.50,STR-03
EOF
cat > pandas_frames_lab.py << 'EOF'
import pandas as pd
from pathlib import Path

root = Path(__file__).resolve().parent
df = pd.read_csv(root / "data" / "raw" / "orders-frames.csv")
print(df.head())
print("shape", df.shape)
print(df.dtypes)
print("missing quantity", int(df["quantity"].isna().sum()))

clean = df.dropna(subset=["quantity"]).copy()
clean["quantity"] = clean["quantity"].astype(int)
clean["line_total"] = clean["quantity"] * clean["unit_price"]
big = clean[clean["line_total"] >= 15]
print(big[["order_id", "line_total"]])
EOF
python pandas_frames_lab.py

What you should see

• Four rows in head (one with missing quantity).
• quantity often floats because of the empty cell.
• After dropna, three rows; filtered subset shows larger line totals.

Common mistakes

• Chained assignment warnings — use .copy() after filters when you will mutate.
• Trusting inferred dtypes — IDs sometimes become integers and lose leading zeros.
• Loading huge files into pandas on a laptop — profile size first; Spark comes later on the roadmap.

Next

Group, join, write