First Compose shift

Do this after lessons 1–12. Lesson 13 helps when something looks “stuck.”

You are pretending to bring up a tiny warehouse for an extract. The website still runs nothing. You run every command in ~/dcubes/docker-lab/shift. Docker Desktop (or the engine) must be running.

What you need

• A working Docker setup from Getting Started
• The sample CSV (right-click, save, then copy into the lab):
  • movies.csv

1. Confirm the workspace

mkdir -p ~/dcubes/docker-lab/shift/data
cd ~/dcubes/docker-lab/shift

Copy the CSV (adjust Downloads for your OS). WSL users: Downloads is often /mnt/c/Users/YOURWINDOWSNAME/Downloads/.

cp ~/Downloads/movies.csv ~/dcubes/docker-lab/shift/data/
ls data

You should see movies.csv. Peek the header:

head -n 2 data/movies.csv

2. Write the load script

The job container will run this. It waits for Postgres (healthcheck should already have passed), creates a table, loads the CSV, then prints a genre count.

cat > load.sh << 'EOF'
#!/bin/sh
set -e
psql -h db -U dcubes -d movies -v ON_ERROR_STOP=1 << SQL
CREATE TABLE IF NOT EXISTS movies (
  title TEXT,
  year INTEGER,
  genre TEXT,
  minutes INTEGER
);
TRUNCATE movies;
SQL
psql -h db -U dcubes -d movies -v ON_ERROR_STOP=1 -c "\\copy movies FROM '/data/movies.csv' CSV HEADER"
psql -h db -U dcubes -d movies -c "SELECT count(*) AS movie_rows FROM movies;"
psql -h db -U dcubes -d movies -c "SELECT genre, count(*) AS n FROM movies GROUP BY genre ORDER BY n DESC, genre;"
EOF
chmod +x load.sh

3. Write compose.yaml

cat > compose.yaml << 'EOF'
services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: dcubes
      POSTGRES_PASSWORD: dcubes
      POSTGRES_DB: movies
    volumes:
      - pgdata:/var/lib/postgresql/data
    ports:
      - "15432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U dcubes -d movies"]
      interval: 3s
      timeout: 3s
      retries: 20

  job:
    image: postgres:16-alpine
    depends_on:
      db:
        condition: service_healthy
    environment:
      PGPASSWORD: dcubes
    volumes:
      - ./data:/data:ro
      - ./load.sh:/load.sh:ro
    command: ["sh", "/load.sh"]

volumes:
  pgdata:
EOF

:ro is a read-only bind mount. The job can read the CSV and the script; it cannot overwrite them.

4. Bring the database up, then run the job

First pull/start db. The first time, Postgres image download can take a minute.

docker compose up -d db
docker compose ps

Wait until db is healthy (run ps twice if needed). Then run the job in the foreground so you see the SQL tables:

docker compose run --rm job

What you should see: CREATE TABLE / TRUNCATE (or similar), COPY, then movie_rows 20, then genre counts. Drama and Sci-Fi should be near the top of this sample (same file as the Linux demo).

run --rm starts a new job container, waits, deletes it. That is the right shape for a one-shot load. compose up job also works but leaves an Exited container behind.

If the job errors with connection refused, db was not healthy yet. docker compose logs db and try run again.

5. Query from the running database

docker compose exec db psql -U dcubes -d movies -c "SELECT title, year FROM movies ORDER BY year DESC LIMIT 5;"

You should see five titles. Newest years in this sample are in the 2010s.

Save a small report on the host (not inside the container layer):

docker compose exec db psql -U dcubes -d movies -c "SELECT genre, count(*) AS n FROM movies GROUP BY genre ORDER BY n DESC;" > genres.txt
cat genres.txt

If exec says the container is not running, docker compose up -d db again.

6. Confirm the CSV mount and disk habit

docker compose exec db ls /data

Empty or “no such file” is correct — we mounted ./data on job, not on db. The database received rows through COPY, not through a mount on db. That is a real pipeline pattern: the warehouse does not need the original file after load.

docker system df

You should see images (Postgres) using the most space. The number is not the test; running df is.

7. Tear down

Leave the data directory on the host. Stop containers:

docker compose down
docker compose ps

ps should be empty. data/movies.csv should still exist on the host.

To wipe the Postgres volume as well (optional, clean slate):

docker compose down -v

You passed if

• data/movies.csv exists under ~/dcubes/docker-lab/shift
• docker compose run --rm job printed 20 movie rows
• Genre counts appeared (Drama / Sci-Fi near the top of this sample)
• docker compose exec db psql … returned titles
• docker compose down left no running project containers
• You ran docker system df at least once

If the row count is 21, you loaded the header as a row — \copy … CSV HEADER is missing. If it is 0, the COPY path is wrong (ls the bind mount from the job: docker compose run --rm job ls /data).

After this module

You can pull an image, run a container, persist files, publish a port, and describe a small stack in Compose. That is the weekly Docker skill for local data work.

Next on the roadmap: Python, then SQL, then Git — still guidance only, still on your machine.