Compose a data stack

What you will be able to do

• Run Postgres with Compose and a named volume
• Wait for the database with a healthcheck before the job starts
• Run a SQL command from a second service on the Compose network

Why this matters for DE and AI

This is the local shape of a pipeline: a database that stays up, a job that runs once, files or SQL in between. The classic bug is the job starting while Postgres is still booting. depends_on without a healthcheck only waits for the container to exist, not for the database to accept connections.

Concepts

Two services on one Compose network. The job talks to db as a hostname — not localhost.

job  --postgres protocol-->  db:5432
you  --optional publish-->   127.0.0.1:15432

We publish 15432 on the host so a later GUI could connect, without colliding with a native Postgres on 5432.

Healthcheck — a command Compose runs inside db. For Postgres:

healthcheck:
  test: ["CMD-SHELL", "pg_isready -U dcubes -d lab"]
  interval: 3s
  timeout: 3s
  retries: 20

depends_on with condition:

depends_on:
  db:
    condition: service_healthy

Named volume for /var/lib/postgresql/data so down without -v keeps the database.

The job can use the same image as Postgres (postgres:16-alpine) just to get psql. That is a common lab trick. Later you will use a job image you built.

First up will download postgres:16-alpine (tens to a hundred-plus MB). Wait for it.

Practice on your machine

mkdir -p ~/dcubes/docker-lab/compose-db
cd ~/dcubes/docker-lab/compose-db

cat > compose.yaml << 'EOF'
services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: dcubes
      POSTGRES_PASSWORD: dcubes
      POSTGRES_DB: lab
    volumes:
      - pgdata:/var/lib/postgresql/data
    ports:
      - "15432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U dcubes -d lab"]
      interval: 3s
      timeout: 3s
      retries: 20

  job:
    image: postgres:16-alpine
    depends_on:
      db:
        condition: service_healthy
    environment:
      PGPASSWORD: dcubes
    command:
      [
        "psql",
        "-h", "db",
        "-U", "dcubes",
        "-d", "lab",
        "-c", "SELECT 1 AS ok;",
      ]

volumes:
  pgdata:
EOF

docker compose up

Stay in the foreground the first time so you can watch health. What you should see: db logs about the database system being ready; job prints a table with ok and 1; job exits. db keeps running. Press Ctrl + C when you have seen the ok row — Compose will stop the stack.

Run again detached and inspect:

docker compose up -d
docker compose ps -a
docker compose logs job

What you should see: db healthy/Up. job Exited (0) (one-shot success). Logs contain ok and 1.

Talk to Postgres from the host with the client in a throwaway container (no local psql required):

docker compose exec db psql -U dcubes -d lab -c 'SELECT current_database();'

You should see lab.

docker compose down

Data remains in pgdata until you docker compose down -v. Leave the volume for the demo unless you want a clean slate (down -v is OK here if you prefer empty).

Common mistakes

• psql -h localhost from the job. localhost is the job container. Use -h db.
• depends_on: [db] only. Job may crash with connection refused. Use service_healthy.
• Publishing 5432:5432 when you already have Postgres on the host. Use 15432:5432 as above.

Next

Debug and cleanup — logs, compose exec, disk, and a careful prune.