DCubes
Docker module

Lesson 8

Persist data

Bind mounts versus named volumes, and why warehouse files must live outside the container layer.

beginner30 minRun on your machine

What you will be able to do

  • Bind-mount a host folder and read it from a container
  • Write a file inside the mount and see it on the host
  • Create a named volume and explain when to use one

Why this matters for DE and AI

Containers are disposable. Warehouses, Parquet drops, and Postgres data directories are not. If the CSV lived only in the container layer, docker rm would delete the extract. Volumes and bind mounts are how data outlives the process — the same idea as “do not store the lake on the Spark worker’s /tmp.”

Concepts

Bind mount — a directory on your machine, mounted at a path in the container.

docker run -v /host/path:/container/path ...

On this module we use ~/dcubes/docker-lab/data:/data. You can ls the files with normal Linux tools. Best for source CSVs, job output you want to inspect, and Compose project files.

Named volume — Docker-managed storage, referenced by a name (dcubes-pg). The engine chooses the host path (you usually do not edit it by hand). Best for database files you should not casually touch.

docker volume create dcubes-vol
docker run -v dcubes-vol:/data ...

The container’s writable layer (no -v) is for temp files. Assume it vanishes.

Permissions

The process inside the container may run as root or as a numeric user. If a bind-mounted file is mode 600 and owned by you, a different user in the container may see Permission denied — the same class of bug as Linux permissions. For this lab, Alpine as root can write your data/ folder.

SELinux / Docker Desktop

On some Linux setups you need -v path:path:Z. Docker Desktop and WSL usually do not. If a mount is empty, check the host path exists before run (mkdir -p). Docker will create a missing host path as root, which is a mess on Linux.

Practice on your machine

mkdir -p ~/dcubes/docker-lab/data
echo 'from host' > ~/dcubes/docker-lab/data/note.txt
cd ~/dcubes/docker-lab

Read the host file from Alpine:

docker run --rm -v ~/dcubes/docker-lab/data:/data alpine:3.20 cat /data/note.txt

What you should see: from host

Append from inside the container:

docker run --rm -v ~/dcubes/docker-lab/data:/data alpine:3.20 sh -c 'echo from container >> /data/note.txt'
cat ~/dcubes/docker-lab/data/note.txt

What you should see: both lines, on the host. The container is already gone (--rm); the file stayed because it lived on the mount.

Named volume:

docker volume create dcubes-vol
docker run --rm -v dcubes-vol:/data alpine:3.20 sh -c 'echo persisted > /data/x.txt'
docker run --rm -v dcubes-vol:/data alpine:3.20 cat /data/x.txt

What you should see: dcubes-vol created, then persisted from the second container. Different containers, same volume.

List volumes:

docker volume ls

Leave dcubes-vol for now. We will remove leftover volumes in the cleanup lesson. Do not run docker volume prune yet.

Common mistakes

  • Mounting a file that does not exist. Docker may create a directory with that name. mkdir and touch on the host first.
  • Copying data into the image with COPY and calling it persistence. Rebuilds replace it. Use a mount for data that changes.
  • Bind-mounting / or your whole home directory. Too much access. Mount the lab data/ folder only.

Next

Publish ports — how localhost:18080 reaches a process in a container.

← Write a DockerfilePublish ports →