Lakehouse lab
Data Engineering · Practice environment
Data lakehouse lab
Bring up a local lakehouse: RustFS object storage with Iceberg, Spark to write, Trino to read, and optional Airflow to orchestrate — on your machine.
Beginner3 lessons~135 min
A lakehouse is object storage plus a table format plus engines that agree on the catalog. This module is the practice environment for later warehouses, Spark, and cloud lessons, plus the concepts behind it: what object storage promises and refuses, and how Apache Iceberg turns Parquet objects into tables.
You will clone amhhaggag/dcubes-lakehouse, then start RustFS (S3 + Iceberg REST catalog), Spark (writes), and Trino (reads) with Docker Compose, then optionally Airflow to run the same path as a DAG. The site never starts the stack for you.
Start here
Finish Docker Getting Started and Airflow Getting Started so Compose is familiar. Keep the Airflow course lab in ~/dcubes/airflow-lab as it is — this is a separate folder. Do Getting Started first; the concept lessons use the running stack.
On-ramp
ReadyYour lakehouse practice environment
Start a local lakehouse with RustFS, Iceberg, Spark, and Trino on Docker Compose, then optionally Airflow.
45 min
Lesson 1
ReadyObject storage, file systems, and block storage
Why the lake lives in buckets and keys, what S3 promises and refuses, and how that differs from the disk and folders you already know.
40 min
Lesson 2
ReadyApache Iceberg: tables on object storage
The problem Iceberg solves, the metadata tree that makes a pile of Parquet files a table, and the features that fall out of it — snapshots, time travel, schema evolution — traced through your lab.
50 min