Schedules and runs

What you will be able to do

• Set schedule=None vs a cron expression on a DAG
• Explain pause, manual trigger, and catchup
• Name the usual reasons a scheduled DAG did not run

Why this matters for data work

“It runs at 6am” is a contract with finance, not a hope. Airflow schedules DAG runs against a clock and a data interval. If you do not know pause, catchup, and timezone, you will backfill three years of empty runs by accident — or wait for a job that is still toggled off.

Concepts

schedule=None — never by the clock. Only Trigger (what we used for hello_airflow). Best while you are writing the graph.

Cron / timetable — schedule="0 6 * * *" is 06:00 UTC unless you set a timezone. Airflow 3 also accepts timetable objects; cron is enough here.

start_date — the earliest interval the DAG is allowed to consider. It is not “first time I clicked trigger.”

Catchup — if the DAG was off and you turn it on, should Airflow create a run for every missed interval since start_date? Airflow 3 defaults catchup to false. We still set catchup=False in files so the intent is visible. Never set start_date to 2018 and catchup=True on a laptop.

Paused — compose creates DAGs paused. A scheduled DAG that is paused does nothing.

Logical date / data interval — a run represents a window of time (for example “yesterday’s data”), which may finish after that window ends. A @daily DAG does not fire at midnight for midnight; it fires when the day has closed. This is the number-one “it didn’t run” surprise.

Keep learning DAGs on schedule=None plus manual trigger unless you are practicing the clock on purpose.

Practice on your machine

Change hello_airflow so you can see a cron field without enabling catchup. Do not leave it unpaused overnight if you do not want extra runs.

cd ~/dcubes/airflow-lab
cat > dags/hello_airflow.py << 'EOF'
from datetime import datetime

from airflow.sdk import dag, task


@dag(
    schedule="0 6 * * *",
    start_date=datetime(2024, 1, 1),
    catchup=False,
    tags=["dcubes"],
)
def hello_airflow():
    @task
    def say_hello() -> str:
        print("Hello from the DCubes Airflow lab")
        return "hello"

    @task
    def shout(word: str) -> None:
        print(word.upper())

    shout(say_hello())


hello_airflow()
EOF

Wait for parse. In the UI, hello_airflow should show a schedule of 0 6 * * *.

1. Leave it paused. Confirm no new automatic run appears.
2. Trigger once anyway — manual runs still work on a scheduled DAG.
3. Set schedule=None again (same file as Your first DAG) so the lab stays quiet.

cd ~/dcubes/airflow-lab
cat > dags/hello_airflow.py << 'EOF'
from datetime import datetime

from airflow.sdk import dag, task


@dag(
    schedule=None,
    start_date=datetime(2024, 1, 1),
    catchup=False,
    tags=["dcubes"],
)
def hello_airflow():
    @task
    def say_hello() -> str:
        print("Hello from the DCubes Airflow lab")
        return "hello"

    @task
    def shout(word: str) -> None:
        print(word.upper())

    shout(say_hello())


hello_airflow()
EOF

What you should see

• Schedule column changes after parse.
• Paused + cron ⇒ no surprise runs.
• Manual trigger still creates a run.

Common mistakes

• UTC vs local clock. 0 6 * * * is 06:00 in the DAG’s timezone (UTC unless you set one).
• Catchup + old start_date. Thousands of queued runs. Keep catchup=False.
• Watching the wall clock for @daily. The run for 1 January often starts at the beginning of 2 January.
• “I triggered it but it is still paused.” Pause blocks the schedule, not always your mental model of Trigger — if Trigger is disabled, unpause first.

Next

When tasks fail