HTTP and simple APIs

What you will be able to do

• Install httpx in a venv and perform a GET request
• Parse JSON responses and handle non-200 status codes
• Read tokens from environment variables, not source code

Why this matters for data work

Extracts often start with HTTP: a REST API, a webhook payload, later a model endpoint. The pattern is the same — URL, headers, status code, JSON body, secrets outside git.

Concepts

httpx — modern HTTP client. requests is the same idea if you see it in older blogs; this module standardizes on httpx.

GET — read data. status_code — 200 ok, 4xx your fault, 5xx server fault.

response.json() — parse JSON body to dict/list.

Environment variables — os.environ.get("API_TOKEN"). You met env vars in Linux; never hardcode tokens in scripts you might commit.

Timeouts — always set one so a hung socket does not block a job forever.

Practice on your machine

cd ~/dcubes/python-lab
source .venv/bin/activate
python -m pip install httpx

Public demo API (no key). Needs network access:

cat > http_lab.py << 'EOF'
import logging
import os
import sys

import httpx

logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
log = logging.getLogger("http_lab")

url = "https://httpbin.org/json"
token = os.environ.get("API_TOKEN")  # unused here; pattern for real APIs
headers = {}
if token:
    headers["Authorization"] = f"Bearer {token}"

try:
    with httpx.Client(timeout=15.0) as client:
        r = client.get(url, headers=headers)
except httpx.HTTPError as exc:
    log.error("request failed: %s", exc)
    sys.exit(1)

log.info("status=%s", r.status_code)
if r.status_code != 200:
    log.error("body=%s", r.text[:200])
    sys.exit(1)

data = r.json()
log.info("top keys=%s", list(data.keys()))
print(data.get("slideshow", {}).get("title"))
EOF
python http_lab.py

You should see status=200, some keys, and a slideshow title string from httpbin.

Offline fallback — skip the live call and parse a saved body:

python -c "import json; print(json.loads('{\"ok\": true}')['ok'])"

What you should see

• INFO log with status 200.
• A title printed from the JSON payload.
• If DNS/network fails, an error and non-zero exit — fix network or use the offline check.

Common mistakes

• No timeout — job hangs.
• Tokens in source — use env or a local .env that is gitignored (load pattern varies; raw os.environ is enough here).
• Assuming the body is always JSON — check Content-Type and status first.
• Calling internal hostnames from the wrong network — same lesson as Docker ports.

Next

NumPy basics