Grapher schema sync¶

The grapher chart-config schema is owned by the web team in owid-grapher and published at https://files.ourworldindata.org/schemas/grapher-schema.NNN.json. It is mutated in place without version bumps (e.g. dumbbell plots landed in grapher-schema.010.json directly), so ETL needs a way to follow upstream changes.

ETL keeps a vendored pin of the upstream schema — a committed snapshot, same model as a lockfile — and derives or checks everything else against it:

owid-grapher (web team)
    │  publishes / mutates grapher-schema.010.json
    ▼
schemas/grapher-schema.010.json              ← vendored pin (the only upstream copy in ETL)
    ├─→ etl/collection/model/schema_types.py    generated from it    ← unit test (offline)
    ├─→ multidim/explorer-schema $refs           resolved against it  ← at runtime (offline)
    └─→ dataset-schema.json embedded block       checked against it   ← unit test (offline, enum-deep)

vendored ↔ live upstream                      ← scheduled workflow + integration tests

Everything inside the repo is machine-checked against the pin on every PR; only the pin itself can lag upstream, and that is what the scheduled workflow watches.

The scheduled workflow¶

.github/workflows/sync-grapher-schema.yml runs on weekdays at 07:00 UTC (plus manual workflow_dispatch). When everything is in sync it is a no-op (two curls).

Failure modes and caveats¶

• Red CI on a bot PR is by design, not a malfunction: test_grapher_config_schema_sync fails when the upstream change also needs manual propagation (see below). The failing test output is the todo list.
• Branch force-update: if upstream changes again before a bot PR is merged, the next run force-updates the auto-sync-grapher-schema branch and can clobber manual commits on it. Finish and merge bot PRs promptly; if you need more time, move the work to your own branch.
• files.ourworldindata.org unreachable, dependency installation or generator failures → the job fails loudly in the Actions tab; nothing is skipped silently.
• Bot PRs are created with the default GITHUB_TOKEN: Buildkite CI triggers normally (external webhook), but GitHub-Actions-based CI would not.

Completing a sync: the /sync-grapher-schema skill¶

The automatic part of a sync (vendored refresh + type regeneration) is committed by the workflow. The judgment part is guided by the internal Claude Code skill /sync-grapher-schema:

• mirroring the upstream diff into the grapher_config block embedded in schemas/dataset-schema.json, preserving the deliberate ETL-side deviations (Jinja oneOf escape hatches, the extra WorldMap chart type, the ETL-only data/includedEntities properties);
• adding $refs for genuinely new properties to schemas/multidim-schema.json / schemas/explorer-schema.json;
• handling the rarer version bump (new grapher-schema.NNN): bumping DEFAULT_GRAPHER_SCHEMA in etl/config.py, updating $refs, re-vendoring.

It can also be run ad-hoc — e.g. when the web team announces a change and you don't want to wait for the cron (the skill performs the refresh itself), or trigger the workflow manually via workflow_dispatch.

Who does what¶

Drift guards reference¶