Skip to content

Low priority: no automated freshness checks for tippecanoe, id-tagging-schema/iD, or osm-testdata grid #555

Description

@brawer

Summary

We don't automatically get notified when upstream cuts a new release for three externally-sourced components. Bumping any of them today requires a human to remember to check and act. Low priority -- nothing is broken, this is just a gap worth tracking.

The three cases

  1. tippecanoe -- pinned by commit SHA in Containerfile (ARG TIPPECANOE_COMMIT, alongside the human-readable ARG TIPPECANOE_VERSION; see Containerfile: pin tippecanoe to a commit SHA instead of a mutable tag #553). Bumping means someone notices a new tippecanoe release and updates both ARGs by hand.

  2. id-tagging-schema (and, incidentally, the iD editor, which the same generator also pulls from) -- scripts/generate_id_tagging_schema.py queries the GitHub "latest release" API for both openstreetmap/id-tagging-schema and openstreetmap/iD at the time it's run, fetches specific source files from those releases, and regenerates src/pipeline/osm/id_tagging_schema.rs. The version used is recorded only in a comment at the top of the generated file (currently id-tagging-schema@v7.0.1). Nobody runs this on a schedule; a human has to notice both projects moved and manually re-run the script and commit the diff.

  3. osm-testdata grid -- scripts/vendor-osm-testdata-grid.sh <commit-sha> vendors grid/data/ from osmcode/osm-testdata into tests/test_data/osm-testdata-grid/, pinned to a commit recorded in tests/test_data/osm-testdata-grid/VENDORED.md. This one's unlikely to ever need a bump in practice (it's a fixed set of test fixtures, not something upstream actively iterates on for our purposes) -- noting it here mainly for completeness, not because we expect to act on it soon.

Why Dependabot doesn't see any of these

Dependabot works per ecosystem: it understands a specific manifest/lockfile format (Cargo.toml+Cargo.lock, package.json, a Dockerfile's FROM line, a workflow's uses: line, .gitmodules, ...) and queries the matching registry for newer versions against that manifest. All three cases above fall outside every ecosystem it supports:

  • tippecanoe: the pin lives in a plain ARG consumed by a git fetch inside a RUN step. Dependabot's docker ecosystem only parses FROM image:tag@digest lines -- it has no concept of an arbitrary shell command using an ARG as a version pin, so it doesn't even know this is a dependency.
  • id-tagging-schema / iD: there's no manifest or lockfile at all -- just a bespoke Python script that hits the GitHub Releases API and writes generated Rust source. Dependabot has no visibility into custom code-generation pipelines; nothing here matches any ecosystem it knows how to parse.
  • osm-testdata grid: vendored (copied) files with the pinned commit recorded only in a free-text VENDORED.md, not in .gitmodules. Since it isn't an actual git submodule, Dependabot's gitsubmodule ecosystem -- the one ecosystem that's otherwise close to this shape -- doesn't apply either.

Options, if/when we want to close this gap

  • Convert to a real git submodule (mainly plausible for osm-testdata grid; less so for id-tagging-schema/iD since those aren't literally vendored as a subtree). Dependabot's gitsubmodule ecosystem can then open bump PRs -- but it tracks the tip of whichever branch the submodule follows, not specifically new release tags, so it may be noisier than "notify me on releases."
  • A small custom scheduled workflow that polls each upstream repo's Releases API (gh api/gh release list), compares against the currently pinned version, and opens an issue or PR when there's a newer one -- for the generated-code cases, it could even run the existing generator script itself and open a PR with the resulting diff. Most precise option, but it's bespoke automation to write and maintain per dependency. (We discussed and deliberately deferred this for tippecanoe specifically when pinning it in Containerfile: pin tippecanoe to a commit SHA instead of a mutable tag #553.)
  • Renovate as a Dependabot alternative -- it supports "custom manager" / regex-based configs that can track an arbitrary version string or commit SHA anywhere in a repo (an ARG in a Containerfile, a comment, a URL) against a generic datasource like GitHub releases or git refs. This is the one option that could plausibly cover all three cases with declarative config instead of bespoke scripts per dependency, so it's worth a look before building something custom.
  • Status quo -- keep relying on someone noticing during occasional manual review. Cheapest, but is exactly the gap this issue is tracking.

No action needed now; just want this written down so it doesn't get silently forgotten.

Metadata

Metadata

Assignees

No one assigned

    Labels

    dependenciesPull requests that update a dependency file

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions