Repository navigation
Add project-scoped calibrations, native footprints, and S3 storage - #32
Merged
Merged
Conversation
Doc 27: calibrations can now vary per project (not just per image), footprints move from a dead-dependency Text column to a native queryable PostgreSQL range type, and FITS storage moves to S3-compatible object storage (MinIO locally). Establishes the schema/storage foundation the ingest service (doc 28) depends on.
Test Results138 tests +10 138 ✅ +10 9s ⏱️ -1s Results for commit 9c33efd. ± Comparison against base commit 7cabc26. This pull request removes 2 and adds 12 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Level2CalibrationtoProject(composite unique onlevel2_image_id+project_id, replacing the old strict 1:1) so the same raw image can carry a distinct calibration per project; enforceLevel2Image.base_filenameuniqueness.moc_strfootprint columns with a nativeint8multirangefootprintcolumn (src/db/spatial_types.MOCType), resolving GitHub issue Migrate Tile.healpix_index to a native range type #26 without depending onhealpix-alchemy— confirmed unmaintained since 2021 and unimportable against modernmocpy. GiST-indexed; a live&&overlap query proves the indexed spatial-match capability the migration exists for.S3StorageService(boto3) + a local MinIO service indocker-compose.ymlfor FITS storage, swappable to real S3 via env vars..github/workflows/ci.yml) — CI previously had no object storage at all.healpix-alchemydependency; correct now-stale "HealpixAlchemy" references across docs and governance files (CLAUDE.md,README.md, architecture docs, the infra topology diagram).CLI Verification
All steps completed prior to PR creation via
genTests.feature/schema-storage-spatial-typesdocker compose build api worker_light worker_heavydocker compose up -d api worker_light worker_heavy0008applied (alembic upgrade head→0008 (head); round-tripped viadowngrade -1/upgrade headto prove the corrected migration, including the GiST indexes, applies cleanly —alembic checkreports no drift)docker compose exec api uv run pytest --cov=src --cov-fail-under=90 -q→ 136 passed, 1 legitimate host-only skip, 98.94% coverage-W)docs/diagrams/infrastructure_topology.mdupdated (newminionode + edges, de-stale'd db-node label) and visually reviewed in rendered HTMLApplication QA
All steps completed and verified prior to PR creation via
genTests.base_filenameuniqueness — duplicate insert correctly rejected withIntegrityErrorsky_fractionand empty symmetric difference; a live&&query returned only the spatially-overlapping calibrationlocalhost:9001, confirmed visuallyS3StorageServicelive round-trip over the real container network — uploaded/downloaded content matched exactlyEndpointConnectionError, no hang); succeeded again immediately after restart🤖 Generated with Claude Code