Repository navigation
Migrate to psycopg3, add point-HEALPix indexing, remove DummyImage - #38
Merged
Merged
Conversation
Doc 29 completes the Stage 1 database foundation: psycopg3 replaces psycopg2, enabling native range/multirange binding and eliminating MOCType's text-literal serialization workaround. Tile, Level2Image, and Level3Mosaic gain an indexed point-in-footprint containment query via the new PointHEALPixType. The Stage-0 DummyImage scaffolding is fully decommissioned now that IngestBatch/Level3Mosaic are the real async-tracking pattern.
Test Results242 tests - 26 242 ✅ - 26 34s ⏱️ ±0s Results for commit 8307c00. ± Comparison against base commit 37da9b2. This pull request removes 42 and adds 16 tests. Note that renamed tests count towards both. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
range/multirange binding and eliminating MOCType's text-literal
serialization workaround (resolves Evaluate migrating the DB driver from psycopg2 to psycopg3 #30).
PointHEALPixType) toTile,Level2Image, andLevel3Mosaic, enabling an indexedfootprint-contains-point spatial query; added
IngestBatch.job_configuration_idahead of doc 30's watchdog work (resolves Migrate Tile.healpix_index to a native range type #26).
DummyImagescaffolding (model, servicefunctions, task, CLI commands, routes, admin view, seed data); simplified
DiffpypeTask.on_failureto be entity-agnostic now that nothing butDummyImageever used its status-write (resolves Decommission Stage-0 DummyImage scaffolding #33).CLI Verification
All steps completed prior to PR creation via
genTests.feature/psycopg3-healpix-dummy-cleanupdocker compose build api worker_light worker_heavydocker compose up -d api worker_light worker_heavy0011(point-HEALPix columns/indexes,IngestBatchFK),0012(drop
dummy_images) — verified viaalembic currentand a full downgrade/upgraderound-trip through both migrations
docker compose exec api uv run pytest --cov=src --cov-fail-under=90 -q(242 passed, 99.4% coverage)
docs/diagrams/infrastructure_topology.md— N/A, no docker-compose service orcontainer-layer changes
Application QA
All steps completed and verified prior to PR creation via
genTests.container; the INSERT+SELECT round-tripped cleanly through the new driver.
Level2Image.healpix_index— ingested 717 real FITS files;every resulting row had
healpix_indexpopulated, none NULL.Tile.healpix_index— tessellated and created 54 realtiles over live ingested data; every tile had
healpix_indexpopulated, andtile↔calibration associations formed correctly against real overlapping footprints.
Level3Mosaic.healpix_index— clustered a real epochand created a mosaic from real constituent calibrations;
ra/decl/healpix_indexall populated as expected for the has-constituents case.
footprint @> healpix_indexcontainment query — ran the real GiST-indexedcontainment query against live tile/image data;
EXPLAINconfirmed it uses theindex, not a sequential scan.
IngestBatch.job_configuration_idFK exist exactly as specified.dummy_imagestable,/jobs/dummyroutes, and
run-dummy/get-dummyCLI commands are all fully removed (404s andargparse errors as expected).
on_failure's simplified behavior — dispatched a real Celery failure throughexecute_cli_tool; confirmed logging + dead-letter dispatch fire correctlyend-to-end with no DB-related crash, verified in both worker logs and Flower.
🤖 Generated with Claude Code