storage_mode="partitioned" is refused as a configuration error today. Nothing in django-absurd drives the partition lifecycle, and an option whose failure mode is silent data concentration a month after you set it is worse than no option. This issue is what "supported" would mean, and what it costs.
Why it was refused
absurd.ensure_partitions runs in exactly two places in the pinned schema — inside ensure_queue_tables at queue creation, and on a default_partition policy flip. Nothing periodic. A partitioned queue therefore works only inside the window created when the queue was created (partition_lookahead defaults to 28 days).
Past that window no partition covers the current week, so every task funnels into the default partition. That state is silent, and it is not recoverable by provisioning: ensure_partitions then tries to create the week partitions the default partition already holds rows for, and Postgres refuses.
Reproduced on a scratch database — partitioned queue, clock advanced past the window, one row written, then provisioning:
psycopg.errors.CheckViolation: updated partition constraint for default partition "t_p1_d" would be violated by some row
An already-funneled database has no in-band exit. Recovering it means detaching the default partition and redistributing rows, which no shipped command does.
What the SDK intends
Partition lifecycle is deliberately database-side. absurd.enable_cron() installs three pg_cron jobs (defaults):
| job |
schedule |
command |
absurd_partitions_* |
5 * * * * |
absurd.ensure_partitions(...) |
absurd_cleanup_* |
17 * * * * |
absurd.cleanup_all_queues(...) |
absurd_detach_plan_* |
29 * * * * |
absurd.schedule_detach_jobs(...) |
schedule_detach_jobs then schedules per-partition one-off * * * * * jobs running a raw ALTER TABLE ... DETACH PARTITION [CONCURRENTLY] plus a follow-up absurd.drop_detached_partition. The two-step exists because DETACH CONCURRENTLY cannot run inside a transaction block, and a cron job is how the schema gets an out-of-transaction statement.
The Python client wraps none of this. It accepts the policy knobs only — partition_lookahead, partition_lookback, detach_mode, detach_min_age. Calling absurd.enable_cron() is left to the operator.
What supporting it would take
Tier 1 — keep partitions alive. ensure_partitions is idempotent (create table if not exists) and re-derives its window on every call (now - lookback .. now + lookahead), so anything on a cadence removes both the funnel and the CheckViolation. A thin Python wrapper mirroring cleanup_queues (same SchemaNotInstalledError translation), driven by the existing CLEANUP job under beat and by a second managed job on the same catalog seam under pg_cron. Needs a test that a queue past its window gets its week partition back.
Tier 2 — detach and drop. absurd.schedule_detach_jobs raises without cron.job, so this is pg_cron-only as shipped. Driving it under beat means reimplementing the pipeline in Python — autocommit connection, a one-pipeline-per-parent guard, the CONCURRENTLY-unless-a-default-partition-exists rule — duplicating schema logic we pin rather than own. Surfacing the native jobs is the cheaper path.
Tier 3 — the runtime debt. No test spawns or runs a task on a partitioned queue. The read-only models are UNION ALL views over the per-queue tables; their behaviour over a partitioned parent (reads, the queue column, the admin changelist) is unverified. Independent of lifecycle and the least bounded of the three.
Recovering the deleted handling
The partitioned-specific handling was removed rather than carried. It is recoverable at 6b2de4270872d1b4e7038a54e7c79ebdfec99226:
git show 6b2de4270872d1b4e7038a54e7c79ebdfec99226:django_absurd/queues.py
git show 6b2de4270872d1b4e7038a54e7c79ebdfec99226:django_absurd/flush.py
What was there: QUEUE_OWNED_TABLE_PREFIXES, find_missing_queue_tables's storage_mode parameter and afind_missing_queue_tables's SQL equivalent, QueuePlan.repair_storage_mode, the storage-mode drift warning (absurd.W002) and its sync-time twin.
Most of it is coupled to the broken behaviour and should not be restored verbatim — the sync test that pinned "leave a provisioned partitioned queue alone" existed only because ensure_partitions would raise, and that premise disappears with Tier 1.
One finding is durable and worth keeping whatever shape support takes: a partitioned queue owns a sixth table, i_<queue>. absurd.spawn_task reserves the idempotency key there before it touches t_<queue>, which makes it the first relation a half-provisioned partitioned queue reports missing — so any completeness probe blind to it leaves such a queue refusing every keyed enqueue with nothing to repair. Found by live testing, not by review.
storage_mode="partitioned"is refused as a configuration error today. Nothing in django-absurd drives the partition lifecycle, and an option whose failure mode is silent data concentration a month after you set it is worse than no option. This issue is what "supported" would mean, and what it costs.Why it was refused
absurd.ensure_partitionsruns in exactly two places in the pinned schema — insideensure_queue_tablesat queue creation, and on adefault_partitionpolicy flip. Nothing periodic. A partitioned queue therefore works only inside the window created when the queue was created (partition_lookaheaddefaults to 28 days).Past that window no partition covers the current week, so every task funnels into the default partition. That state is silent, and it is not recoverable by provisioning:
ensure_partitionsthen tries to create the week partitions the default partition already holds rows for, and Postgres refuses.Reproduced on a scratch database — partitioned queue, clock advanced past the window, one row written, then provisioning:
An already-funneled database has no in-band exit. Recovering it means detaching the default partition and redistributing rows, which no shipped command does.
What the SDK intends
Partition lifecycle is deliberately database-side.
absurd.enable_cron()installs three pg_cron jobs (defaults):absurd_partitions_*5 * * * *absurd.ensure_partitions(...)absurd_cleanup_*17 * * * *absurd.cleanup_all_queues(...)absurd_detach_plan_*29 * * * *absurd.schedule_detach_jobs(...)schedule_detach_jobsthen schedules per-partition one-off* * * * *jobs running a rawALTER TABLE ... DETACH PARTITION [CONCURRENTLY]plus a follow-upabsurd.drop_detached_partition. The two-step exists becauseDETACH CONCURRENTLYcannot run inside a transaction block, and a cron job is how the schema gets an out-of-transaction statement.The Python client wraps none of this. It accepts the policy knobs only —
partition_lookahead,partition_lookback,detach_mode,detach_min_age. Callingabsurd.enable_cron()is left to the operator.What supporting it would take
Tier 1 — keep partitions alive.
ensure_partitionsis idempotent (create table if not exists) and re-derives its window on every call (now - lookback..now + lookahead), so anything on a cadence removes both the funnel and theCheckViolation. A thin Python wrapper mirroringcleanup_queues(sameSchemaNotInstalledErrortranslation), driven by the existingCLEANUPjob under beat and by a second managed job on the same catalog seam under pg_cron. Needs a test that a queue past its window gets its week partition back.Tier 2 — detach and drop.
absurd.schedule_detach_jobsraises withoutcron.job, so this is pg_cron-only as shipped. Driving it under beat means reimplementing the pipeline in Python — autocommit connection, a one-pipeline-per-parent guard, theCONCURRENTLY-unless-a-default-partition-exists rule — duplicating schema logic we pin rather than own. Surfacing the native jobs is the cheaper path.Tier 3 — the runtime debt. No test spawns or runs a task on a partitioned queue. The read-only models are UNION ALL views over the per-queue tables; their behaviour over a partitioned parent (reads, the
queuecolumn, the admin changelist) is unverified. Independent of lifecycle and the least bounded of the three.Recovering the deleted handling
The partitioned-specific handling was removed rather than carried. It is recoverable at
6b2de4270872d1b4e7038a54e7c79ebdfec99226:What was there:
QUEUE_OWNED_TABLE_PREFIXES,find_missing_queue_tables'sstorage_modeparameter andafind_missing_queue_tables's SQL equivalent,QueuePlan.repair_storage_mode, the storage-mode drift warning (absurd.W002) and its sync-time twin.Most of it is coupled to the broken behaviour and should not be restored verbatim — the sync test that pinned "leave a provisioned partitioned queue alone" existed only because
ensure_partitionswould raise, and that premise disappears with Tier 1.One finding is durable and worth keeping whatever shape support takes: a partitioned queue owns a sixth table,
i_<queue>.absurd.spawn_taskreserves the idempotency key there before it touchest_<queue>, which makes it the first relation a half-provisioned partitioned queue reports missing — so any completeness probe blind to it leaves such a queue refusing every keyed enqueue with nothing to repair. Found by live testing, not by review.