Repository navigation
feat(db): align migration prototype with single-table RFC - #1772
Merged
Merged
Conversation
…ase-migrations-implementation
…pagates Repeatedly cancelling a transaction could keep its pool slot checked out. Two driver-level gaps combine. SQLAlchemy can drive both a graceful close and a stop on the same aiosqlite connection, whose stop is not idempotent and enqueues a second request behind a worker that already exited. And a statement cancelled mid-await leaves its native cursor held by the exception stack, so the rollback that follows fails with "Connection closed" and the slot is never returned. Close the cursor tracked for the connection before retiring it, make the driver stop idempotent per connection, interrupt the statement SQLite is still running, and finish the whole exit inside a managed task so a repeated cancellation cannot strand the cleanup. Only the transaction that owns the connection runs it, so a nested exit cannot close its caller's cursors. A clean exit stays in the caller's task, which the seekdb shutdown ordering depends on. Without the cursor step the new regression fails: the only pool slot is still checked out after the tenth cancelled transaction. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…mitting part of it Interrupting a SQLite statement rolls the whole native transaction back, so writes that already reported success are gone while SQLAlchemy still trusts the transaction. Keeping `is_disconnect = False` avoided discarding the connection -- an in-memory StaticPool holds its entire database in it -- but it also let a caller that caught the error keep writing and commit a partial result: insert row 3, interrupt a long UPDATE, insert row 4, and only row 4 survives. Record the interruption on the connection and refuse to commit when the transaction exits, raising PendingRollbackError the way the unmodified baseline does. The connection still returns to the pool, so an in-memory database survives. * `test_interrupted_write_cannot_commit_earlier_work[file|memory]` fails without the guard. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ransaction The mark lives on the pooled connection, and the usage recorder runs its own transaction without going through `AsyncDatabase.transaction()`, so an interrupt recorded there was never consumed. It survived on the connection until an unrelated transaction picked that connection up and raised PendingRollbackError for a rollback it had nothing to do with -- which is how this first showed up, as a topic-memory supervisor test failing on 3.11. Clear any leftover mark when a transaction starts, so only an interrupt raised inside it can fail it. * `test_interrupt_marked_outside_the_transaction_does_not_fail_it` fails without the cleanup. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ection Two follow-ups from review. `connection.info` re-resolves the DBAPI connection, so once a cancelled statement invalidated it through SQLAlchemy's normal path, reading the interruption mark from the failure path raised PendingRollbackError and skipped the rest of `_finish_transaction()`. That left the transaction counted as active and kept `database.close()` from finishing. Hold the info dict itself instead; it stays usable after the connection is gone. The mark was also only checked before COMMIT ran. A commit cancelled while in flight records the mark afterwards, so it survived on the pooled connection and failed the next borrower's transaction. Clear it in a `finally` around the exit, which covers cancellation during COMMIT or ROLLBACK too. * `test_invalidated_connection_still_reaches_transaction_cleanup` fails without the first part. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Zxf-xufeng
marked this pull request as ready for review
September 30, 2026 09:20
Zxf-xufeng
requested review from
AlexStocks,
PsiACE,
Teingi,
frostming and
zhanghuidinah
as code owners
September 30, 2026 09:20
guamiao
added a commit
to guamiao/powercontext
that referenced
this pull request
Oct 9, 2026
Bring in the database-migration prototype (oceanbase#1771/oceanbase#1772), loopback proxy bypass (oceanbase#1888), topic-memory window fixes (oceanbase#1842/oceanbase#1898) and related work. Resolve three conflicts: - database.py: keep the split transaction/_transaction so cancellation cleanup still owns the connection; adopt the upstream nested-snapshot comment. - test_sqlite_profile.py: keep this branch's SyncConnection/event imports and its invalidation + commit-mark regression tests. - test_topic_memory_generic_api.py: adopt upstream's reworked cancellation e2e (completion-flush rendezvous with busy_timeout_ms=100), which retains the cancellation, normal/storage-failure, settlement and recovery-usage assertions through the public flush boundary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue or RFC does this PR close?
Refs #1756 and implements the acceptance phase of RFC #1771. The implementation branch contains no RFC documents. #1716 remains independent; this PR does not close the remaining production rollout requirements.
Rationale for this change
Explicit database maintenance needs a trustworthy schema version and recoverable execution without adding run, step, or task control tables. Installing software remains separate from changing database structure. A successful four-table migration must not be mistaken for complete Server readiness.
What changes are included in this PR?
pc_schema_revision(version_num)control table. Keep durable maintenance intent, resource checksums, backup references, and recovery progress outside the business database.server db-migrate status/plan/apply/verifyfor persistent SQLite, embedded seekdb, and OceanBase MySQL-mode databases through the configured maintenance adapter. These connections do not initialize current application metadata or open a business Repository. Plans bind target identity, schema, resources, configuration, backup policy, and maintenance scope.--lock-coordination single-hostand one persistent--evidence-diron the fixed maintenance host. Business Servers may remain distributed.GET_LOCK(), fixed physical session IDs, or PROCESS privileges. Normal completion and successful connection closure clear the run receipt. An interrupted run blocks verification and ready/no-op shortcuts, even if its Alembic head is current. After a DBA confirms that prior remote execution and background DDL have ended, recovery requires an exact--acknowledge-previous-runtoken bound to the reviewed plan and rechecked under the lock. PC does not automatically establish remote quiescence; acknowledgement never bypasses schema, data, resource, backup, or original-journal checks.BackupProviderimplementations, unchecked manual-backup declarations, and explicit no-backup consent. SQLite Online Backup includes committed WAL. Native Fork feature detection remains separate from accepted restoration workflows; unavailable automatic backup never silently falls back. The current executor also rejects automatic table Fork before writes because same-database backup-object tracking is not yet registered.service stop/start/restartand a separately tested maintenance controller. Stopped services retain registration with native activation suppressed. Startup validates the loaded definition, active manager state, and readiness; interruptions and cleanup errors preserve the failure and stopped-state evidence. Complete service switching remains disabled for this partial database bundle.Are there any user-facing changes?
This is Phase A acceptance. The installed bundle covers only
pc_artifacts,pc_artifact_heads,pc_artifact_candidate_versions, andpc_artifact_tags. Output reportsreadiness_scope=registered_bundleandserver_ready=false. Complete Server databases with unmanaged objects are rejected. Normal application startup andprocessing-migrateremain outside this bundle, and--manage-servicecannot switch a business service.All three backends now have deployment executors for registered layouts. Existing seekdb/OceanBase databases can use declared manual backup or explicitly accepted no-backup risk. Automatic Fork backup remains unavailable until release-owned recovery acceptance exists; a version number alone does not enable it. The engine feature minima are OceanBase AI Database 4.6.2 for both Fork forms, seekdb 1.1.0 for table Fork, and seekdb 1.2.0 for database Fork. Empty initialization requires no backup.
OceanBase requires access to tenant and cluster identity metadata, one fixed maintenance host, and an explicit persistent evidence directory. Minimal isolated four-table acceptance passed through an existing ODP endpoint on OceanBase AI Database 4.6.3. Normal upgrades retain one confirmation; an interrupted OceanBase run additionally requires the explicit DBA declaration described above. Complete historical baselines, legacy-task/data/projection validators, startup/SDK/Worker gates, full index coverage, and mandatory schema-change checks remain production prerequisites. No existing business database or personal service was modified during this work.
How was this change tested?
UV_NO_SYNC=1 UV_OFFLINE=1 PREK_NO_CONCURRENCY=1 make check: lock consistency, workflow/reference checks, pre-commit hooks, full type checks, and 33 integration-manifest tests passed.pylibseekdb1.4.0.post1): 20 passed, including all 10 runner cases, schema reflection/data integrity, dedicated connections, and real cross-process exclusion through a symlink alias. The selection skips the OceanBase-only interrupted-run acknowledgement case.git diff --checkpassed.AI usage statement
Implemented, independently reviewed, and tested with OpenAI Codex. Incorporated #1782 fixes preserve their original authors and Claude co-author attribution. Backend evidence distinguishes real execution from uncompleted production acceptance.