Repository navigation
[SDK Sentinel] Prevent replay shutdown from dropping nondeterminism results - #1692
Conversation
|
@sdk-sentinel-bot This looks like it's consistently hanging / breaking long_local_activity_with_update |
SDK-Sentinel-Request: temporalio#1692/comment-6352006343
|
SDK Sentinel could not complete this response during publication. The responder commit is present at Failure: verify updated PR approval gate: workflow approval state did not settle before timeout Retry the failed central publish job to reconcile the existing mutation; do not recreate it manually. |
|
@sdk-sentinel-bot This looks like it's consistently hanging / breaking long_local_activity_with_update |
|
You're right about the original patch: replay failures such as the intentionally unregistered workflow in |
|
@sdk-sentinel-bot Please update this PR: rebase onto current main, resolve merge conflicts, and rerun long_local_activity_with_update. Preserve the existing fix unless testing shows it needs changes. If you cannot complete this, explain what blocked you. |
SDK-Sentinel-Request: temporalio#1692/comment-6374405122
f012297 to
f4e6cf2
Compare
|
SDK Sentinel could not complete this response during publication. The responder commit is present at Failure: The published responder commit conflicts with base branch main at aaa0db5; GitHub may not create pull_request runs for a conflicting PR. Reconcile the PR against the current base and create a new response request; approval was not verified. Follow the recovery action above; do not blindly retry this captured patch or force push. |
Caution
This PR contains untrusted AI-generated code. Do not approve or run CI until a maintainer has reviewed the diff. SDK Sentinel verified that GitHub Actions remained approval-gated with zero executable jobs when this PR was opened.
Summary
Batch workflow replay intermittently reported a final nondeterministic history as successful in two target-branch Windows integration jobs. The [preceding equivalent Windows suite](https://github.com/temporalio/sdk-rust/actions/runs/39270905943) passed before the [first captured failure](https://github.com/temporalio/sdk-rust/actions/runs/39286666336).
Root cause
The replay mock released the next history immediately after accepting a workflow-task failure. For the final history, the next poll exhausted the stream and could cancel the worker before the nondeterminism eviction reached the result observer. The unfilled result retained its default successful state.
Fix
Advance replay only through the existing post-activation eviction signal. This preserves the failed history's outcome before polling or shutdown can proceed.
Validation
The unchanged baseline and fixed repository validator passed formatting, lint, test-lint, and check. The focused integration test could not compile locally because
zstd-syshit a sandbox permission error. Windows and macOS integration CI remain required; the residual risk is a possible replay stall after a failed workflow task.Validation status:
validation-incompletehighhighhighstandard— 3 changed files; 2.4 KiBunavailable— The exact focused integration test could not compile locally because zstd-sys build.rs was denied an artifact operation by the managed sandbox.not-run— The existing regression test is the exact test that failed twice in CI, but local execution was blocked before build completion; no redundant test was added.Cargo formatting, linting, and checks) — passedCargo formatting, linting, and checks) — passedInvestigator-run checks
CARGO_BUILD_JOBS=2 timeout 180 cargo integ-test workflow_replayer_returns_ordered_results_for_multiple_histories— failed; 3 attempt(s). All attempts stopped in zstd-sys build.rs with sandbox PermissionDenied before the integration test ran.CARGO_BUILD_JOBS=2 cargo test -p temporalio-sdk-core replay::tests::client_override_describe_namespace_precedes_fallback— failed; 1 attempt(s). Compilation stopped in zstd-sys build.rs with the same sandbox PermissionDenied error before the unit test ran..ci-flake-runtime/input/validate.sh— passed; 2 attempt(s). Both candidate validations passed nightly formatting, cargo lint, cargo test-lint, and cargo check. The trusted unchanged baseline also passed.Required target CI
Residual risks
Automation provenance
0a5d646279824adcf07eeaa52d9179dcd934ab46If this finding should not be fixed, apply the
sdk-sentinel:false-positivelabel and close the PR. Sentinel will suppress the finding until its affected source changes.SDK Sentinel recurrence history
2026-10-20 18:27 UTC2026-10-20 18:27 UTC