Report completed transfers verbatim when cancellation races completion - #362
Merged
Merged
Conversation
…mpletion A stop token observed at resume no longer overrides what the operation actually did. The awaitable layer short-circuits a pre-stopped token before dispatch, so a cancelled-before-initiation op performs no I/O at all, and the shared completion decode lets a transfer outrank the cancellation flag: a completed read or write reports its error code and byte count unchanged, and the next operation on the still-stopped token reports canceled. The cancellation flag decides only when nothing was transferred, which is the signature of an op the cancellation actually terminated — and it still outranks the EOF mapping there, so an aborted read stays canceled rather than eof. The byte-count store moves into decode_io_result so the no-discard rule is structural rather than a per-backend promise.
… bases Every front-end header carried its own copy of the awaitable envelope — token capture, the pre-set-ec_ ready check, and a resume-time stop-token override — and each copy re-implemented the cancellation behavior the previous commit corrected. The native fronts, acceptors, files, resolver, and signal sets now derive from bytes_op_base / value_op_base / void_op_base, so the pre-dispatch short-circuit and verbatim result reporting live in one place. A derived awaitable keeps only its payload members, its dispatch call, and — where it builds a socket or transfers a peer impl on resume — a shadowing await_resume whose error comes solely from the decoded ec_.
…ing transferred With bytes transferred, the completed result is reported verbatim — error included. With nothing transferred, a raced stop or local teardown now reports canceled rather than whatever raw error the teardown surfaced: a cancellation request is what tears pending ops down locally (close, stop), and the flag normalizes the resulting completion error across backends, as the IOCP close path has always documented.
The POSIX thread-pool file ops and the IOCP random-access ops hand-rolled their completion decode with the cancellation flag outranking a completed transfer — a stop that raced the pool or the completion port zeroed the reported byte count (POSIX) or attached canceled to delivered data (IOCP), even though the bytes were already on disk or in the caller's buffer. Delegating to decode_io_result gives every file path the stream-contract priority: a transfer is reported verbatim, and canceled means nothing moved.
…opped tokens A write longer than the remaining expect script reported the full request as written while only the validated prefix was consumed — the surplus silently vanished. It is now a partial write of the validated prefix. The provide/expect/fuse fast paths completed inside await_ready, where no io_env exists, so a pre-stopped token was invisible and staged data was consumed anyway. All decisions now wait for await_suspend, which short-circuits a pre-stopped token with canceled before touching any staged data — matching the library streams.
The latch exists because some wolfSSL builds clear the shutdown bitmask on a read that follows a completed close, yet received_shutdown() re-queried only the live bitmask — on those builds a shutdown after an announced close was misreported as stream_truncated. The read/write paths already consult the latch; now the accessor does too. Also drop the orphaned engine doc blocks about drained-ciphertext handling: the OpenSSL one contradicted the driver (the unsent tail is retained and re-flushed, never dropped), and both were attached to no declaration.
The TLS driver narrowed each buffer's size to int for the engines' transfer API, and the IOCP stream paths narrowed to ULONG/DWORD for WSABUF and ReadFile/WriteFile: a single buffer at an exact multiple of the narrow type's range truncated to a zero-length kernel op, which decodes as a spurious EOF on read or a zero-byte success on write. Clamping to the type's maximum turns an oversized buffer into an ordinary partial transfer, which the stream contracts permit. Datagram sends are left alone: clamping would truncate a datagram, and oversized datagrams are rejected by the kernel anyway.
|
An automated preview of the documentation is available at https://362.corosio.prtest3.cppalliance.org/index.html If more commits are pushed to the pull request, the docs will rebuild at the same URL. 2026-09-23 17:14:50 UTC |
|
GCOVR code coverage report https://362.corosio.prtest3.cppalliance.org/gcovr/index.html Build time: 2026-09-23 17:31:06 UTC |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #358. Closes #361. Supersedes #360.
Problem
write_some/read_somere-checked the stop token at resume time and, if itwas signalled, replaced the completed result with
{operation_canceled, 0}—discarding the byte count of a transfer that had already happened (#358,
op_base.hpp). The same check was dead for pre-cancellation: the token onlyreaches the awaitable in
await_suspend, so a pre-stopped token did not stopanything — the syscall ran anyway (writes reached the wire, reads consumed and
discarded buffered data) and only the report was masked (#361). One layer
down, the shared completion decode gave the cancellation flag top priority, so
a completed op whose stop callback fired before the completion was processed
was relabeled
canceledas well.Semantics
The capy
ReadStream/WriteStreamcontracts require that on errornis thecount transferred before the contingency (never discarded), and that
n == buffer_sizeimplies success — a completed transfer is reportedverbatim, with the contingency deferred to a subsequent call. The concept docs
name
when_all/when_anycomposition as the rationale, which is exactly thereporter's scenario. This PR implements:
{canceled, 0}with no I/O performed. Theawaitable layer short-circuits before dispatch.
verbatim (success or error, full or partial, byte count intact). The next
operation on the still-stopped token reports
canceled.{canceled, 0}— and thezero is truthful: a reactor op that parks has transferred nothing (a
partial write completes rather than parks), and an aborted SQE/IRP carries
no bytes.
A single
read_some/write_sometherefore never pairscanceledwith anonzero count. The composed algorithms are where
{canceled, n > 0}legitimately appears, with
nthe honest accumulated total —capy::write'sdocumented postcondition falls out of these semantics without changes.
Implementation
detail/op_base.hpp:await_suspendshort-circuits a pre-stopped token;await_resumereturns the decoded result verbatim (no token check).decode_io_result(shared by every backend): priority is nowtransfer → cancelled → error → EOF, and it owns the
*bytes_outstore so"counts are never discarded" is structural rather than a per-backend
promise. With nothing transferred the flag outranks the raw completion
error, preserving the documented close/teardown normalization.
acceptors, files, resolver, and signal sets — each carrying its own copy of
the buggy machinery — now derive from the three
op_baseCRTP bases(net −520 lines). The bug is unrepresentable:
await_resumehas no token.defect in corners with private decodes: the POSIX thread-pool file ops
(count zeroed on a raced cancel — retry would duplicate data), the IOCP
random-access ops (
canceledattached to delivered data), themockettest double (oversize writes over-reported; fast paths blind to pre-stop),
a wolfSSL shutdown-mapping latch, and >2 GiB length truncations at
int/ULONG transfer boundaries.
Verification
The reproducer from #358 against this branch (wire bytes verified
server-side):
when_all(writer){canceled, 0}co_await{success, 5}"hello"The reported count now equals the wire count in every case. Variant (1)
resolves differently than the issue anticipated: its interleaving stops the
token before the write initiates, so pre-cancellation (#361) now prevents
the I/O entirely rather than reporting the 5 bytes late. The suggested
one-character fix (
bytes_for0) would have produced{canceled, 5}—still a contract violation on full transfers, and five bytes on the wire from
an operation cancelled before it started.
New regression suites pin all three interleavings per backend:
cancel_race.cpp(deterministic completion-vs-stop race via a zero inlinebudget; pre-stop tests assert zero wire bytes and no consumed data),
native_resume_cancel.cpp(rewritten — it previously asserted the maskingbehavior), plus race-invariant tests for the file ops and mocket. Full matrix
green: Linux (epoll/select/uring, GCC/Clang, tsan/asan/coverage), FreeBSD
(kqueue/select), macOS (kqueue/select, tsan/asan), Windows (IOCP, MSVC
14.34/14.44).
Notes for review (answers to the questions in #360)
standard symmetric transfer, and since nothing was dispatched there is no
executor hop to restore;
dispatch_coroelsewhere serves completionsarriving off-executor.
many times." That is the consolidation commit; the machinery now exists
once.
hole): they don't need to — a transfer (
bytes > 0) outranks thecancellation flag in the shared decode, partial success is already legal
under the contract, and full-vs-partial needs no distinction.
One pre-existing test changed meaning:
claim_paths.cppassertedcanceledfor a parked writer whose send buffer can drain into the peer's receive
window mid-test (macOS timing); the old decode masked the resulting genuine
partial completion. The assertion is now the contract disjunction — parked
and claimed (
{canceled, 0}) or completed first ({success, partial}) —and what it forbids is precisely the bug class: a discarded count, or a
completed transfer labeled canceled.