Raise publish_async ack errors on the returned future - #994
Conversation
APIError.from_error raises instead of returning, so the error branch in _handle_async_reply threw before set_exception ran. The exception escaped into the delivery task (logged as "nats: encountered error") and the caller's ack future was left pending forever. Capture the raised error and attach it to the future so awaiting the ack surfaces it.
|
Claude finished @caspervonb's task in 1m 37s —— View job PR Review
Verdict: Correct fix, one minor nit. The root cause is clear and the description is accurate: What the fix does right:
One nit (non-blocking): The Test: The regression test is well-constructed — triggering a server-side error via |
renerocksai
left a comment
There was a problem hiding this comment.
Please bear in mind, I am fairly new to nats-py and might be wrong in my assessments. The review focused on idiomatic / good-quality async Python code; I haven't evaluated NATS-specific design choices.
-
I think the same "future left pending forever" hole is still open on the non-error branch of
_handle_async_reply— the fallback handler only catchesCancelledError/InvalidStateError, so aJSONDecodeErrorout ofjson.loadsor aTypeErrorout ofPubAck.from_responseescapes exactly the way theAPIErrordid. Details inline. -
The root cause looks like
APIError.from_errorbeing an unannotated always-raising factory, sitting right underfrom_msg(...) -> NoReturn. No inline for this one sincenats/src/nats/js/errors.pyisn't in the diff: annotatingfrom_erroras-> NoReturn(errors.py:80) would turnerr = APIError.from_error(...)into something a type checker can flag, rather than a runtime surprise that needs a comment to explain. The larger alternative — havingfrom_errorreturn the error and updating the threeraise APIError.from_error(...)call sites — would drop the try/except round-trip and the comment entirely, but it changes public behaviour, so quite possibly out of scope for a fix PR. -
The rest are small test tightenings — hedged inline, take or leave.
| @@ -165,8 +165,13 @@ async def _handle_async_reply(self, msg: Msg) -> None: | |||
| try: | |||
| resp = json.loads(msg.data) | |||
There was a problem hiding this comment.
I think this try block still leaves the non-error branch exposed to the very bug this PR fixes. The handler at the bottom catches only asyncio.CancelledError / asyncio.InvalidStateError, but two statements inside it can raise something else:
json.loads(msg.data)→json.JSONDecodeError(aValueError) on a payload that isn't JSONapi.PubAck.from_response(resp)on line 177 →TypeError, sinceBase.from_responseends incls(**params)andPubAck.stream/PubAck.seqhave no defaults
Either one propagates out of the subscription callback into Subscription._wait_for_msgs, which hands it to error_cb and carries on — so the future is never resolved, handle_done never runs, the token stays in _publish_async_futures, the semaphore permit is never released, and publish_async_completed() blocks forever. Same failure mode as #985, just reached down a different branch.
Since it's the same defect class, it might be worth having the fallback route to the future as well:
except (asyncio.CancelledError, asyncio.InvalidStateError):
pass
except Exception as err:
if not future.done():
future.set_exception(err)That keeps the current behaviour for the two existing cases and stops anything else from stranding the future.
|
|
||
| # Expected-stream mismatch forces the server to return an error ack. | ||
| future = await js.publish_async("aerr", b"data", stream="WRONGSTREAM") | ||
| with pytest.raises(nats.js.errors.APIError): |
There was a problem hiding this comment.
Two small tightenings, both hedged:
-
pytest.raises(nats.js.errors.APIError)pins only the base class that every JS API error inherits from. The point of the fix is that the specific error, with its fields intact, now reaches the caller — so asserting that would be stronger, e.g.with pytest.raises(...) as exc_info:plus a check onexc_info.value.err_code/.code(or amatch=on the description). If an expected-stream mismatch comes back as a 400,BadRequestErrorwould be the concrete type. -
The other half of the regression isn't covered. Before the fix the future never completed, so
handle_donenever ran:publish_async_pending()stayed at 1 andpublish_async_completed()would hang.test_publish_asyncjust above already asserts exactly that, so mirroring it here would lock in the leak side too:
await asyncio.wait_for(js.publish_async_completed(), timeout=2)
self.assertEqual(js.publish_async_pending(), 0)(Wrapped in wait_for so a regression fails rather than hangs.)
Very minor: the file does from nats.js.errors import * and the neighbouring tests use the bare names (NoStreamResponseError, TooManyStalledMsgsError), so an unqualified APIError here would match the surrounding style.
APIError.from_errorraises rather than returns, so the error branch in_handle_async_replythrew beforeset_exception; the error escaped into the delivery task and the ack future was left pending forever. Capture and attach it to the future.Fixes #985.