-
Notifications
You must be signed in to change notification settings - Fork 21
Description
Summary
Currently, SDKs do not consistently handle gRPC request timeouts across implementations.
While the Java SDK includes a configurable per-request grpcDeadline, other SDKs either lack this feature or require it to be manually set per transaction (e.g., via Transaction.setGrpcDeadline()).
This results in inconsistent behavior and potentially poor user experience - especially in cases where a node becomes unresponsive or experiences long response times (e.g., Cloudflare 522 timeout scenarios).
The goal of this issue is to introduce a global, SDK-wide grpcDeadline setting that defines a maximum timeout per gRPC request, after which the SDK marks the node as unhealthy and automatically rotates to the next available node.
Background
- When a node is slow or unresponsive, the SDK can currently hang for extended periods (e.g., 30–40 seconds) before failing.
- Inconsistent timeout handling across SDKs leads to different behavior between languages, making it harder for developers to reason about network reliability and retry behavior.
- The Java SDK already provides a
Client.setGrpcDeadline(Duration)setting, which applies a consistent timeout to all requests. - Other SDKs rely on per-transaction configuration or lack timeout enforcement entirely.
Proposal
Introduce a global grpcDeadline configuration at the client level for all SDKs:
const client = Client.forMainnet();
client.setGrpcDeadline(5_000); // 5 seconds per gRPC requestBehavior
- Applies to all outgoing gRPC requests (queries, transactions, etc.).
- If a request exceeds this deadline:
- The SDK aborts the request.
- Marks the node as temporarily unhealthy.
- Rotates to the next healthy node automatically.
- Can be overridden per transaction or query, maintaining backward compatibility with
Transaction.setGrpcDeadline()(or equivalent).
Relationship with requestTimeout
The SDK already defines a requestTimeout property on the client, which controls the total time budget for a complete Transaction or Query execute operation — including:
- Attempting the first node.
- Failing due to a node error.
- Applying retry/backoff logic.
- Retrying with another healthy node.
To ensure consistent and predictable behavior:
requestTimeoutmust be greater than or equal to the per-requestgrpcDeadline.- The SDK should enforce this constraint at runtime or initialization (e.g., throw or adjust automatically if misconfigured).
- A sensible default should be introduced for
requestTimeout(e.g., 2 minutes) if not already defined.
This ensures that the SDK does not abort the overall operation prematurely while still respecting per-node timeouts.
Defaults and Recommendations
| Parameter | Description | Default | Notes |
|---|---|---|---|
grpcDeadline |
Global timeout per gRPC request | 10 seconds | Per-node timeout |
requestTimeout |
Maximum total duration for a Transaction or Query execute operation (including retries) | 2 minutes | Must be ≥ grpcDeadline |
Benefits
- Consistent timeout handling across SDKs.
- Prevents long “hang” periods caused by slow or unresponsive nodes.
- Improves UX by failing fast and rotating predictably.
- Aligns with Java SDK’s existing behavior.
- Introduces clear relationship between per-request and total operation timeouts.
Tasks
- Define
grpcDeadlineproperty on client config for all SDKs. - Enforce
requestTimeout >= grpcDeadline. - Ensure all gRPC-based requests respect the global deadline.
- Implement consistent error classification and node rotation behavior.
- Add defaults for both settings (
grpcDeadlineandrequestTimeout). - Update SDK reference docs and examples.
Acceptance Criteria
- All SDKs have a global
grpcDeadlineconfiguration. - SDK enforces
requestTimeout >= grpcDeadline. - Timeout logic is consistent across SDKs.
- SDK rotates to next node on timeout.
- Documentation clearly explains both timeouts and how to override them.
Labels: enhancement, cross-sdk, network, timeout, ux