Skip to content

Add support for history streaming #229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Add support for history streaming #229

wants to merge 3 commits into from

Conversation

Copilot
Copy link

@Copilot Copilot AI commented May 23, 2025

This PR adds support for history event streaming in the Java SDK, which is a mechanism intended to unblock scenarios where the size of the orchestration instance history exceeds the gRPC message size limit.

Implementation Details

  • Added WORKER_CAPABILITY_HISTORY_STREAMING capability to GetWorkItemsRequest to advertise support for history streaming
  • Added logic to detect and handle orchestrator requests that have requiresHistoryStreaming flag set
  • Implemented processOrchestrationWithStreamingHistory method to efficiently stream history events from the sidecar
  • Added streamInstanceHistory methods to DurableTaskGrpcClient for direct API access
  • Included robust error handling for different gRPC status codes
  • Added convenience methods, input validation, and detailed logging

Usage

The implementation is transparent to users of the SDK. When an orchestration instance's history becomes too large, the backend service will set the requiresHistoryStreaming flag, and the worker will automatically use the streaming API to fetch history events.

For direct API access, developers can use:

// Stream history events with execution ID
Iterator<HistoryChunk> historyStream = durableTaskClient.streamInstanceHistory(
    "instanceId", "executionId", false);

// Or without execution ID
Iterator<HistoryChunk> historyStream = durableTaskClient.streamInstanceHistory(
    "instanceId", false);

This implementation parallels the approach taken in the .NET SDK: microsoft/durabletask-dotnet#370

Fixes #228.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • cdn.fwupd.org
    • Triggering command: /usr/bin/fwupdmgr refresh (dns block)
  • plugins-artifacts.gradle.org
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED --add-opens java.base/java.nio.charset=ALL-UNNAMED --add-opens java.base/java.net=ALL-UNNAMED --add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED -XX:MaxMetaspaceSize=256m -XX:&#43;HeapDumpOnOutOfMemoryError -Xms256m -Xmx512m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp /home/REDACTED/.gradle/wrapper/dists/gradle-7.4-bin/c0gwcg53nkjbqw7r0h0umtfvt/gradle-7.4/lib/gradle-launcher-7.4.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 7.4 (dns block)

If you need me to access, download, or install something from one of these locations, you can either:


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits May 23, 2025 05:57
@Copilot Copilot AI changed the title [WIP] Add support for history streaming Add support for history streaming May 23, 2025
@Copilot Copilot AI requested a review from YunchuWang May 23, 2025 06:02
Copilot finished work on behalf of YunchuWang May 23, 2025 06:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for history streaming
2 participants