Migrate gRPC transport executor to ForkJoinPool #19685
Conversation
github-actions
bot
added
enhancement
jungeunyooon
force-pushed
the
feature/19370-migrate-grpc-to-forkjoinpool
branch
from
42def60 to
cb0269f
Compare
|
❌ Gradle check result for cb0269f: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: yoonjungeun1 <[email protected]>
Signed-off-by: yoonjungeun1 <[email protected]>
jungeunyooon
force-pushed
the
feature/19370-migrate-grpc-to-forkjoinpool
branch
from
cb0269f to
32618f4
Compare
|
❌ Gradle check result for 32618f4: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
| - Onboarding new maven snapshots publishing to s3 ([#19619](https://github.com/opensearch-project/OpenSearch/pull/19619)) | ||
| - Remove MultiCollectorWrapper and use MultiCollector in Lucene instead ([#19595](https://github.com/opensearch-project/OpenSearch/pull/19595)) | ||
| - Change implementation for `percentiles` aggregation for latency improvement ([#19648](https://github.com/opensearch-project/OpenSearch/pull/19648)) | ||
| - Migrate gRPC transport executor from FixedExecutorBuilder to ForkJoinPoolExecutorBuilder for improved performance ([#19370](https://github.com/opensearch-project/OpenSearch/issues/19370)) |
There was a problem hiding this comment.
| - Migrate gRPC transport executor from FixedExecutorBuilder to ForkJoinPoolExecutorBuilder for improved performance ([#19370](https://github.com/opensearch-project/OpenSearch/issues/19370)) | |
| - Migrate gRPC transport executor from FixedExecutorBuilder to ForkJoinPoolExecutorBuilder for improved performance ([#19685](https://github.com/opensearch-project/OpenSearch/issues/19685)) |
| * | ||
| * @param settings The node settings | ||
| * @param threadPool The thread pool | ||
| * @param settings The node settings |
There was a problem hiding this comment.
Could you run ./gradlew spotlessApply to remove all the formatting changes in this PR?
There was a problem hiding this comment.
./gradlew precommit can also be run locally to ensure the Gradle Check CI passes on the PR before pushing
| // Create a ThreadPool with the gRPC executor | ||
| Settings settings = Settings.builder().put("node.name", "test-node").put("grpc.netty.executor_count", 4).build(); | ||
| ExecutorBuilder<?> grpcExecutorBuilder = new FixedExecutorBuilder(settings, "grpc", 4, 1000, "thread_pool.grpc"); | ||
| ExecutorBuilder<?> grpcExecutorBuilder = new ForkJoinPoolExecutorBuilder("grpc", 4, "thread_pool.grpc"); |
There was a problem hiding this comment.
Could you run start a local cluster, then run this API and share the output to ensure this change is working as expected?
https://github.com/opensearch-project/OpenSearch/blob/main/modules/transport-grpc/README.md#thread-pool-monitoring
2 participants
Description
This PR migrates the gRPC transport executor from
FixedExecutorBuildertoForkJoinPoolExecutorBuilderto improve performance through work-stealing and better load balancing.GrpcPlugin.javato useForkJoinPoolExecutorBuilderinstead ofFixedExecutorBuilderRelated Issues
Resolves #19370
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.