[ML] Resolve duplicate key exception in GetDatafeedRunningStateAction #125477
+22
−1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implement merge function for duplicate datafeed states when a datafeed is force-stopped and restarted before cancellation completes. Select the most appropriate state based on: 1. Prefer state with more recent searchInterval.startMs when both exist 2. Prefer states with searchInterval over those without 3. Default to second state when all criteria are equal
elasticsearchmachine
added
needs:triage
hye-on
changed the title
elasticsearchmachine
added
the
Team:Search Relevance
|
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
elasticsearchmachine
removed
the
needs:triage
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes issue #104160 where a duplicate key exception occurs in
GetDatafeedRunningStateAction.Response.fromResponses(). The issue happens when a datafeed is force-stopped and restarted before its local task cancellation completes. This creates a situation where two local tasks for the same datafeed temporarily coexist on the ML node (one cancelling, one starting), causing the duplicate key error when both report their state.The solution implements a merge function in the toMap collector that selects the most appropriate state when duplicates are found, based on the
searchIntervaldata.The solution implements a merge function in the toMap collector that selects the most appropriate state when duplicates are found, based on the searchInterval data.
Select the most appropriate state based on:
Comment
I'm new to Elasticsearch and open source contributions in general. I went with searchInterval.startMs as the selection criteria, but I'd appreciate any feedback on whether there might be a better approach for handling these duplicate states. Thank you for your guidance! :)