-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Add item_count
metric to field data cache stats API
#19174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add item_count
metric to field data cache stats API
#19174
Conversation
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
❌ Gradle check result for b56cbf2: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Peter Alfonsi <[email protected]>
❌ Gradle check result for dce88a7: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Peter Alfonsi <[email protected]>
❌ Gradle check result for a297d35: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Peter Alfonsi <[email protected]>
❌ Gradle check result for 91cbbc9: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Peter Alfonsi <[email protected]>
❌ Gradle check result for f1d7105: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
private FieldMemoryStats fieldMemorySizes; | ||
@Nullable | ||
private FieldMemoryStats fieldItemCounts; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't look right to me. I think we should update FieldMemoryStats
itself to have item_count
as well along with the existing memory_size_in_bytes
?
And then we can the reuse the existing fields.writeTo
and fields.toXContent
, and it will automatically spit out the per field item count as well. Also makes more sense logically having item count tracked at FieldMemoryStats
level.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I initially tried to do something like this but there are 2 problems:
- FieldMemoryStats is used in other places like CompletionStatsCache which don't necessarily need item counts, plus it's PublicApi so we'd have to introduce new ctors with the items as a param + deprecate all the old ones which isn't ideal.
- The logic of keeping track of memory and item count is identical, besides how it formats the XContent at the end, so I tried having the FieldMemoryStats object itself provide an additional toXContent method which wouldn't format it as a memory. But, now the memory and the item objects don't know they are meant to go together in the higher-level stats object, and we get XContent that looks like this:

So, I figured the best approach was to have the field-level memory and item counts live in separate FieldMemoryStats instances, and let the higher-level stats object handle arranging the counts so they show up properly like
"indices" : {
"fielddata" : {
"memory_size_in_bytes" : 31184,
"evictions" : 0,
"item_count" : 10,
"fields" : {
"text0" : {
"memory_size_in_bytes" : 15580,
"item_count" : 5
}
server/src/main/java/org/opensearch/index/fielddata/ShardFieldData.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
❌ Gradle check result for 3cdca80: null Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Description
Adds an
item_count
metric to the existing_nodes/stats/indices/fielddata
API. Like memory size, it can be aggregated by field by passing afielddata_fields
param listing the desired fields.Example API response for
curl -XGET "localhost:9200/_nodes/stats/indices/fielddata?pretty&fielddata_fields=text0,text2"
:Related Issues
Resolves #19173
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.