Skip to content

Conversation

peteralfonsi
Copy link
Contributor

Description

Adds an item_count metric to the existing _nodes/stats/indices/fielddata API. Like memory size, it can be aggregated by field by passing a fielddata_fields param listing the desired fields.

Example API response for curl -XGET "localhost:9200/_nodes/stats/indices/fielddata?pretty&fielddata_fields=text0,text2":

"indices" : {
        "fielddata" : {
          "memory_size_in_bytes" : 31184,
          "evictions" : 0,
          "item_count" : 10,
          "fields" : {
            "text0" : {
              "memory_size_in_bytes" : 15580,
              "item_count" : 5
            },
            "text2" : {
              "memory_size_in_bytes" : 15604,
              "item_count" : 5
            }
          }
        }
      }

Related Issues

Resolves #19173

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Peter Alfonsi added 2 commits August 27, 2025 12:02
Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: Peter Alfonsi <[email protected]>
Peter Alfonsi and others added 2 commits August 28, 2025 14:21
Signed-off-by: Peter Alfonsi <[email protected]>
Copy link
Contributor

❌ Gradle check result for b56cbf2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Peter Alfonsi <[email protected]>
Copy link
Contributor

❌ Gradle check result for dce88a7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Peter Alfonsi <[email protected]>
Copy link
Contributor

❌ Gradle check result for a297d35: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@peteralfonsi
Copy link
Contributor Author

Hmm, I had mistakenly renamed a PublicApi method so detect-breaking-change was failing. But now that I've fixed it the job seems to be having some issue with Failed to load eclipse jdt formatter: run 1, run 2

Signed-off-by: Peter Alfonsi <[email protected]>
Copy link
Contributor

github-actions bot commented Sep 2, 2025

❌ Gradle check result for 91cbbc9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Peter Alfonsi <[email protected]>
Copy link
Contributor

github-actions bot commented Sep 2, 2025

❌ Gradle check result for f1d7105: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Comment on lines +68 to +70
private FieldMemoryStats fieldMemorySizes;
@Nullable
private FieldMemoryStats fieldItemCounts;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look right to me. I think we should update FieldMemoryStats itself to have item_count as well along with the existing memory_size_in_bytes ?

And then we can the reuse the existing fields.writeTo and fields.toXContent, and it will automatically spit out the per field item count as well. Also makes more sense logically having item count tracked at FieldMemoryStats level.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially tried to do something like this but there are 2 problems:

  1. FieldMemoryStats is used in other places like CompletionStatsCache which don't necessarily need item counts, plus it's PublicApi so we'd have to introduce new ctors with the items as a param + deprecate all the old ones which isn't ideal.
  2. The logic of keeping track of memory and item count is identical, besides how it formats the XContent at the end, so I tried having the FieldMemoryStats object itself provide an additional toXContent method which wouldn't format it as a memory. But, now the memory and the item objects don't know they are meant to go together in the higher-level stats object, and we get XContent that looks like this:
Screenshot 2025-08-27 at 10 59 07 AM

So, I figured the best approach was to have the field-level memory and item counts live in separate FieldMemoryStats instances, and let the higher-level stats object handle arranging the counts so they show up properly like

"indices" : {
        "fielddata" : {
          "memory_size_in_bytes" : 31184,
          "evictions" : 0,
          "item_count" : 10,
          "fields" : {
            "text0" : {
              "memory_size_in_bytes" : 15580,
              "item_count" : 5
            }

Signed-off-by: Peter Alfonsi <[email protected]>
Copy link
Contributor

github-actions bot commented Sep 4, 2025

❌ Gradle check result for 3cdca80: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Other v3.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Add item count metric to field data cache stats API
2 participants