[Metrics] Perform async Re-Aggregation when Spatial Attributes are dropped #3691

nikhilbhatia08 · 2025-10-12T20:14:41Z

Performing Reaggregation when spatial dimensions are dropped in async counters. There are also unit tests provided to the change.

Changes

Please provide a brief description of the changes here.

For significant contributions please make sure you have completed the following items:

CHANGELOG.md updated for non-trivial changes
Unit tests have been added
Changes in public API reviewed

…opped

codecov · 2025-10-12T20:31:36Z

Codecov Report

❌ Patch coverage is 79.41176% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.75%. Comparing base (755411e) to head (927c117).

Files with missing lines	Patch %	Lines
...try/sdk/metrics/state/measurement_attributes_map.h	65.86%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3691      +/-   ##
==========================================
- Coverage   89.95%   89.75%   -0.20%     
==========================================
  Files         225      226       +1     
  Lines        7273     7326      +53     
==========================================
+ Hits         6542     6575      +33     
- Misses        731      751      +20

Files with missing lines	Coverage Δ
...nclude/opentelemetry/sdk/metrics/observer_result.h	`100.00% <100.00%> (ø)`
...telemetry/sdk/metrics/state/async_metric_storage.h	`95.66% <100.00%> (+1.06%)`	⬆️
...e/opentelemetry/sdk/metrics/state/metric_storage.h	`100.00% <ø> (ø)`
...telemetry/sdk/metrics/state/multi_metric_storage.h	`100.00% <100.00%> (ø)`
sdk/src/metrics/meter.cc	`85.89% <100.00%> (ø)`
...try/sdk/metrics/state/measurement_attributes_map.h	`65.86% <65.86%> (ø)`

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull Request Overview

This PR addresses the issue of performing async re-aggregation when spatial attributes are dropped in OpenTelemetry metrics. The main purpose is to ensure that when attributes are filtered out by attribute processors in async counters, the measurements are properly reaggregated to combine values that now have identical attribute sets.

Key changes include:

Introduced a new MeasurementAttributes type for better handling of attribute-value mappings
Modified async metric storage to perform reaggregation during attribute processing
Updated observer result handling to use the new measurement structure

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
sdk/include/opentelemetry/sdk/metrics/state/measurement_attributes_map.h	New header defining custom hash/equality functions and MeasurementAttributes type for attribute maps
sdk/include/opentelemetry/sdk/metrics/observer_result.h	Updated ObserverResultT to use new MeasurementAttributes type and removed dependency on AttributesProcessor
sdk/include/opentelemetry/sdk/metrics/state/async_metric_storage.h	Added ConvertMeasurementsToMetricAttributes method to handle reaggregation and attributes processor integration
sdk/include/opentelemetry/sdk/metrics/state/metric_storage.h	Updated AsyncWritableMetricStorage interface to use MeasurementAttributes type
sdk/include/opentelemetry/sdk/metrics/state/multi_metric_storage.h	Updated AsyncMultiMetricStorage to use new MeasurementAttributes type
sdk/src/metrics/meter.cc	Modified async metric storage registration to pass attributes processor
sdk/src/metrics/state/observable_registry.cc	Updated to create ObserverResultT without attributes processor parameter
sdk/test/metrics/async_metric_storage_test.cc	Updated tests to use new types and added attributes processor setup
sdk/test/metrics/observer_result_test.cc	Simplified test to use new ObserverResultT constructor
sdk/test/metrics/sum_aggregation_test.cc	Added comprehensive tests for observable counter reaggregation scenarios

sdk/test/metrics/sum_aggregation_test.cc

sdk/include/opentelemetry/sdk/metrics/state/measurement_attributes_map.h

sdk/test/metrics/async_metric_storage_test.cc

lalitb · 2025-10-16T17:46:08Z

Assigned it to self. Will review during this week.

Reneg973 · 2025-10-25T12:04:21Z

sdk/include/opentelemetry/sdk/metrics/observer_result.h

-  explicit ObserverResultT(const AttributesProcessor *attributes_processor = nullptr)
-      : attributes_processor_(attributes_processor)
-  {}
+  explicit ObserverResultT() = default;


explicit not required for default constructor. Here it can also be removed (rule of 0)

Reneg973 · 2025-10-25T12:21:13Z

sdk/include/opentelemetry/sdk/metrics/state/measurement_attributes_map.h

+    AttributeMapHash,
+    AttributeMapEqual>;
+
+}  // namespace metrics


add another type here for use in AsyncMetricStorage:
using MeasurementsKeyType = typename Measurements::key_type;

I am not sure that it's worth, adding another runtime expensive associative container just for a list of some attributes. Unordered_map gets efficient with thousands and thousands of entries. But still very slow in comparison to a boost::flat_map or absl::flat_hash_map or a simple std::vector.

with the type std::unordered_map<nostd::string_view, opentelemetry::common::AttributeValue> it will also lead to dangling references. It will work when the string_view is created with "abcd", because this will stay in memory, but putting in a std::string... > crash

Reneg973 · 2025-10-25T12:25:59Z

sdk/include/opentelemetry/sdk/metrics/observer_result.h

  void Observe(T value) noexcept override
  {
-    data_[MetricAttributes{{}, attributes_processor_}] = value;
+    std::unordered_map<nostd::string_view, opentelemetry::common::AttributeValue> empty;


Can be replaced with new type MeasurementsKeyType

Reneg973 · 2025-10-25T12:26:47Z

sdk/include/opentelemetry/sdk/metrics/observer_result.h

  {
-    data_[MetricAttributes{attributes, attributes_processor_}] =
-        value;  // overwrites the previous value if present
+    std::unordered_map<nostd::string_view, opentelemetry::common::AttributeValue> attr_map;


replace with new type MeasurementsKeyType

Reneg973 · 2025-10-25T12:32:38Z

sdk/src/metrics/meter.cc

       exemplar_filter_type
 #endif
-  ](const View &view) {
+  ](const View &view) -> bool {


superfluous, compiler knows it

dbarker · 2025-11-01T23:13:05Z

sdk/test/metrics/sum_aggregation_test.cc

+  opentelemetry::sdk::metrics::FilterAttributeMap allowedattr;
+
+  std::unique_ptr<opentelemetry::sdk::metrics::AttributesProcessor> attrproc{
+      new opentelemetry::sdk::metrics::FilteringAttributesProcessor(allowedattr)};


please replace new with std::make_unique or std::make_shared accordingly. Rationale - CppCoreGuidelines#r22 and r23

dbarker · 2025-11-01T23:19:45Z

sdk/include/opentelemetry/sdk/metrics/state/measurement_attributes_map.h

+
+// Hash function for unordered_map of key-value pairs
+// This Custom Hash is only applied to strings and int for now
+struct AttributeMapHash


Please see sdk/common/attributemap_hash.h -> GetHash and GetHashForAttributeValueVisitor. Consider if these can be used.

dbarker · 2025-11-01T23:29:35Z

Thanks for the PR @nikhilbhatia08! I've left a few nitpick comments but I'm interested to dig more into the specification and requirements for this change to provide a better review.

The original linked issue #1724 is a bit old and it references an opentelemetry-specification issue 2905 that has since been closed as not planned.

Can you update the description of the PR with a bit more context on the use cases enabled and links to the relevant sections of otel specification? Thanks!

[Metrics] Perform async Re-Aggregation when Spatial Attributes are dr…

095da8d

…opped

nikhilbhatia08 requested a review from a team as a code owner October 12, 2025 20:14

iwyu fix and minor changes

c48f6e1

lalitb requested a review from Copilot October 13, 2025 04:08

Copilot AI reviewed Oct 13, 2025

View reviewed changes

nikhilbhatia08 and others added 4 commits October 13, 2025 10:52

format and iwyu fix

5487b02

Measurements name change

36d47c2

Merge branch 'main' into async_re_aggregation

a6cbbbd

resolve merge conflicts

880567f

lalitb self-requested a review October 16, 2025 17:45

lalitb self-assigned this Oct 16, 2025

lalitb removed their request for review October 16, 2025 17:45

nikhilbhatia08 added 3 commits October 18, 2025 23:14

Merge branch 'main' into async_re_aggregation

881c1e7

Merge branch 'main' into async_re_aggregation

f12966d

Merge branch 'main' into async_re_aggregation

927c117

Reneg973 reviewed Oct 25, 2025

View reviewed changes

dbarker reviewed Nov 1, 2025

View reviewed changes

[Metrics] Perform async Re-Aggregation when Spatial Attributes are dropped #3691

Are you sure you want to change the base?

[Metrics] Perform async Re-Aggregation when Spatial Attributes are dropped #3691

Conversation

nikhilbhatia08 commented Oct 12, 2025

Changes

Uh oh!

codecov bot commented Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lalitb commented Oct 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reneg973 Oct 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dbarker commented Nov 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Oct 12, 2025 •

edited

Loading

Reneg973 Oct 25, 2025 •

edited

Loading