Component(s)
OpenTelemetry.Exporter.Geneva (metric exporter)
Is your feature request related to a problem? Please describe.
The Geneva metric exporter performs one Unix-domain-socket Socket.Send syscall per individual metric point during each export interval, in OtlpProtobufSerializer.SendMetricPoint.
At realistic cardinality (e.g. CardinalityLimit=4000 across multiple histograms ⇒ up to ~20K points/scrape), this dominates CPU on the exporter thread. Profile evidence from a production high-throughput .NET service shows Interop+Sys.Send at ~13% exclusive CPU, with the Geneva metric exporter as the dominant contributor.
The same pattern was identified in the Rust contrib repo for the Geneva log exporter: open-telemetry/opentelemetry-rust-contrib#306. The .NET Geneva log exporter already does buffered batching; only the metric exporter has the per-event syscall pattern.
Describe the solution you'd like
Batch metric-point events into bounded UDS sends.
Wire-format guarantee: when prefixBufferWithUInt32LittleEndianLength is set (the only mode used by MetricUnixDomainSocketDataTransport), each serialized event is already framed with a uint32-LE length prefix. The Geneva MDSD receiver already demultiplexes multiple events from a single read by that prefix, so concatenating events into one Send requires no wire-format change and no receiver change.
Proposed API additions to IMetricDataTransport (internal):
bool TryAppendOtlpProtobufEvent(byte[] buffer, int size);
void FlushOtlpProtobufEvents();
MetricUnixDomainSocketDataTransport accumulates events into an internal buffer (sized to match the existing per-event allocation, GenevaMetricExporter.BufferSize = 65360 bytes) and flushes via the existing UDS path.
MetricUnixUserEventsDataTransport and MetricWindowsEventTracingDataTransport preserve per-event semantics — each ETW / user_events write is a logical event for the consumer, so they forward TryAppend directly to SendOtlpProtobufEvent and treat Flush as a no-op.
OtlpProtobufSerializer.SendMetricPoint becomes TryAppend → Flush+retry → fallback to standalone Send (for any single event that exceeds the accumulation buffer; preserves current behavior).
SerializeAndSendMetrics calls FlushOtlpProtobufEvents() once at the end.
TlvMetricExporter uses a different framing (BinaryHeader + MetricEventType.TLV per event) and is intentionally out of scope.
This is intended to be transparent — no opt-in feature flag — since the wire format is unchanged.
Describe alternatives you've considered
- Behind a feature flag — rejected: increases surface area and adds a flag users would never want to leave off.
- Batching at the receiver side — not possible; the bottleneck is in the producer's syscall path.
- Switching to
SOCK_DGRAM — the existing transport is SOCK_STREAM, and dgram has its own per-message kernel limits; not necessary given the existing length-prefix framing.
Expected impact
- Syscall reduction: 100×–1000× depending on metric-point count per scrape.
- CPU reduction: in the profile that motivated this fix,
Interop+Sys.Send was 12.82% exclusive with the metric exporter as the majority contributor. Recovering ~10% exclusive CPU on a high-throughput worker is the realistic target.
- No wire change, no receiver (MDSD) change, no SDK change.
Additional context
Implementation ready as a PR — will link below. Tests include byte-for-byte equivalence between batched bytes and concatenated per-event sends, fallback for oversize events, and flush-failure semantics.
Component(s)
OpenTelemetry.Exporter.Geneva(metric exporter)Is your feature request related to a problem? Please describe.
The Geneva metric exporter performs one Unix-domain-socket
Socket.Sendsyscall per individual metric point during each export interval, inOtlpProtobufSerializer.SendMetricPoint.At realistic cardinality (e.g.
CardinalityLimit=4000across multiple histograms ⇒ up to ~20K points/scrape), this dominates CPU on the exporter thread. Profile evidence from a production high-throughput .NET service showsInterop+Sys.Sendat ~13% exclusive CPU, with the Geneva metric exporter as the dominant contributor.The same pattern was identified in the Rust contrib repo for the Geneva log exporter: open-telemetry/opentelemetry-rust-contrib#306. The .NET Geneva log exporter already does buffered batching; only the metric exporter has the per-event syscall pattern.
Describe the solution you'd like
Batch metric-point events into bounded UDS sends.
Wire-format guarantee: when
prefixBufferWithUInt32LittleEndianLengthis set (the only mode used byMetricUnixDomainSocketDataTransport), each serialized event is already framed with a uint32-LE length prefix. The Geneva MDSD receiver already demultiplexes multiple events from a single read by that prefix, so concatenating events into oneSendrequires no wire-format change and no receiver change.Proposed API additions to
IMetricDataTransport(internal):MetricUnixDomainSocketDataTransportaccumulates events into an internal buffer (sized to match the existing per-event allocation,GenevaMetricExporter.BufferSize= 65360 bytes) and flushes via the existing UDS path.MetricUnixUserEventsDataTransportandMetricWindowsEventTracingDataTransportpreserve per-event semantics — each ETW / user_events write is a logical event for the consumer, so they forwardTryAppenddirectly toSendOtlpProtobufEventand treatFlushas a no-op.OtlpProtobufSerializer.SendMetricPointbecomesTryAppend → Flush+retry → fallback to standalone Send(for any single event that exceeds the accumulation buffer; preserves current behavior).SerializeAndSendMetricscallsFlushOtlpProtobufEvents()once at the end.TlvMetricExporteruses a different framing (BinaryHeader+MetricEventType.TLVper event) and is intentionally out of scope.This is intended to be transparent — no opt-in feature flag — since the wire format is unchanged.
Describe alternatives you've considered
SOCK_DGRAM— the existing transport isSOCK_STREAM, and dgram has its own per-message kernel limits; not necessary given the existing length-prefix framing.Expected impact
Interop+Sys.Sendwas 12.82% exclusive with the metric exporter as the majority contributor. Recovering ~10% exclusive CPU on a high-throughput worker is the realistic target.Additional context
Implementation ready as a PR — will link below. Tests include byte-for-byte equivalence between batched bytes and concatenated per-event sends, fallback for oversize events, and flush-failure semantics.