DRIVERS-719 OpenTelementry specification #1826

nhachicha · 2025-08-08T16:04:07Z

DRIVERS-719

Update schema to 1.26
Added invalid test to unified test runner
Added nominal valid tests for the 3 tracing grouping (Operation, Cursor and Transaction)
Update description of the new field observeTracingMessages and expectTracingMessages
Merge back the draft spec into this

…logy. Adding toplogy requirements for transaction tests

source/open-telemetry/open-telemetry.md

source/unified-test-format/tests/invalid/expectedTracingSpans-spanMalformedMissingName.yml

source/unified-test-format/tests/invalid/expectedTracingSpans-spanMalformedTagsMustBeObject.yml

jmikola · 2025-08-14T00:34:23Z

source/unified-test-format/unified-test-format.md

+        - `enableCommandPayload`: Optional boolean. When set to `true`, enables capturing of command payload details in
+            tracing spans.
+            - If `true`, the test runner SHOULD capture detailed command payload information in tracing spans.
+            - If `false` or omitted, the test runner SHOULD exclude command payload details.


I realize this impacts the test runner, but should it also refer to configuring the MongoClient to enable tracing?

This point is similar to #1826 (comment) ?
This option is both configured via the test runner and should be available to toggle on/off via MongoClient. This should be covered by a prose test 👍

Co-authored-by: Jeremy Mikola <[email protected]>

…TracingMessages-additionalProperties.yml Co-authored-by: Jeremy Mikola <[email protected]>

…spanMalformedTagsMustBeObject.yml Co-authored-by: Jeremy Mikola <[email protected]>

…TracingMessages-additionalPropertyType.yml Co-authored-by: Jeremy Mikola <[email protected]>

…missingPropertySpans.yml Co-authored-by: Jeremy Mikola <[email protected]>

…spanMalformedMissingName.yml Co-authored-by: Jeremy Mikola <[email protected]>

…spanMalformedTagsMustBeObject.yml Co-authored-by: Jeremy Mikola <[email protected]>

- error.type is dropped in favour of using https://opentelemetry.io/docs/specs/semconv/exceptions/exceptions-spans/ - db.response.status_code is set in case of an error - namespace and collection names are reported as following: - 1: If the operation/command operates at the user collection level, then these fields should be set - 2: If the operation/command operates at the admin db, then only db.namespace should be set - 3: For bulkWrite collection name is not reported (db name is admin) - 4: For [abort|commit]Transaction no db name or collection reported

…//opentelemetry.io/docs/specs/semconv/registry/attributes/db/

…summary with examples - Adding test for abortTransaction - Renaming transaction test to core_api

source/open-telemetry/open-telemetry.md

ShaneHarvey · 2025-09-10T17:02:11Z

source/open-telemetry/open-telemetry.md

+- **MongoClient Level**: Drivers SHOULD provide a configuration option for `MongoClient`'s Configuration/Settings that
+    enables or disables tracing for operations and commands executed with this client. This option MUST override
+    settings on higher levels. This configuration can be implemented with a `MongoClient` option, for example,
+    `tracing.enabled`.


Agreed. We usually call out if an option should be URI, MongoClient settings, or both. Like for versioned api:
https://github.com/mongodb/specifications/blob/master/source/versioned-api/versioned-api.md#no-uri-options

Also can you clarify how tracing.enabled is supposed to be translated into code? Is it MongoClient(tracing=True), MongoClient(tracing="enabled"), or something else?

ShaneHarvey · 2025-09-10T17:03:27Z

source/open-telemetry/open-telemetry.md

+
+#### Tracer Attributes
+
+If a driver creates a Tracer using OpenTelemetry API, drivers MUST use the following attributes:


Should this say "When a driver"? Is creating a Tracer optional?

In the Ruby driver I implemented tracing attribute as a Hash, so the code should be like Mongo::Client.new('mongodb://...', tracing: { enabled: true }). Do you think we should specify the syntax here?

As mentioned above, some driver may decide not to use OpenTelemetry API. In this case they do not have a Tracer.

If you're using a Hash, do you anticipate there will be a need for additional options beyond enabled? My first impression is that such a design conflicts with the Defy Augury mantra for our specs.

Java uses MongoClientSettings builder to pass in the tracer instance (interface abstracting the underlying collector) . Drivers should be free to define this in their idiomatic way.

source/open-telemetry/open-telemetry.md

ShaneHarvey · 2025-09-10T17:20:43Z

source/open-telemetry/open-telemetry.md

+
+A common complaint from our support team is that they don't know how to easily get debugging information from drivers.
+Some drivers provide debug logging, but others do not. For drivers that do provide it, the log messages produced and the
+mechanisms for enabling debug logging are inconsistent.


Wasn't this the motivation for standardized logging? Could this section reference Standardized Logging and explain why OTel is a better (or necessary) alternative?

I think we could get the motivation part from the scope - https://docs.google.com/document/d/1h6qNCY0UCcfbGHCL4qzTCeYHf6W_IINJlLRQaTmhUZc/edit?tab=t.0

@ShaneHarvey I updated the motivation based on the scope, please let me know if you think it looks better now.

ShaneHarvey · 2025-09-10T17:22:53Z

source/open-telemetry/open-telemetry.md

+
+## Covered operations
+
+The OpenTelemetry specification covers the following operations:


Should this say something like "covers all driver operations including but not limited to"? My understanding is that every driver api should produce OTel info, even non-spec driver specific helpers.

Should this say something like "including but not limited to

Agree, in this iteration, we tried to include all the operations supported by the unified test runner. Example AFAIK there's no dropDatabase test in the UTR but the operation is reported in OTel.

Good point, thanks. Updated accordingly.

ShaneHarvey · 2025-09-10T17:25:46Z

source/open-telemetry/open-telemetry.md

+| `abortTransaction`       | [tests/transaction/transaction.yml](tests/transaction/transaction.yml)               |
+| `createCollection`       | [tests/transaction/create_collection.yml](tests/operation/create_collection.yml)     |
+| `createIndexes`          | [tests/transaction/create_indexes.yml](tests/operation/create_indexes.yml)           |
+| `createView`             | [tests/transaction/create_view.yml](tests/operation/create_view.yml)                 |


createView is not a spec method. I don't think it should be included here or in the unified tests.

It's a fair point, my understanding is that some operations predate the spec ex createView defined in https://jira.mongodb.org/browse/DRIVERS-309 or createCollection only defined in unified tests. To your point in #1826 (comment) We should aim to cover the majority of operations that will help the developer trace and troubleshoot their application

So are we going to remove createView here? Or mention that it's not standard?

It's included (with its corresponding test) https://github.com/mongodb/specifications/blob/DRIVERS-719/source/open-telemetry/open-telemetry.md#covered-operations 👍

source/open-telemetry/open-telemetry.md

jmikola · 2025-09-12T12:38:49Z

source/open-telemetry/open-telemetry.md

+A Tracer is responsible for creating spans, and using a tracer is the only way to create a span. A Tracer is not
+responsible for configuration; this should be the responsibility of the TracerProvider instead.
+
+**OpenTelemetry API and SDK**


Is this section intended to be under ### Terms? It definitely doesn't look like it belongs under #### Tracer.

It seems relevant to the paragraph under #### External Dependencies below, so perhaps drop the title here and just move the following paragraph to that section.

I completely removed the Tracer section, and moved the API and SDK to the External Dependencies.

jmikola · 2025-09-12T12:40:46Z

source/open-telemetry/open-telemetry.md

+
+OpenTelemetry SHOULD be disabled by default.
+
+Drivers SHOULD support configuring OpenTelemetry on multiple levels.


@nhachicha @comandeo-mongo: Feel free to resolve this if the question is no longer relevant. Otherwise, respond and I'll resolve after reading.

jmikola · 2025-09-12T12:44:14Z

source/open-telemetry/open-telemetry.md

+- **MongoClient Level**: Drivers SHOULD provide a configuration option for `MongoClient`'s Configuration/Settings that
+    enables or disables tracing for operations and commands executed with this client. This option MUST override
+    settings on higher levels. This configuration can be implemented with a `MongoClient` option, for example,
+    `tracing.enabled`.


@ShaneHarvey: thanks for bringing up the typing. If this really is just a boolean type, I think tracing or enableTracing (more active language than tracingEnabled) with a default value of false would be most consistent with other driver option names in PHP. But since these aren't URI options there should be more naming flexibility available for drivers.

It looks like @comandeo-mongo is talking about this in a comment below, too: https://github.com/mongodb/specifications/pull/1826/files#r2339648026

jmikola · 2025-09-12T12:47:58Z

source/open-telemetry/open-telemetry.md

+
+#### Tracer Attributes
+
+If a driver creates a Tracer using OpenTelemetry API, drivers MUST use the following attributes:


If you're using a Hash, do you anticipate there will be a need for additional options beyond enabled? My first impression is that such a design conflicts with the Defy Augury mantra for our specs.

jmikola · 2025-09-15T17:01:17Z

source/open-telemetry/open-telemetry.md

+
+This attribute contains the full database command executed serialized to extended JSON. If not truncated, the content of
+this attribute SHOULD be equivalent to the `document` field of the CommandStartedEvent of the command monitoring
+excluding the following fields: `lsid`, `$db`, `$clusterTime`, `signature`.


The PHP team was recently discussing query shapes and how one might remove dynamic values (e.g. 5 in { x: { $gt: 5 }}) to yield a parameterized query for purposes of logging or query aggregation. In the case of CSFLE, the query might already have BSON binary values (with the encrypted subtype) by the time the query is sent along for logging, which can also be considered a form of parameterization. In that case, a driver could easily replace those binary values with placeholders (assuming the encrypted blobs are irrelevant for logging).

There's may also be some overlap here with making security redactions for other, sensitive commands.

With regard to this spec, I'd suggest mentioning query parameterization as a potential todo item in a "Future Work" section (as is found in other specs).

cc: @GromNaN @paulinevos @alcaeus

jmikola · 2025-09-15T17:03:16Z

source/open-telemetry/tests/cursor/cursor.yml

@@ -0,0 +1,129 @@
+description: cursor retrieval


If it's not too much work, I'd advocate for making these descriptions consistent with the base filename (sans .yml extension). So if "cursor retrieval" is the logical name to group tests in this file, I'd suggest using cursor-retrieval here and naming the file cursor-retrieval.yml.

jmikola · 2025-09-15T17:04:48Z

source/open-telemetry/tests/cursor/cursor.yml

+                  exception.type: { $$exists: false }
+                  exception.stacktrace: { $$exists: false }
+                  server.address: { $$type: string }
+                  server.port: { $$type: ['int', 'long'] }


I noticed you don't quote single values for $$type above, so consider using [int, long] here in the interest of consistency and keeping the tests as concise as possible (without sacrificing readability and introducing ambiguity in YAML parsing).

jmikola · 2025-09-15T17:05:47Z

source/open-telemetry/tests/operation/aggregate.yml

+      - name: aggregate
+        object: *collection0
+        arguments:
+          pipeline: &pipeline1


Nit: was this test originally taken from a larger file? I would have expected &pipeline0 here.

source/open-telemetry/tests/operation/atlas_search.yml

Co-authored-by: Jeremy Mikola <[email protected]>

nhachicha added 5 commits July 11, 2025 18:57

added first tracing tests

97d8a5b

using unsetOrMatches for txNumber, since it's not set for single topo…

c82a6a2

…logy. Adding toplogy requirements for transaction tests

updating tests

61270d6

Tests and schema update for the upcoming Otel spec

581faa1

Merge branch 'nh/tracing'

c447d10

nhachicha self-assigned this Aug 8, 2025

nhachicha and others added 5 commits August 8, 2025 16:05

Update generated files

3a578ee

update schema to latest (1.26)

dbb1a9c

update tests

29e25a1

DRIVERS-719 OpenTelementry specification

f801d78

Cursor operations

9d66a8a

nhachicha changed the title ~~OpenTelemetry~~ DRIVERS-719 OpenTelementry specification Aug 8, 2025

nhachicha assigned comandeo-mongo Aug 8, 2025

nhachicha mentioned this pull request Aug 8, 2025

DRIVERS-719 OpenTelementry specification #1821

Closed

3 tasks

GromNaN reviewed Aug 8, 2025

View reviewed changes

source/open-telemetry/open-telemetry.md Outdated Show resolved Hide resolved

source/open-telemetry/open-telemetry.md Show resolved Hide resolved

nhachicha added 6 commits August 9, 2025 16:54

update doc

269221c

moved tests under open-telemtry directroy

452696d

Fixing formatting

9f05c59

update formatting using pre-commit mdformat

0523751

Adding Test README

8466c1b

Adding retries test, fixing duplicate exception tag

18ad723

jmikola reviewed Aug 14, 2025

View reviewed changes

nhachicha and others added 8 commits August 14, 2025 17:48

Update source/open-telemetry/open-telemetry.md

04b947a

Co-authored-by: Jeremy Mikola <[email protected]>

Update source/open-telemetry/open-telemetry.md

88c7be0

Co-authored-by: Jeremy Mikola <[email protected]>

Update source/unified-test-format/tests/invalid/entity-client-observe…

ced82eb

…TracingMessages-additionalProperties.yml Co-authored-by: Jeremy Mikola <[email protected]>

Update source/unified-test-format/tests/invalid/expectedTracingSpans-…

f1804c3

…spanMalformedTagsMustBeObject.yml Co-authored-by: Jeremy Mikola <[email protected]>

Update source/unified-test-format/tests/invalid/entity-client-observe…

03d4e3c

…TracingMessages-additionalPropertyType.yml Co-authored-by: Jeremy Mikola <[email protected]>

Update source/unified-test-format/tests/invalid/expectedTracingSpans-…

3c69ed5

…missingPropertySpans.yml Co-authored-by: Jeremy Mikola <[email protected]>

Update source/unified-test-format/tests/invalid/expectedTracingSpans-…

240d009

…spanMalformedMissingName.yml Co-authored-by: Jeremy Mikola <[email protected]>

Update source/unified-test-format/tests/invalid/expectedTracingSpans-…

6a4d217

…spanMalformedTagsMustBeObject.yml Co-authored-by: Jeremy Mikola <[email protected]>

comandeo-mongo and others added 6 commits September 2, 2025 17:42

Adjust spec format

f78b541

Clarify cursor operation and transactions nesting

67d4c90

Fix bulkWrite test

b478c9e

Merge remote-tracking branch 'origin/master' into DRIVERS-719

7d0d83d

Fixing typo in test file

0edc807

nhachicha force-pushed the DRIVERS-719 branch from 8532f58 to 0edc807 Compare September 9, 2025 09:55

nhachicha and others added 7 commits September 9, 2025 11:15

Fixing test

3822daf

Added convenient transaction test

9987dbf

Bring spec in accordance with the new tests

51fc929

db.response.status_code type is string per the OTel convention https:…

2841df3

…//opentelemetry.io/docs/specs/semconv/registry/attributes/db/

Fix review remarks

5af87ec

Fix review remarks

46bc709

Fix section names

ae0d15e

nhachicha requested a review from ShaneHarvey September 10, 2025 12:24

nhachicha marked this pull request as ready for review September 10, 2025 12:24

nhachicha requested review from a team as code owners September 10, 2025 12:24

nhachicha requested review from alcaeus, jmikola, GromNaN and blink1073 and removed request for a team and blink1073 September 10, 2025 12:24

- Clarifying rules for db.namespace, db.collection.name and db.query.…

52974c0

…summary with examples - Adding test for abortTransaction - Renaming transaction test to core_api

ShaneHarvey requested changes Sep 10, 2025

View reviewed changes

jmikola reviewed Sep 15, 2025

View reviewed changes

comandeo-mongo and others added 3 commits September 16, 2025 10:24

Update source/open-telemetry/open-telemetry.md

00c1dc3

Co-authored-by: Jeremy Mikola <[email protected]>

Respond to code review remarks

cafe19c

Update source/open-telemetry/tests/operation/atlas_search.yml

99f7ed3

Co-authored-by: Jeremy Mikola <[email protected]>


		#### Tracer Attributes

		If a driver creates a Tracer using OpenTelemetry API, drivers MUST use the following attributes:


		## Covered operations

		The OpenTelemetry specification covers the following operations:


		OpenTelemetry SHOULD be disabled by default.

		Drivers SHOULD support configuring OpenTelemetry on multiple levels.

DRIVERS-719 OpenTelementry specification #1826

Are you sure you want to change the base?

DRIVERS-719 OpenTelementry specification #1826

Conversation

nhachicha commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ShaneHarvey Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ShaneHarvey Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nhachicha commented Aug 8, 2025 •

edited

Loading

ShaneHarvey Sep 10, 2025 •

edited

Loading

ShaneHarvey Sep 10, 2025 •

edited

Loading