feat(codecs): convert native Vector logs to OTLP in otlp serializer by thomasqueirozb · Pull Request #26605 · vectordotdev/vector

thomasqueirozb · 2026-10-08T21:50:41Z

Summary

The otlp codec now converts native Vector log events (without a resourceLogs field) to OTLP log records instead of rejecting them. The mapping is the inverse of the opentelemetry source decoding, for both log namespaces. Fields that have no OTLP slot, or that have the wrong type for one, are sent as log record attributes instead of being dropped.

Traces are handled separately on top of #26590.

References

Related: feat(opentelemetry sink): add automatic native log and trace to OTLP conversion #24621

Vector configuration

sources:
  demo:
    type: demo_logs
    format: syslog
    interval: 0.5
    count: 3

transforms:
  enrich:
    type: remap
    inputs: [demo]
    source: |
      .severity_text = "INFO"
      .severity_number = 9
      .trace_id = "0102030405060708090a0b0c0d0e0f10"
      .span_id = "not-a-span-id"
      .resources."service.name" = "smoke"
      .attributes.region = "eu-west-1"

sinks:
  to_collector:
    type: opentelemetry
    inputs: [enrich]
    protocol:
      type: http
      uri: http://127.0.0.1:44318/v1/logs
      encoding:
        codec: otlp

How did you test this PR?

Unit tests: OTLP -> Vector -> OTLP round trip in both log namespaces gives an identical request; native log mapping; fields with invalid OTLP values kept as attributes; Vector namespace fallbacks.
Sent the config above to an OpenTelemetry Collector 0.111.0 (debug exporter) and to the Vector opentelemetry source. Both received the expected body, timestamps, severity, trace ID, and resource and record attributes.

Does this PR include user facing changes?

Yes. Please add a changelog fragment based on our guidelines.
No. A maintainer will apply the no-changelog label to this PR.

thomasqueirozb · 2026-10-08T22:36:29Z

@codex review

chatgpt-codex-connector · 2026-10-08T22:36:47Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-09T22:31:43.671287Z	`47b9ab0`	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector · 2026-10-08T22:38:50Z

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 751cb62de9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 751cb62de9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

datadoghq-integration

Legacy logs using metadata-prefixed message paths lose their bodies during conversion. All-zero trace and span IDs also bypass the invalid-value attribute fallback.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 751cb62 · @DataDog review to ask questions}

… key

… round trips

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 48f4e0c219

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…s in native log round trips

…y resource and scope

… grouping

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d74d78f69a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-10-09T19:08:39Z

+    match hex::decode(&hex) {
+        Ok(id) if id.is_empty() || id.len() == LEN => Ok(id),
+        _ => Err(Value::Bytes(hex)),


Reject all-zero trace and span identifiers

When a native event supplies a correctly sized all-zero trace_id or span_id, this length-only check accepts it and removes the original field from the attributes. The vendored OTLP definition explicitly considers all-zero IDs invalid, and receivers are instructed to treat the record as unassociated, so the identifier is effectively lost instead of being retained as an attribute like other invalid IDs. Require at least one nonzero byte in addition to the expected length.

Useful? React with 👍 / 👎.

Declining, same as #26605 (comment). We send back the same bytes we received. It is not the encoder's job to check that a trace_id is valid.

…mestamp key

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7411498c2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

… into feat/otlp-native-logs

…trings

datadoghq-integration

Legacy logs whose configured timestamp is nested under the message lose their OTLP event time. The new tests also lack an isolated-crate dependency feature and conflict when run in one process.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 3c57c6a}

datadoghq-integration · 2026-10-09T21:57:52Z

+            if let Some(path) = schema.message_key_target_path() {
+                record.body = log.remove_prune(path, true).and_then(into_body);
+            }
+            // The timestamp key can also point into metadata. Many sources and decoders write it
+            // there with its full target path, but the `opentelemetry` source writes it to the
+            // event root, so read the metadata first and then the event root.
+            let metadata_time = schema
+                .timestamp_key_target_path()
+                .filter(|path| path.prefix == PathPrefix::Metadata)
+                .and_then(|path| log.remove(path))
+                .and_then(|value| into_timestamp_nanos(value).ok());
+            let (mut fields, _) = log.into_parts();
+            if let Some(path) = source_type_key {
+                fields.remove(&path.path, true);
+            }
+            record.time_unix_nano = metadata_time
+                .or_else(|| {
+                    schema
+                        .timestamp_key()
+                        .and_then(|path| take(&mut fields, path, into_timestamp_nanos))
+                })
+                .unwrap_or_default();


Read timestamps before removing the message subtree

With message_key = .payload and timestamp_key = .payload.timestamp, removing the message subtree erases the timestamp before conversion reads it. A valid event timestamp therefore becomes time_unix_nano = 0, including for decoded OpenTelemetry logs. Cache the timestamp before removing the body so collectors retain the event's original time.

Suggested change

if let Some(path) = schema.message_key_target_path() {

record.body = log.remove_prune(path, true).and_then(into_body);

}

// The timestamp key can also point into metadata. Many sources and decoders write it

// there with its full target path, but the `opentelemetry` source writes it to the

// event root, so read the metadata first and then the event root.

let metadata_time = schema

.timestamp_key_target_path()

.filter(|path| path.prefix == PathPrefix::Metadata)

.and_then(|path| log.remove(path))

.and_then(|value| into_timestamp_nanos(value).ok());

let (mut fields, _) = log.into_parts();

if let Some(path) = source_type_key {

fields.remove(&path.path, true);

}

record.time_unix_nano = metadata_time

.or_else(|| {

schema

.timestamp_key()

.and_then(|path| take(&mut fields, path, into_timestamp_nanos))

})

.unwrap_or_default();

// The configured timestamp can be inside the message subtree.

let timestamp_before_body = schema

.timestamp_key_target_path()

.and_then(|path| log.get(path))

.and_then(Value::as_timestamp)

.and_then(timestamp_nanos)

.or_else(|| {

schema

.timestamp_key()

.and_then(|path| log.get((PathPrefix::Event, path)))

.and_then(Value::as_timestamp)

.and_then(timestamp_nanos)

});

if let Some(path) = schema.message_key_target_path() {

record.body = log.remove_prune(path, true).and_then(into_body);

}

// The timestamp key can also point into metadata. Many sources and decoders write it

// there with its full target path, but the `opentelemetry` source writes it to the

// event root, so read the metadata first and then the event root.

let metadata_time = schema

.timestamp_key_target_path()

.filter(|path| path.prefix == PathPrefix::Metadata)

.and_then(|path| log.remove(path))

.and_then(|value| into_timestamp_nanos(value).ok());

let (mut fields, _) = log.into_parts();

if let Some(path) = source_type_key {

fields.remove(&path.path, true);

}

let extracted_time = metadata_time.or_else(|| {

schema

.timestamp_key()

.and_then(|path| take(&mut fields, path, into_timestamp_nanos))

});

record.time_unix_nano = timestamp_before_body.or(extracted_time).unwrap_or_default();

Was this helpful? React 👍 or 👎
_{🤖 Bits Code Review} _{· Open Bits AI session}

datadoghq-integration · 2026-10-09T21:57:52Z

+    #[test]
+    fn native_legacy_log_reads_metadata_timestamp_key() {
+        init_metadata_timestamp_key();
+        let mut log = LogEvent::from("disk full");


Enable the constructor feature for isolated crate tests

The new LogEvent::from(&str) calls cannot compile when opentelemetry-proto is tested independently: vector-core gates that implementation behind its test feature, which this crate does not enable. Workspace feature unification can hide the failure. Enable vector-core/test through a development dependency or replace all affected calls with the production constructor.

Was this helpful? React 👍 or 👎
_{🤖 Bits Code Review} _{· Open Bits AI session}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 47b9ab03d4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-10-09T22:31:41Z

+            if let Some(path) = schema.message_key_target_path() {
+                record.body = log.remove_prune(path, true).and_then(into_body);


Remove nested source markers before extracting the body

When log_schema.source_type_key is nested beneath log_schema.message_key (for example, .payload.source_type beneath .payload), this removes the entire message into record.body before the later source-type removal runs against the remaining fields. Logs produced by sources that insert the marker at the configured path therefore send the internal source_type marker inside the OTLP body, despite the documented promise to omit it; strip the applicable marker from the extracted body as well.

Useful? React with 👍 / 👎.

feat(codecs): convert native Vector logs to OTLP in otlp serializer

751cb62

github-actions Bot added domain: external docs Anything related to Vector's external, public documentation docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. labels Oct 8, 2026

thomasqueirozb mentioned this pull request Oct 8, 2026

feat(codecs): convert native OTLP traces through the typed trace model in otlp serializer #26606

Draft

1 of 2 tasks

thomasqueirozb marked this pull request as ready for review October 8, 2026 22:41

thomasqueirozb requested review from a team as code owners October 8, 2026 22:41

chatgpt-codex-connector Bot reviewed Oct 8, 2026

View reviewed changes

Comment thread lib/opentelemetry-proto/src/logs.rs

datadoghq-integration Bot reviewed Oct 8, 2026

View reviewed changes

Comment thread lib/opentelemetry-proto/src/logs.rs Outdated

Comment thread lib/opentelemetry-proto/src/logs.rs

thomasqueirozb added 2 commits October 9, 2026 11:17

fix(codecs): read Legacy OTLP log body from metadata-prefixed message…

e5749f7

… key

docs(opentelemetry sink): note schema URLs are not kept in native log…

48f4e0c

… round trips

chatgpt-codex-connector Bot reviewed Oct 9, 2026

View reviewed changes

Comment thread lib/opentelemetry-proto/src/logs.rs Outdated

Comment thread lib/opentelemetry-proto/src/common.rs Outdated

thomasqueirozb added 5 commits October 9, 2026 13:26

docs(opentelemetry sink): document bytes and missing timestamp change…

39eae8a

…s in native log round trips

docs(codecs): add native log example to otlp changelog

05493b5

docs(opentelemetry sink): document reserved OTLP root fields

d74d78f

docs(opentelemetry sink): note native log records are not regrouped b…

984a1b6

…y resource and scope

docs(opentelemetry sink): point to use_otlp_decoding to keep OTLP log…

2fae247

… grouping

chatgpt-codex-connector Bot reviewed Oct 9, 2026

View reviewed changes

fix(codecs): read Legacy OTLP log timestamp from metadata-prefixed ti…

f741149

…mestamp key

chatgpt-codex-connector Bot reviewed Oct 9, 2026

View reviewed changes

Comment thread lib/opentelemetry-proto/src/logs.rs Outdated

thomasqueirozb added 3 commits October 9, 2026 16:59

fix(codecs): preserve payload source type with metadata markers

6b94fec

Merge remote-tracking branch 'origin/fix/bump-vrl-as-str-deprecation'…

c9fe8a4

… into feat/otlp-native-logs

refactor(opentelemetry): use Value::to_str_lossy when encoding OTLP s…

3c57c6a

…trings

thomasqueirozb changed the base branch from master to fix/bump-vrl-as-str-deprecation October 9, 2026 21:50

thomasqueirozb added this pull request to stack #26634 October 9, 2026 21:50

datadoghq-integration Bot reviewed Oct 9, 2026

View reviewed changes

test(opentelemetry): enable helpers for isolated crate tests

dafb63d

fix(codecs): preserve timestamps nested in log messages

47b9ab0

chatgpt-codex-connector Bot reviewed Oct 9, 2026

View reviewed changes

		if let Some(path) = schema.message_key_target_path() {
		record.body = log.remove_prune(path, true).and_then(into_body);

Conversation

thomasqueirozb commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

References

Vector configuration

How did you test this PR?

Does this PR include user facing changes?

Uh oh!

thomasqueirozb commented Oct 8, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot commented Oct 8, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

datadoghq-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

thomasqueirozb Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

datadoghq-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

datadoghq-integration Bot Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

datadoghq-integration Bot Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

thomasqueirozb commented Oct 8, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 8, 2026 •

edited

Loading