Console exporter for OTAP/OTLP logs #1849

jmacd · 2026-01-21T03:14:02Z

Part of #1771.

Part of #1736.

Uses the new internal logging support to format OTLP logs data. This prints RESOURCE and SCOPE lines with ASCII or Unicode pipe structures to identify the OTLP hierarchy:

2026-01-21T03:12:22.165Z  RESOURCE   v1.Resource: [fake_data_generator=v1]
2026-01-21T03:12:22.165Z  │ SCOPE    v1.InstrumentationScope:
2026-01-21T03:12:22.165Z  │ ├─ INFO  session.start:  [session.id=00112233-4455-6677-8899-aabbccddeeff, session.previous_id=00112233-4455-6677-8899-aabbccddeeff]
2026-01-21T03:12:22.165Z  │ ├─ INFO  session.end:  [session.id=00112233-4455-6677-8899-aabbccddeeff]
2026-01-21T03:12:22.165Z  │ ├─ INFO  device.app.lifecycle:  [ios.app.state=active, android.app.state=created]
2026-01-21T03:12:22.165Z  │ ├─ INFO  rpc.message:  [rpc.message.type=SENT, rpc.message.id=42, rpc.message.compressed_size=42, rpc.message.uncompressed_size=42]

…d/console_exp_9

codecov · 2026-01-21T03:19:23Z

Codecov Report

❌ Patch coverage is 80.00000% with 87 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.69%. Comparing base (9ef8217) to head (45a6177).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1849      +/-   ##
==========================================
- Coverage   84.72%   84.69%   -0.03%     
==========================================
  Files         503      504       +1     
  Lines      149956   150323     +367     
==========================================
+ Hits       127048   127322     +274     
- Misses      22374    22467      +93     
  Partials      534      534

Components	Coverage Δ
otap-dataflow	`86.07% <80.00%> (-0.05%)`	⬇️
query_abstraction	`80.61% <ø> (ø)`
query_engine	`90.52% <ø> (ø)`
syslog_cef_receivers	`∅ <ø> (∅)`
otel-arrow-go	`53.50% <ø> (ø)`
quiver	`90.66% <ø> (ø)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

cijothomas · 2026-01-21T04:45:34Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+//! Console exporter that prints OTLP data with hierarchical formatting.


do we want to offer stability guarantees about output format or we want to explicitly mention that output format is not stable and not recommended to be relied for automation/parsing.?

Good question. We've seen users relying on SDK's ConsoleExporter in production with the expectation that the output format won't change.

Noted as unstable for now! We can open an explicit stabilization issue.

cijothomas · 2026-01-21T04:47:34Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+        let event_name = log_record
+            .event_name()
+            .map(|s| String::from_utf8_lossy(s).into_owned())
+            .unwrap_or_else(|| "event".to_string());


why "event" instead of leaving it empty?

That was a mistake, fixed.

utpilla · 2026-01-21T16:10:13Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+            OtapPayload::OtlpBytes(bytes) => {
+                self.formatter.format_logs_bytes(bytes);
+            }
+            OtapPayload::OtapArrowRecords(records) => match OtapLogsView::try_from(records) {
+                Ok(logs_view) => {
+                    self.formatter.format_logs_arrow(&logs_view);
+                }


It'd be nice to implement impl<'a> TryFrom<&'a OtlpProtoBytes> for OtapLogsView<'a> as well similar to OtapArrowRecords. That would simplify the code here by having both paths go through the same set of steps.

OtlpBytes/OtapArrowRecords -> OtapLogsView -> Write to console

@utpilla - Wondering if this would need decoding the protobuf into rows and then building Arrow arrays (and the OTAP layout). How about keeping RawLogsData for OTLP bytes and OtapLogsView for Arrow; a generic format_logs<L: LogsDataView> to keep call sites tidy?

This should not require decoding! Trying...

Oh yes, but looking into the code, there may be lifetime issue.

I cleaned this up somewhat, but yes, I'm not sure how to handle this kind of try-various-impls pattern here. I did add a try_from so that at least the two are symmetric:

fn export_logs(&self, payload: &OtapPayload) { match payload { OtapPayload::OtlpBytes(bytes) => match RawLogsData::try_from(bytes) { Ok(logs_view) => { self.formatter.print_logs_data(&logs_view); } Err(e) => { otel_error!("Failed to create OTLP logs view", error = ?e); } }, OtapPayload::OtapArrowRecords(records) => match OtapLogsView::try_from(records) { Ok(logs_view) => { self.formatter.print_logs_data(&logs_view); } Err(e) => { otel_error!("Failed to create OTAP logs view", error = ?e); } }, } }

This looks good actually. The trait-based approach is the right pattern here.

utpilla · 2026-01-21T16:18:00Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+
+impl ConsoleExporter {
+    fn export(&self, data: &OtapPdata) {
+        let (_, payload) = data.clone().into_parts();


We could avoid the cloning of payload here for each export. We are only using references here in the export_* methods. It could be worth adding another method to OtapPData which returns the reference to payload without consuming the payload:

/// Returns a reference to the payload #[must_use] pub const fn payload_ref(&self) -> &OtapPayload { &self.payload }

Done! 3985183

utpilla · 2026-01-21T16:23:02Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+            .map(nanos_to_time)
+            .unwrap_or(SystemTime::UNIX_EPOCH);
+
+        let prefix = format!("{} ", self.tree.vertical);


This would allocate a new string for each scope. Since tree.vertical is a &static str, we could pre-compute the prefix as a const.

utpilla · 2026-01-21T16:33:16Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+        // Format each scope
+        let scopes: Vec<_> = resource_logs.scopes().collect();
+        let scope_count = scopes.len();
+        for (i, scope_logs) in scopes.into_iter().enumerate() {
+            let is_last_scope = i == scope_count - 1;
+            self.format_scope_logs_to(&scope_logs, is_last_scope, output);
+        }


It looks like we're allocating a vec here just to identify the last element. Could we use peekable() instead to avoid the allocation?

utpilla · 2026-01-21T16:33:39Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+        // Format each log record
+        let records: Vec<_> = scope_logs.log_records().collect();
+        let record_count = records.len();
+        for (i, log_record) in records.into_iter().enumerate() {
+            let is_last_record = i == record_count - 1;
+            self.format_log_record_to(&log_record, is_last_scope, is_last_record, output);
+        }


Same as #1849 (comment)

Fixed! Thanks. 978d44b

utpilla · 2026-01-21T16:36:02Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+        let version = scope
+            .as_ref()
+            .and_then(|s| s.version())
+            .map(|v| String::from_utf8_lossy(v).into_owned());


We could rid of the into_owned and avoid the string allocation here by using a Cow<str> or raw bytes directly.

Done (raw bytes works).

rust/otap-dataflow/crates/otap/src/console_exporter.rs

utpilla · 2026-01-21T16:47:45Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+    pub fn format_logs_bytes(&self, bytes: &OtlpProtoBytes) {
+        let mut output = Vec::new();
+        self.format_logs_bytes_to(bytes, &mut output);
+        let _ = std::io::stdout().write_all(&output);


Use the async version?

let _ = tokio::io::stdout().write_all(&output).await;

utpilla · 2026-01-21T16:49:14Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+    pub fn format_logs_arrow(&self, logs_view: &OtapLogsView<'_>) {
+        let mut output = Vec::new();
+        self.format_logs_data_to(logs_view, &mut output);
+        let _ = std::io::stdout().write_all(&output);


Do we want to log errors here if write_all fails?

Now reads:

if let Err(err) = tokio::io::stdout().write_all(&output).await { otel_error!("could not write to console", error = ?err); }

utpilla

Most of my comments are perf related which may not be as much of a concern for a ConsoleExporter.

There is one comment about using the std I/O blocking write_all vs tokio's async std I/O which I think we should address.

jmacd · 2026-01-21T17:04:16Z

I appreciate all of this feedback! I see this component function as not a high priority, however all the points made about performance are likely to impact other export paths so I wouldn't dismiss any it.

lalitb · 2026-01-21T18:01:49Z

rust/otap-dataflow/crates/telemetry/src/self_tracing/formatter.rs

+    /// Write severity number with appropriate color and padding.
+    /// Severity numbers follow OTLP conventions (1-24, where INFO=9).
+    #[inline]
+    pub fn write_severity(&self, w: &mut BufWriter<'_>, severity: Option<i32>) {


should we extend this method to also write severity-text along with the severity number?

ah, yes! I had not considered severity text since I dropped it from the LogRecord encoding as being redundant for the 5 Tokio levels.

Added, tested.

lalitb · 2026-01-21T18:27:54Z

rust/otap-dataflow/crates/otap/src/console_exporter.rs

+    where
+        F: FnOnce(&mut BufWriter<'_>),
+    {
+        let mut buf = [0u8; LOG_BUFFER_SIZE];


More for later PR, should be good to add TODO for now - this is silently truncating if the writable size if over 4KB. Which means - A log record with a big body or lots of attributes will get cut off without warning. We can detect overflow, and append truncation marker, so user doesn't get misled with truncated content. Or else not to truncate by using growable buffer.

I have this and some related topics recorded in #1746 🚀

lalitb

LGTM, with non-blocking comments.

…d/console_exp_9

…o jmacd/console_exp_9

Co-authored-by: Utkarsh Umesan Pillai <66651184+utpilla@users.noreply.github.com>

Part of open-telemetry#1771. Part of open-telemetry#1736. Uses the new internal logging support to format OTLP logs data. This prints RESOURCE and SCOPE lines with ASCII or Unicode pipe structures to identify the OTLP hierarchy: ``` 2026-01-21T03:12:22.165Z RESOURCE v1.Resource: [fake_data_generator=v1] 2026-01-21T03:12:22.165Z │ SCOPE v1.InstrumentationScope: 2026-01-21T03:12:22.165Z │ ├─ INFO session.start: [session.id=00112233-4455-6677-8899-aabbccddeeff, session.previous_id=00112233-4455-6677-8899-aabbccddeeff] 2026-01-21T03:12:22.165Z │ ├─ INFO session.end: [session.id=00112233-4455-6677-8899-aabbccddeeff] 2026-01-21T03:12:22.165Z │ ├─ INFO device.app.lifecycle: [ios.app.state=active, android.app.state=created] 2026-01-21T03:12:22.165Z │ ├─ INFO rpc.message: [rpc.message.type=SENT, rpc.message.id=42, rpc.message.compressed_size=42, rpc.message.uncompressed_size=42] ``` --------- Co-authored-by: Utkarsh Umesan Pillai <66651184+utpilla@users.noreply.github.com>

jmacd added 5 commits January 20, 2026 15:33

wip

808d611

simpler

e322778

looks good

36916db

test console exp

3bb0fa1

Merge branch 'main' of github.com:open-telemetry/otel-arrow into jmac…

31081e6

…d/console_exp_9

jmacd requested a review from a team as a code owner January 21, 2026 03:14

github-project-automation bot added this to OTel-Arrow Jan 21, 2026

github-actions bot added the rust Pull requests that update Rust code label Jan 21, 2026

fmt

aaf3334

cijothomas reviewed Jan 21, 2026

View reviewed changes

utpilla reviewed Jan 21, 2026

View reviewed changes

rust/otap-dataflow/crates/otap/src/console_exporter.rs Outdated Show resolved Hide resolved

utpilla reviewed Jan 21, 2026

View reviewed changes

utpilla approved these changes Jan 21, 2026

View reviewed changes

make event name optional

0aa4b62

lalitb reviewed Jan 21, 2026

View reviewed changes

lalitb approved these changes Jan 21, 2026

View reviewed changes

jmacd added 3 commits January 21, 2026 13:44

simplify/fix test

94758cb

Merge branch 'main' of github.com:open-telemetry/otel-arrow into jmac…

7c7a291

…d/console_exp_9

wip

39ddba1

jmacd marked this pull request as draft January 21, 2026 22:53

jmacd and others added 10 commits January 21, 2026 17:10

nicer

61fa657

Merge branch 'main' of github.com:open-telemetry/otel-arrow into jmac…

9664fe3

…d/console_exp_9

Merge branch 'jmacd/console_exp_9' of github.com:jmacd/otel-arrow int…

195d8f6

…o jmacd/console_exp_9

payload_ref()

3985183

remove dead format

d81ff77

peekable x 2

978d44b

use raw bytes not strings for scope name/version:

ee543d3

Update rust/otap-dataflow/crates/otap/src/console_exporter.rs

5f01f2e

Co-authored-by: Utkarsh Umesan Pillai <66651184+utpilla@users.noreply.github.com>

async print / tokio write_all

dc2fabc

test severity_text

332349a

jmacd marked this pull request as ready for review January 22, 2026 04:21

lint

45a6177

jmacd added this pull request to the merge queue Jan 22, 2026

Merged via the queue into open-telemetry:main with commit 2fae2f7 Jan 22, 2026
43 of 44 checks passed

jmacd deleted the jmacd/console_exp_9 branch January 22, 2026 05:10

github-project-automation bot moved this to Done in OTel-Arrow Jan 22, 2026

Console exporter for OTAP/OTLP logs #1849

Console exporter for OTAP/OTLP logs #1849

Uh oh!

Conversation

jmacd commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

utpilla Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lalitb Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmacd Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

utpilla left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmacd commented Jan 21, 2026

Uh oh!

lalitb Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmacd commented Jan 21, 2026 •

edited

Loading

codecov bot commented Jan 21, 2026 •

edited

Loading

utpilla Jan 21, 2026 •

edited

Loading

lalitb Jan 21, 2026 •

edited

Loading

jmacd Jan 22, 2026 •

edited

Loading

utpilla left a comment •

edited

Loading

lalitb Jan 21, 2026 •

edited

Loading

lalitb Jan 21, 2026 •

edited

Loading