Skip to content

Commit 917747f

Browse files
eschabelleschabell
committed
Updated opentelemetry input plugin docs to reflect several enhancements for 4.2. Fixes #2209.
Signed-off-by: Eric D. Schabell <[email protected]>
1 parent 48cbc51 commit 917747f

File tree

1 file changed

+104
-3
lines changed

1 file changed

+104
-3
lines changed

‎pipeline/inputs/opentelemetry.md‎

Lines changed: 104 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ Fluent Bit has a compliant implementation which fully supports `OTLP/HTTP` and `
3232
| `net.share_port` | Allow multiple plugins to bind to the same port. | `false` |
3333
| `port` | The port for Fluent Bit to listen for incoming connections. | `4318` |
3434
| `profiles_support` | This is an experimental feature, feel free to test it but don't enable this in production environments. | `false` |
35-
| `raw_traces` | Forward traces without processing. | `false` |
35+
| `raw_traces` | Forward traces without processing. When set to `false` (default), traces are processed using the unified JSON parser with strict validation. When set to `true`, trace data is forwarded as raw log messages without validation or processing. | `false` |
3636
| `routable` | If set to `true`, the data generated by the plugin will be routable, meaning that it can be forwarded to other plugins or outputs. If set to `false`, the data will be discarded. | `true` |
3737
| `storage.pause_on_chunks_overlimit` | Enable pausing on an input when they reach their chunks limit. | _none_ |
3838
| `storage.type` | Sets the storage type for this input, one of: `filesystem`, `memory` or `memrb`. | `memory` |
@@ -57,7 +57,7 @@ Fluent Bit has a compliant implementation which fully supports `OTLP/HTTP` and `
5757
| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` |
5858
| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ |
5959

60-
Raw traces means that any data forwarded to the traces endpoint (`/v1/traces`) will be packed and forwarded as a log message, and won't be processed by Fluent Bit. The traces endpoint by default expects a valid `protobuf` encoded payload, but you can set the `raw_traces` option in case you want to get trace telemetry data to any of the Fluent Bit supported outputs.
60+
When `raw_traces` is set to `false` (default), the traces endpoint (`/v1/traces`) processes incoming trace data using the unified JSON parser with strict validation. The endpoint accepts both `protobuf` and `JSON` encoded payloads. When `raw_traces` is set to `true`, any data forwarded to the traces endpoint will be packed and forwarded as a log message without processing, validation, or conversion to the Fluent Bit internal trace format.
6161

6262
### OpenTelemetry transport protocol endpoints
6363

@@ -92,7 +92,7 @@ The OpenTelemetry input plugin supports the following telemetry data types:
9292
|---------|---------------|----------------|------------|
9393
| Logs | Stable | Stable | Stable |
9494
| Metrics | Unimplemented | Stable | Stable |
95-
| Traces | Unimplemented | Stable | Stable |
95+
| Traces | Stable | Stable | Stable |
9696

9797
A sample configuration file to get started will look something like the following:
9898

@@ -135,3 +135,104 @@ A sample curl request to POST JSON encoded log data would be:
135135
```shell
136136
curl --header "Content-Type: application/json" --request POST --data '{"resourceLogs":[{"resource":{},"scopeLogs":[{"scope":{},"logRecords":[{"timeUnixNano":"1660296023390371588","body":{"stringValue":"{\"message\":\"dummy\"}"},"traceId":"","spanId":""}]}]}]}' http://0.0.0.0:4318/v1/logs
137137
```
138+
139+
## OpenTelemetry trace improvements
140+
141+
Fluent Bit includes enhanced support for OpenTelemetry traces with improved JSON parsing, error handling, and validation capabilities.
142+
143+
### Unified trace JSON parser
144+
145+
Fluent Bit provides a unified interface for processing OpenTelemetry trace data in JSON format. The parser converts OpenTelemetry JSON trace payloads into the Fluent Bit internal trace representation, supporting the full OpenTelemetry trace specification including:
146+
147+
- Resource spans with attributes
148+
- Instrumentation scope information
149+
- Span data (names, IDs, timestamps, status)
150+
- Span events and links
151+
- Trace and span ID validation
152+
153+
The unified parser handles the OpenTelemetry JSON encoding format, which wraps attribute values in type-specific containers (for example, `stringValue`, `intValue`, `doubleValue`, `boolValue`).
154+
155+
### Error status propagation
156+
157+
The OpenTelemetry input plugin provides detailed error status information when processing trace data. If trace processing fails, the plugin returns specific error codes that help identify the issue:
158+
159+
- `FLB_OTEL_TRACES_ERR_INVALID_JSON` - Invalid JSON format
160+
- `FLB_OTEL_TRACES_ERR_INVALID_TRACE_ID` - Invalid trace ID format or length
161+
- `FLB_OTEL_TRACES_ERR_INVALID_SPAN_ID` - Invalid span ID format or length
162+
- `FLB_OTEL_TRACES_ERR_INVALID_PARENT_SPAN_ID` - Invalid parent span ID
163+
- `FLB_OTEL_TRACES_ERR_STATUS_FAILURE` - Invalid span status code
164+
- `FLB_OTEL_TRACES_ERR_INVALID_ATTRIBUTES` - Invalid attribute format
165+
- `FLB_OTEL_TRACES_ERR_INVALID_EVENT_ENTRY` - Invalid span event
166+
- `FLB_OTEL_TRACES_ERR_INVALID_LINK_ENTRY` - Invalid span link
167+
168+
Error messages are logged with the specific error status code to help diagnose issues with incoming trace data.
169+
170+
### Strict ID decoding
171+
172+
Fluent Bit enforces strict validation for trace and span IDs to ensure data integrity:
173+
174+
- **Trace IDs**: Must be exactly 32 hexadecimal characters (16 bytes)
175+
- **Span IDs**: Must be exactly 16 hexadecimal characters (8 bytes)
176+
- **Parent Span IDs**: Must be exactly 16 hexadecimal characters (8 bytes) when present
177+
178+
The validation process:
179+
1. Verifies the ID length matches the expected size
180+
2. Validates that all characters are valid hexadecimal digits (0-9, a-f, A-F)
181+
3. Decodes the hexadecimal string to binary format
182+
4. Rejects invalid IDs with appropriate error codes
183+
184+
Invalid IDs result in error status codes (`FLB_OTEL_TRACES_ERR_INVALID_TRACE_ID`, `FLB_OTEL_TRACES_ERR_INVALID_SPAN_ID`, and so on) and the trace data is rejected to prevent processing of corrupted or malformed trace information.
185+
186+
### Example: JSON trace payload
187+
188+
The following example shows a valid OpenTelemetry JSON trace payload that can be sent to the `/v1/traces` endpoint:
189+
190+
```json
191+
{
192+
"resourceSpans": [
193+
{
194+
"resource": {
195+
"attributes": [
196+
{
197+
"key": "service.name",
198+
"value": {
199+
"stringValue": "my-service"
200+
}
201+
}
202+
]
203+
},
204+
"scopeSpans": [
205+
{
206+
"scope": {
207+
"name": "my-instrumentation",
208+
"version": "1.0.0"
209+
},
210+
"spans": [
211+
{
212+
"traceId": "0123456789abcdef0123456789abcdef",
213+
"spanId": "0123456789abcdef",
214+
"name": "my-span",
215+
"kind": 1,
216+
"startTimeUnixNano": "1660296023390371588",
217+
"endTimeUnixNano": "1660296023391371588",
218+
"status": {
219+
"code": "OK"
220+
},
221+
"attributes": [
222+
{
223+
"key": "http.method",
224+
"value": {
225+
"stringValue": "GET"
226+
}
227+
}
228+
]
229+
}
230+
]
231+
}
232+
]
233+
}
234+
]
235+
}
236+
```
237+
238+
Trace IDs must be exactly 32 hex characters and span IDs must be exactly 16 hex characters. Invalid IDs will be rejected with appropriate error messages.

0 commit comments

Comments
 (0)