Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 77 additions & 3 deletions docs/benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,83 @@ We run two types of automated benchmark tests for Phase 2:
Benchmarks](https://open-telemetry.github.io/otel-arrow/benchmarks/nightly/)**
- Comprehensive test suites run nightly

Both provide performance metrics for the OTAP dataflow engine, for various
scenarios.
<!-- TODO: Add details on the scenario -->
Both provide performance metrics for the OTAP dataflow engine for various
scenarios. Unless otherwise specified, all tests run on a single CPU core.

#### 1. Filter + OTel Collector Comparison

**URL:** <https://open-telemetry.github.io/otel-arrow/benchmarks/nightly/filter/>

Tests a filter scenario where a filter processor drops 95% of logs. Processes
approximately 100k logs/sec input with ~5k logs/sec output. The benchmark page
includes a direct comparison with the equivalent OTel Collector performing the
same filtering operation.

#### 2. Backpressure

**URL:** <https://open-telemetry.github.io/otel-arrow/benchmarks/nightly/backpressure/>

Measures backpressure impact with `wait_for_result` set to true on the dataflow
engine receivers. Processes approximately 100k logs/sec input and output. The
pipeline includes an attribute processor configured to rename an attribute,
which forces the dataflow engine to perform in-memory representation and
conversion rather than operating in pass-through mode.

#### 3. Syslog

**URL:** <https://open-telemetry.github.io/otel-arrow/benchmarks/nightly/syslog/>

Tests syslog ingestion via UDP with two variations:

- Basic syslog message format
- CEF (Common Event Format) formatted messages

Processes approximately 5k logs/sec input and output.

#### 4. Normal Load

**URL:** <https://open-telemetry.github.io/otel-arrow/benchmarks/continuous/>

Standard load test processing 100k records/sec input and output on a single CPU
core. This test runs with each commit to main.

#### 5. Saturation and Scaling

**URL:** <https://open-telemetry.github.io/otel-arrow/benchmarks/continuous-saturation/>

Tests performance at saturation across different CPU configurations: 1, 2, 4, 8,
and 16 cores.

*TODO: Update test output to include scalability ratios in addition to raw
throughput numbers.*

#### 6. Idle State

**URL:** <https://open-telemetry.github.io/otel-arrow/benchmarks/continuous-idle-state/>

Measures resource consumption in idle state with two variations:

- Single core configuration
- All-cores configuration (128 cores)

#### 7. Binary Size

**URL:** <https://open-telemetry.github.io/otel-arrow/benchmarks/binary-size/>

Tracks the binary size of the dataflow engine for Linux ARM64 and AMD64
architectures over time.

### Metrics Collected

All benchmark tests measure the following metrics:

- **Logs/sec input** - Input throughput
- **Logs/sec output** - Output throughput
- **RAM** - Average and maximum memory usage
- **Normalized CPU** - Average and maximum CPU usage, normalized to 0-100% where
100% represents full utilization of all available cores. For example, in a
4-core test, 80% means 3.2 cores are being used (0.8 X 4 cores)
- **Network bytes/sec** - Input and output network bandwidth

### Learn More About Phase 2

Expand Down
Loading