diff --git a/docs/operator-guides/collecting-metrics.md b/docs/operator-guides/collecting-metrics.md index b2ce15d837ce9..89ab5974ac0d5 100644 --- a/docs/operator-guides/collecting-metrics.md +++ b/docs/operator-guides/collecting-metrics.md @@ -16,28 +16,146 @@ Airbyte uses Datadog to monitor Airbyte Cloud performance on a [number of metric  - +1. Deploy an OpenTelemetry collector if you don't already have one. See the [OpenTelemetry documentation](https://opentelemetry.io/docs/collector/getting-started/#kubernetes) for help doing this. If you use Datadog as your monitoring tool, they have an excellent guide to [set up a collector and exporter](https://docs.datadoghq.com/opentelemetry/collector_exporter/). + +2. Update your `values.yaml` file to enable OpenTelemetry. + + ```yaml + global: + edition: enterprise # This is an enterprise-only feature + metrics: + enabled: true + otlp: + enabled: true + collectorEndpoint: "YOUR_ENDPOINT" # The OTel collector endpoint Airbyte sends metrics to. You configure this endpoint outside of Airbyte as part of your OTel deployment. + ``` + +3. Redeploy Airbyte with the updated values. + +Airbyte sends metrics to the collector you specified in your configuration. + +### Available metrics + +The following metrics are available. They're published every minute. + +
Metric | +Tag | +Example Value | +
---|---|---|
airbyte.syncs |
+ connection_id |
+ 653a067e-cd0b-4cab-96b5-5e5cb03f159b | +
workspace_id |
+ bed3b473-1518-4461-a37f-730ea3d3a848 | +|
job_id |
+ 23642492 | +|
status |
+ success, failed | +|
attempt_count |
+ 3 | +|
version |
+ 1.5.0 | +|
source_connector_id |
+ 82c7fb2d-7de1-4d4e-b12e-510b0d61e374 | +|
destination_connector_id |
+ 3cb42982-755b-4644-9ed4-19651b53ebdd | +|
airbyte.gb_moved |
+ connection_id |
+ 653a067e-cd0b-4cab-96b5-5e5cb03f159b | +
workspace_id |
+ bed3b473-1518-4461-a37f-730ea3d3a848 | +|
job_id |
+ 23642492 | +|
source_connector_id |
+ 82c7fb2d-7de1-4d4e-b12e-510b0d61e374 | +|
destination_connector_id |
+ 3cb42982-755b-4644-9ed4-19651b53ebdd | +|
version |
+ 1.5.0 | +|
airbyte.sync_duration |
+ connection_id |
+ 653a067e-cd0b-4cab-96b5-5e5cb03f159b | +
workspace_id |
+ bed3b473-1518-4461-a37f-730ea3d3a848 | +|
job_id |
+ 23642492 | +|
source_connector_id |
+ 82c7fb2d-7de1-4d4e-b12e-510b0d61e374 | +|
destination_connector_id |
+ 3cb42982-755b-4644-9ed4-19651b53ebdd | +|
version |
+ 1.5.0 | +|
airbyte.api_requests |
+ workspace_id |
+ bed3b473-1518-4461-a37f-730ea3d3a848 | +
endpoint |
+ /v1/connections/sync | +|
status |
+ 200 | +