Skip to content

Add ability to configure sampling rate based on transaction span name #1531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion docs/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -677,6 +677,18 @@ Limits the amount of spans that are recorded per transaction. This is helpful in
By default, the agent will sample every transaction (e.g. request to your service). To reduce overhead and storage requirements, you can set the sample rate to a value between `0.0` and `1.0`. We still record overall time and the result for unsampled transactions, but no context information, tags, or spans. The sample rate will be rounded to 4 digits of precision.


### `transaction_sample_rate_by_name` [config-transaction-sample-rate-by-name]

| | | | |
| --- | --- | --- | --- |
| Environment | `Config` key | Default | Example |
| `ELASTIC_APM_TRANSACTION_SAMPLE_RATE_BY_NAME` | `transaction_sample_rate_by_name` | `{}` | `{"UsersController#index" => 1.0, "HealthController#ping" => 0.0}` |

Configure specific sampling rates for transactions with particular names. This option takes a hash where the keys are transaction names and the values are sampling rates between `0.0` and `1.0`.

This is useful when you want to ensure critical transactions are always sampled while reducing sampling for high-volume, less important transactions. When a transaction name matches an entry in this hash, the specified sampling rate takes precedence over the global `transaction_sample_rate`.


### `verify_server_cert` [config-verify-server-cert]

| | | |
Expand Down Expand Up @@ -731,4 +743,3 @@ Elastic APM patches `Kernel#require` to auto-detect and instrument supported thi
To get around this patch, set the environment variable `ELASTIC_APM_SKIP_REQUIRE_PATCH` to `"1"`.

The agent might need some additional tweaking to make sure the third-party libraries are picked up and instrumented. Make sure you require the agent *after* you require your other dependencies.

2 changes: 2 additions & 0 deletions lib/elastic_apm/config.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
require 'elastic_apm/config/log_level_map'
require 'elastic_apm/config/options'
require 'elastic_apm/config/round_float'
require 'elastic_apm/config/round_float_hash_value'
require 'elastic_apm/config/regexp_list'
require 'elastic_apm/config/wildcard_pattern_list'
require 'elastic_apm/deprecations'
Expand Down Expand Up @@ -98,6 +99,7 @@ class Config
option :transaction_ignore_urls, type: :list, default: [], converter: WildcardPatternList.new
option :transaction_max_spans, type: :int, default: 500
option :transaction_sample_rate, type: :float, default: 1.0, converter: RoundFloat.new
option :transaction_sample_rate_by_name, type: :hash, default: {}, converter: RoundFloatHashValue.new
option :use_elastic_traceparent_header, type: :bool, default: true
option :verify_server_cert, type: :bool, default: true

Expand Down
35 changes: 35 additions & 0 deletions lib/elastic_apm/config/round_float_hash_value.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Licensed to Elasticsearch B.V. under one or more contributor
# license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright
# ownership. Elasticsearch B.V. licenses this file to you under
# the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# frozen_string_literal: true

module ElasticAPM
class Config
# @api private
class RoundFloatHashValue
def initialize
@float_converter = RoundFloat.new
end

def call(hash)
return {} unless hash

hash.transform_values { |value| @float_converter.call(value) }
end
end
end
end
26 changes: 23 additions & 3 deletions lib/elastic_apm/instrumenter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ def start_transaction(
sampled = trace_context.recorded?
sample_rate = trace_context.tracestate.sample_rate
else
sampled = random_sample?(config)
sampled = random_sample?(config.transaction_sample_rate)
sample_rate = sampled ? config.transaction_sample_rate : 0
end

Expand Down Expand Up @@ -195,6 +195,19 @@ def start_span(
current_transaction
end
return unless transaction

unless trace_context
span_sample_rate = transaction_sample_rate_for_name(name, transaction.config)
# if the span sample rate is different from the transaction sample rate,
# we need to check if the span should be sampled
# and update the transaction's sample rate accordingly
if transaction.started_spans == 0 && span_sample_rate && span_sample_rate != transaction.sample_rate
span_sampled = random_sample?(span_sample_rate)
transaction.sampled = span_sampled
transaction.sample_rate = span_sampled ? span_sample_rate : 0
end
end

return unless transaction.sampled?
return unless transaction.inc_started_spans!

Expand Down Expand Up @@ -278,10 +291,17 @@ def inspect
'>'
end

def transaction_sample_rate_for_name(name, config)
return if !name || config.transaction_sample_rate_by_name.empty?
return unless config.transaction_sample_rate_by_name.key?(name)

config.transaction_sample_rate_by_name[name]
end

private

def random_sample?(config)
rand <= config.transaction_sample_rate
def random_sample?(sample_rate)
rand <= sample_rate
end

def update_transaction_metrics(transaction)
Expand Down
15 changes: 11 additions & 4 deletions lib/elastic_apm/transaction.rb
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,15 @@ def initialize(
end
# rubocop:enable Metrics/ParameterLists

attr_accessor :name, :type, :result, :outcome
attr_accessor(
:name,
:type,
:result,
:outcome,
:sampled,
:sample_rate,
:started_spans
)

attr_reader(
:breakdown_metrics,
Expand All @@ -98,12 +106,11 @@ def initialize(
:framework_name,
:notifications,
:self_time,
:sample_rate,
:span_frames_min_duration,
:started_spans,
:timestamp,
:trace_context,
:transaction_max_spans
:transaction_max_spans,
:config
)

alias :collect_metrics? :collect_metrics
Expand Down
99 changes: 99 additions & 0 deletions spec/elastic_apm/instrumenter_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -469,5 +469,104 @@ module ElasticAPM
end
end
end

describe '#transaction_sample_rate_for_name' do
let(:default_rate) { 0.5 }
let(:special_rate) { 1.0 }

context 'with nil name' do
it 'returns nil' do
config = Config.new(transaction_sample_rate: default_rate)
expect(subject.transaction_sample_rate_for_name(nil, config)).to be_nil
end
end

context 'with empty transaction_sample_rate_by_name' do
it 'returns nil' do
config = Config.new(transaction_sample_rate_by_name: {})
expect(subject.transaction_sample_rate_for_name('Something', config)).to be_nil
end
end

context 'with matching name in transaction_sample_rate_by_name' do
it 'returns matching sample rate' do
config = Config.new(
transaction_sample_rate: default_rate,
transaction_sample_rate_by_name: { 'Something' => special_rate }
)
expect(subject.transaction_sample_rate_for_name('Something', config)).to eq(special_rate)
end
end

context 'with non-matching name in transaction_sample_rate_by_name' do
it 'returns default sample rate' do
config = Config.new(
transaction_sample_rate_by_name: { 'SomethingElse' => special_rate }
)
expect(subject.transaction_sample_rate_for_name('Something', config)).to be_nil
end
end
end

describe 'span-based sampling' do
context 'when starting a span with a name that has a different sampling rate' do
it 'can change the sampling decision of the transaction' do
config = Config.new(
transaction_sample_rate: 0.0,
transaction_sample_rate_by_name: { 'ImportantOperation' => 1.0 }
)

# Force predictable random sampling
allow(subject).to receive(:rand).and_return(0.5)

transaction = subject.start_transaction('Test', config: config)
expect(transaction).not_to be_sampled

# Verify sampling is updated when first span is created
allow(subject).to receive(:random_sample?).and_return(true)
span = subject.start_span('ImportantOperation')

# The span should exist (since the transaction is now sampled)
expect(span).not_to be_nil
expect(transaction).to be_sampled
end

it 'only considers the first span for changing the sampling decision' do
config = Config.new(
transaction_sample_rate: 0.0,
transaction_sample_rate_by_name: {
'FirstSpan' => 1.0,
'SecondSpan' => 0.0,
'ThirdSpan' => 0.5
}
)

# Force predictable random sampling
allow(subject).to receive(:rand).and_return(0.5)

transaction = subject.start_transaction('Test', config: config)
expect(transaction).not_to be_sampled

# First span changes sampling because it's the first span
first_span = subject.start_span('FirstSpan')
expect(first_span).not_to be_nil
expect(transaction).to be_sampled
expect(transaction.sample_rate).to eq(1.0)

# Initial sampling rate stored
original_sample_rate = transaction.sample_rate

# Second span should not change sampling even though it has a different rate
second_span = subject.start_span('SecondSpan')
expect(second_span).not_to be_nil
expect(transaction.sample_rate).to eq(original_sample_rate)

# Third span should not change sampling either
third_span = subject.start_span('ThirdSpan')
expect(third_span).not_to be_nil
expect(transaction.sample_rate).to eq(original_sample_rate)
end
end
end
end
end