Skip to content

Replace the internal reactivelimiter with a wrapped Failsafe-go limiter #12773

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Oct 2, 2025
Merged

Replace the internal reactivelimiter with a wrapped Failsafe-go limiter #12773

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
26e6869
Replace the internal reactivelimiter with a wrapped Failsafe-go limit…
jhalterman Sep 23, 2025
f80a37c
Remove unneeded utils
jhalterman Sep 23, 2025
1c1122c
Update changelog
jhalterman Sep 25, 2025
37de024
Improve docs a bit
jhalterman Sep 25, 2025
9a14140
Update about versioning
jhalterman Oct 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
* [CHANGE] Cost attribution: Reduce the default maximum per-user cardinality of cost attribution labels to 2000. #12625
* [CHANGE] Querier, query-frontend: Add `_total` suffix to `cortex_mimir_query_engine_common_subexpression_elimination_duplication_nodes_introduced`, `cortex_mimir_query_engine_common_subexpression_elimination_selectors_eliminated` and `cortex_mimir_query_engine_common_subexpression_elimination_selectors_inspected` metric names. #12636
* [CHANGE] Alertmanager: Change the severity for InitialSyncFailed from 'critical' to 'warning'. #12824
* [CHANGE] Ingester: Renamed experimental reactive limiter options. #12773
* [FEATURE] Ingester: Expose the number of active series ingested via OTLP as `cortex_ingester_active_otlp_series`. #12678
* [FEATURE] Distributor, ruler: Add experimental `-validation.name-validation-scheme` flag to specify the validation scheme for metric and label names. #12215
* [FEATURE] Ruler: Add support to use a Prometheus-compatible HTTP endpoint for remote rule evaluation. See [remote evaluation mode](https://grafana.com/docs/mimir/latest/operators-guide/architecture/components/ruler/#remote-over-http-https) for more details. This feature can be used to federate data from multiple Mimir instances. #11415
Expand Down
176 changes: 88 additions & 88 deletions cmd/mimir/config-descriptor.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4388,7 +4388,7 @@
"kind": "field",
"name": "enabled",
"required": false,
"desc": "Enable reactive limiting when making requests to ingesters",
"desc": "Enable reactive limiting when making requests to a service",
"fieldValue": null,
"fieldDefaultValue": false,
"fieldFlag": "ingester.push-reactive-limiter.enabled",
Expand All @@ -4397,101 +4397,101 @@
},
{
"kind": "field",
"name": "short_window_min_duration",
"name": "min_limit",
"required": false,
"desc": "Minimum duration of the window that is used to determine the recent, short-term load on the system",
"desc": "Minimum inflight requests limit",
"fieldValue": null,
"fieldDefaultValue": 1000000000,
"fieldFlag": "ingester.push-reactive-limiter.short-window-min-duration",
"fieldType": "duration",
"fieldDefaultValue": 2,
"fieldFlag": "ingester.push-reactive-limiter.min-limit",
"fieldType": "int",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "short_window_max_duration",
"name": "max_limit",
"required": false,
"desc": "Maximum duration of the window that is used to determine the recent, short-term load on the system",
"desc": "Maximum inflight requests limit",
"fieldValue": null,
"fieldDefaultValue": 30000000000,
"fieldFlag": "ingester.push-reactive-limiter.short-window-max-duration",
"fieldType": "duration",
"fieldDefaultValue": 200,
"fieldFlag": "ingester.push-reactive-limiter.max-limit",
"fieldType": "int",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "short_window_min_samples",
"name": "initial_limit",
"required": false,
"desc": "Minimum number of samples that must be recorded in the window",
"desc": "Initial inflight requests limit",
"fieldValue": null,
"fieldDefaultValue": 50,
"fieldFlag": "ingester.push-reactive-limiter.short-window-min-samples",
"fieldDefaultValue": 20,
"fieldFlag": "ingester.push-reactive-limiter.initial-limit",
"fieldType": "int",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "long_window",
"name": "max_limit_factor",
"required": false,
"desc": "Short-term window measurements that are stored in an exponentially weighted moving average window, representing the long-term baseline inflight time",
"desc": "The maximum inflight limit as a multiple of current inflight requests",
"fieldValue": null,
"fieldDefaultValue": 60,
"fieldFlag": "ingester.push-reactive-limiter.long-window",
"fieldType": "int",
"fieldDefaultValue": 5,
"fieldFlag": "ingester.push-reactive-limiter.max-limit-factor",
"fieldType": "float",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "sample_quantile",
"name": "recent_window_min_duration",
"required": false,
"desc": "The quantile of recorded response times to consider when adjusting the concurrency limit",
"desc": "Minimum duration of the window that is used to collect recent response time samples",
"fieldValue": null,
"fieldDefaultValue": 0.9,
"fieldFlag": "ingester.push-reactive-limiter.sample-quantile",
"fieldType": "float",
"fieldDefaultValue": 1000000000,
"fieldFlag": "ingester.push-reactive-limiter.recent-window-min-duration",
"fieldType": "duration",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "min_inflight_limit",
"name": "recent_window_max_duration",
"required": false,
"desc": "Minimum inflight requests limit",
"desc": "Maximum duration of the window that is used to collect recent response time samples",
"fieldValue": null,
"fieldDefaultValue": 2,
"fieldFlag": "ingester.push-reactive-limiter.min-inflight-limit",
"fieldType": "int",
"fieldDefaultValue": 30000000000,
"fieldFlag": "ingester.push-reactive-limiter.recent-window-max-duration",
"fieldType": "duration",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "max_inflight_limit",
"name": "recent_window_min_samples",
"required": false,
"desc": "Maximum inflight requests limit",
"desc": "Minimum number of samples that must be recorded in the recent window before updating the limit",
"fieldValue": null,
"fieldDefaultValue": 200,
"fieldFlag": "ingester.push-reactive-limiter.max-inflight-limit",
"fieldDefaultValue": 50,
"fieldFlag": "ingester.push-reactive-limiter.recent-window-min-samples",
"fieldType": "int",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "initial_inflight_limit",
"name": "recent_quantile",
"required": false,
"desc": "Initial inflight requests limit",
"desc": "The quantile of recent recorded response times to consider when adjusting the concurrency limit",
"fieldValue": null,
"fieldDefaultValue": 20,
"fieldFlag": "ingester.push-reactive-limiter.initial-inflight-limit",
"fieldType": "int",
"fieldDefaultValue": 0.9,
"fieldFlag": "ingester.push-reactive-limiter.recent-quantile",
"fieldType": "float",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "max_limit_factor",
"name": "baseline_window_age",
"required": false,
"desc": "The maximum limit as a multiple of current inflight requests",
"desc": "The average age of baseline samples aggregated recent samples are added to",
"fieldValue": null,
"fieldDefaultValue": 5,
"fieldFlag": "ingester.push-reactive-limiter.max-limit-factor",
"fieldType": "float",
"fieldDefaultValue": 10,
"fieldFlag": "ingester.push-reactive-limiter.baseline-window-age",
"fieldType": "int",
"fieldCategory": "experimental"
},
{
Expand Down Expand Up @@ -4541,7 +4541,7 @@
"kind": "field",
"name": "enabled",
"required": false,
"desc": "Enable reactive limiting when making requests to ingesters",
"desc": "Enable reactive limiting when making requests to a service",
"fieldValue": null,
"fieldDefaultValue": false,
"fieldFlag": "ingester.read-reactive-limiter.enabled",
Expand All @@ -4550,101 +4550,101 @@
},
{
"kind": "field",
"name": "short_window_min_duration",
"name": "min_limit",
"required": false,
"desc": "Minimum duration of the window that is used to determine the recent, short-term load on the system",
"desc": "Minimum inflight requests limit",
"fieldValue": null,
"fieldDefaultValue": 1000000000,
"fieldFlag": "ingester.read-reactive-limiter.short-window-min-duration",
"fieldType": "duration",
"fieldDefaultValue": 2,
"fieldFlag": "ingester.read-reactive-limiter.min-limit",
"fieldType": "int",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "short_window_max_duration",
"name": "max_limit",
"required": false,
"desc": "Maximum duration of the window that is used to determine the recent, short-term load on the system",
"desc": "Maximum inflight requests limit",
"fieldValue": null,
"fieldDefaultValue": 30000000000,
"fieldFlag": "ingester.read-reactive-limiter.short-window-max-duration",
"fieldType": "duration",
"fieldDefaultValue": 200,
"fieldFlag": "ingester.read-reactive-limiter.max-limit",
"fieldType": "int",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "short_window_min_samples",
"name": "initial_limit",
"required": false,
"desc": "Minimum number of samples that must be recorded in the window",
"desc": "Initial inflight requests limit",
"fieldValue": null,
"fieldDefaultValue": 50,
"fieldFlag": "ingester.read-reactive-limiter.short-window-min-samples",
"fieldDefaultValue": 20,
"fieldFlag": "ingester.read-reactive-limiter.initial-limit",
"fieldType": "int",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "long_window",
"name": "max_limit_factor",
"required": false,
"desc": "Short-term window measurements that are stored in an exponentially weighted moving average window, representing the long-term baseline inflight time",
"desc": "The maximum inflight limit as a multiple of current inflight requests",
"fieldValue": null,
"fieldDefaultValue": 60,
"fieldFlag": "ingester.read-reactive-limiter.long-window",
"fieldType": "int",
"fieldDefaultValue": 5,
"fieldFlag": "ingester.read-reactive-limiter.max-limit-factor",
"fieldType": "float",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "sample_quantile",
"name": "recent_window_min_duration",
"required": false,
"desc": "The quantile of recorded response times to consider when adjusting the concurrency limit",
"desc": "Minimum duration of the window that is used to collect recent response time samples",
"fieldValue": null,
"fieldDefaultValue": 0.9,
"fieldFlag": "ingester.read-reactive-limiter.sample-quantile",
"fieldType": "float",
"fieldDefaultValue": 1000000000,
"fieldFlag": "ingester.read-reactive-limiter.recent-window-min-duration",
"fieldType": "duration",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "min_inflight_limit",
"name": "recent_window_max_duration",
"required": false,
"desc": "Minimum inflight requests limit",
"desc": "Maximum duration of the window that is used to collect recent response time samples",
"fieldValue": null,
"fieldDefaultValue": 2,
"fieldFlag": "ingester.read-reactive-limiter.min-inflight-limit",
"fieldType": "int",
"fieldDefaultValue": 30000000000,
"fieldFlag": "ingester.read-reactive-limiter.recent-window-max-duration",
"fieldType": "duration",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "max_inflight_limit",
"name": "recent_window_min_samples",
"required": false,
"desc": "Maximum inflight requests limit",
"desc": "Minimum number of samples that must be recorded in the recent window before updating the limit",
"fieldValue": null,
"fieldDefaultValue": 200,
"fieldFlag": "ingester.read-reactive-limiter.max-inflight-limit",
"fieldDefaultValue": 50,
"fieldFlag": "ingester.read-reactive-limiter.recent-window-min-samples",
"fieldType": "int",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "initial_inflight_limit",
"name": "recent_quantile",
"required": false,
"desc": "Initial inflight requests limit",
"desc": "The quantile of recent recorded response times to consider when adjusting the concurrency limit",
"fieldValue": null,
"fieldDefaultValue": 20,
"fieldFlag": "ingester.read-reactive-limiter.initial-inflight-limit",
"fieldType": "int",
"fieldDefaultValue": 0.9,
"fieldFlag": "ingester.read-reactive-limiter.recent-quantile",
"fieldType": "float",
"fieldCategory": "experimental"
},
{
"kind": "field",
"name": "max_limit_factor",
"name": "baseline_window_age",
"required": false,
"desc": "The maximum limit as a multiple of current inflight requests",
"desc": "The average age of baseline samples aggregated recent samples are added to",
"fieldValue": null,
"fieldDefaultValue": 5,
"fieldFlag": "ingester.read-reactive-limiter.max-limit-factor",
"fieldType": "float",
"fieldDefaultValue": 10,
"fieldFlag": "ingester.read-reactive-limiter.baseline-window-age",
"fieldType": "int",
"fieldCategory": "experimental"
},
{
Expand Down
Loading
Loading