docs: Update HA tracker docs to use memberlist by default (#13002)

dimitarvdimitrov · web-flow · commit 7df8ab14e012 · 2025-10-18T19:21:25.000+02:00
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -156,6 +156,7 @@
 
 * [CHANGE] Remove references to queriers having a Prometheus HTTP API. Instead, the query-frontend is now required for a Prometheus HTTP API. #12949
 * [CHANGE] Helm: Remove GEM (Grafana Enterprise Metrics) references from Helm chart documentation. #13019 #13020 #13021
+* [CHANGE] Update HA tracker documentation to use memberlist as the default KV store instead of consul/etcd. Consul and etcd are now marked as deprecated for the HA tracker as of Mimir 3.0. #13002
 * [ENHANCEMENT] Add migration guide for HA tracker from Consul or etcd to memberlist. #13011
 * [ENHANCEMENT] Improve the MimirIngesterReachingSeriesLimit runbook. #12356
 * [ENHANCEMENT] Improve the description of how to limit the number of buckets in native histograms. #12797
diff --git a/docs/sources/helm-charts/mimir-distributed/run-production-environment-with-helm/_index.md b/docs/sources/helm-charts/mimir-distributed/run-production-environment-with-helm/_index.md
@@ -337,7 +337,7 @@ If you need redundancy on the write path before it reaches Mimir, then you
 can set up redundant instances of Prometheus or Grafana Alloy to
 write metrics to Mimir.
 
-For more information, see [Configure high-availability deduplication with Consul](configure-helm-ha-deduplication-consul/).
+For more information, refer to [Configure high-availability deduplication](configure-helm-ha-deduplication/).
 
 ## Deploy on OpenShift
 
diff --git a/docs/sources/helm-charts/mimir-distributed/run-production-environment-with-helm/configure-helm-ha-deduplication-consul/ha-tracker-status.png b/docs/sources/helm-charts/mimir-distributed/run-production-environment-with-helm/configure-helm-ha-deduplication-consul/ha-tracker-status.png
diff --git a/docs/sources/helm-charts/mimir-distributed/run-production-environment-with-helm/configure-helm-ha-deduplication/ha-tracker-status.png b/docs/sources/helm-charts/mimir-distributed/run-production-environment-with-helm/configure-helm-ha-deduplication/ha-tracker-status.png
diff --git a/docs/sources/helm-charts/mimir-distributed/run-production-environment-with-helm/configure-helm-ha-deduplication/index.md b/docs/sources/helm-charts/mimir-distributed/run-production-environment-with-helm/configure-helm-ha-deduplication/index.md
@@ -3,37 +3,38 @@ aliases:
   - ../configuring/configure-helm-ha-deduplication-consul/
 description:
   Learn how to configure the Grafana Mimir Helm chart to handle HA Prometheus
-  server deduplication with Consul.
-menuTitle: Configure high-availability deduplication with Consul
-title:
-  Configuring Grafana Mimir-Distributed Helm Chart for high-availability deduplication
-  with Consul
+  server deduplication.
+menuTitle: Configure high-availability deduplication
+title: Configure high-availability deduplication with the Mimir distributed Helm chart
 weight: 70
 ---
 
-# Configuring Grafana Mimir-Distributed Helm Chart for high-availability deduplication with Consul
+# Configure high-availability deduplication with the Mimir distributed Helm chart
 
-Grafana Mimir can deduplicate data from a high-availability (HA) Prometheus setup. You can configure
-the deduplication by using the Grafana Mimir helm chart deployment that uses external Consul. For more information, see [Configure high availability](https://grafana.com/docs/mimir/<MIMIR_VERSION>/configure/configure-high-availability-deduplication/).
+Grafana Mimir can deduplicate data from a high-availability (HA) Prometheus setup. Starting from Mimir 3.0, the HA tracker uses `memberlist` by default, which requires no external dependencies. For more information, see [Configure high availability](https://grafana.com/docs/mimir/<MIMIR_VERSION>/configure/configure-high-availability-deduplication/).
+
+{{< admonition type="note" >}}
+Prior to Mimir 3.0, the HA tracker required an external key-value store such as Consul or etcd. These backends are now deprecated. If you're currently using Consul or etcd for the HA tracker, refer to the migration guide.
+{{< /admonition >}}
 
 ## Before you begin
 
 You need to have a Grafana Mimir installed via the mimir-distributed Helm chart.
 
 For conceptual information about how Mimir deduplicates incoming HA samples, refer to [Configure high availability](https://grafana.com/docs/mimir/<MIMIR_VERSION>/configure/configure-high-availability-deduplication/).
 
-You also need to configure HA for Prometheus or Grafana Agent. Lastly, you need a key-value store such as Consul KV.
+You also need to configure HA for Prometheus or Grafana Alloy.
 
 {{< docs/shared source="alloy" lookup="agent-deprecation.md" version="next" >}}
 
-## Configure Prometheus or Grafana Agent to send HA external labels
+## Configure Prometheus or Grafana Alloy to send HA external labels
 
-Configure the Prometheus or Grafana Agent HA setup by setting the labels named `cluster` and `__replica__`,
+Configure the Prometheus or Grafana Alloy HA setup by setting the labels named `cluster` and `__replica__`,
 which are the default labels for a HA setup in Grafana Mimir. If you want to change the HA labels,
-make sure to change them in Mimir as well. This ensures that the configurations of Grafana Mimir, Prometheus, and Grafana Agent all match each other. Otherwise, HA deduplication will not work.
+make sure to change them in Mimir as well. This ensures that the configurations of Grafana Mimir, Prometheus, and Grafana Alloy all match each other. Otherwise, HA deduplication will not work.
 
-- The value of the `cluster` label must be same across replica that belong to the same cluster.
-- The value of the `__replica__` label must be unique across different replica within the same cluster.
+- The value of the `cluster` label must be same across replicas that belong to the same cluster.
+- The value of the `__replica__` label must be unique across replicas within the same cluster.
 
 ```yaml
 global:
@@ -42,15 +43,9 @@ global:
     cluster: my-prometheus
 ```
 
-Reload or restart Prometheus or Grafana Agent after you update the configuration.
-
-> **Note:** [Configure high availability](https://grafana.com/docs/mimir/<MIMIR_VERSION>/configure/configure-high-availability-deduplication/).
-> document contains the same information on Prometheus setup for HA dedup.
-
-## Install Consul using Helm
+Reload or restart Prometheus or Grafana Alloy after you update the configuration.
 
-To get and install Consul on Kubernetes, use the [Consul helm chart](https://github.com/hashicorp/consul-k8s/tree/main/charts/consul).
-Make a note of the Consul endpoint, because you will need it later to configure Mimir.
+> **Note:** [Configure high availability](https://grafana.com/docs/mimir/<MIMIR_VERSION>/configure/configure-high-availability-deduplication/) contains the same information on Prometheus setup for HA deduplication.
 
 ## Configure Mimir high availability
 
@@ -75,12 +70,14 @@ mimir:
       ha_tracker:
         enable_ha_tracker: true
         kvstore:
-          store: consul
-          consul:
-            host: <consul-endpoint> # example: http://consul.consul.svc.cluster.local:8500
+          store: memberlist
 ```
 
-2. Upgrade the Mimir's helm release using the following command:
+{{< admonition type="note" >}}
+If memberlist is already configured for other Mimir components (such as the hash ring), the HA tracker will automatically use that configuration. In most Helm deployments, memberlist is already configured, so no additional configuration is needed.
+{{< /admonition >}}
+
+2. Upgrade the Mimir Helm release using the following command:
 
 ```bash
  helm -n <mimir-namespace> upgrade mimir grafana/mimir-distributed -f custom.yaml
@@ -105,9 +102,7 @@ mimir:
       ha_tracker:
         enable_ha_tracker: true
         kvstore:
-          store: consul
-          consul:
-            host: <consul-endpoint> # example: http://consul.consul.svc.cluster.local:8500
+          store: memberlist
 runtimeConfig:
   overrides:
     <tenant-id>: # put real tenant ID here
@@ -116,8 +111,8 @@ runtimeConfig:
       ha_replica_label: __replica__
 ```
 
-The `mimir` configuration block is similar with Globally configure HA deduplication setup. The `runtimeConfig` block
-is the configuration for per tenant HA deduplication.
+The `mimir` configuration block is similar to the global HA deduplication configuration. The `runtimeConfig` block
+configures per-tenant HA deduplication.
 
 2. Upgrade the Mimir's helm release using the following command:
 
@@ -127,7 +122,7 @@ is the configuration for per tenant HA deduplication.
 
 ## Verifying deduplication
 
-After Consul, Prometheus and Mimir running we can verify deduplication in several way.
+After Prometheus and Mimir are running, you can verify deduplication in several ways.
 
 ### ha_tracker's page
 
diff --git a/docs/sources/helm-charts/mimir-distributed/run-production-environment-with-helm/configure-helm-ha-deduplication/verify-deduplication.png b/docs/sources/helm-charts/mimir-distributed/run-production-environment-with-helm/configure-helm-ha-deduplication/verify-deduplication.png
diff --git a/docs/sources/mimir/configure/about-versioning.md b/docs/sources/mimir/configure/about-versioning.md
@@ -107,8 +107,6 @@ The following features are currently experimental:
     - `-distributor.otel-created-timestamp-zero-ingestion-enabled`
   - Promote a certain set of OTel resource attributes to labels
     - `-distributor.otel-promote-resource-attributes`
-  - Add experimental `memberlist` key-value store for ha_tracker. Note that this feature is `experimental`, as the upper limits of propagation times have not yet been validated. Additionally, cleanup operations have not yet been implemented for the memberlist entries.
-    - `-distributor.ha-tracker.kvstore.store`
   - Allow keeping OpenTelemetry `service.instance.id`, `service.name` and `service.namespace` resource attributes in `target_info` on top of converting them to the `instance` and `job` labels.
     - `-distributor.otel-keep-identifying-resource-attributes`
   - Enable conversion of OTel explicit bucket histograms into native histograms with custom buckets.
@@ -312,5 +310,6 @@ The following features or configuration parameters are currently deprecated and
 - Rule group configuration file
   - `evaluation_delay` field: use `query_offset` instead
 - The `-store-gateway.sharding-ring.auto-forget-enabled` is deprecated and will be removed in a future release. Set the `-store-gateway.sharding-ring.auto-forget-unhealthy-periods` flag to 0 to disable the auto-forget feature. Deprecated since Mimir 2.17.
-- etcd is deprecated as an option for backend storage for the HA tracker since Mimir 2.17.
+- Consul and etcd are deprecated as backend storage options for the HA tracker as of Mimir 3.0.
+- Use `memberlist` instead. Refer to the migration guide for instructions on migrating from Consul or etcd to `memberlist` for the HA tracker.
 - The `-distributor.otel-start-time-quiet-zero` parameter no longer has any effect and will be removed in a future release. Deprecated since Mimir 2.18.
diff --git a/docs/sources/mimir/configure/configure-high-availability-deduplication.md b/docs/sources/mimir/configure/configure-high-availability-deduplication.md
@@ -106,16 +106,20 @@ Alternatively, you can enable the HA tracker only on a per-tenant basis, keeping
 #### Configure the HA tracker KV store
 
 The HA tracker requires a key-value (KV) store to coordinate which replica is currently elected.
-In [Grafana Mimir versions 2.17](https://github.com/grafana/mimir/releases/tag/mimir-2.17.0) and later, use `memberlist` as the KV store backend for the HA tracker. The `consul` and `etcd` backends are deprecated.
+Starting from Mimir 3.0, `memberlist` is the recommended and default KV store backend for the HA tracker.
+
+{{< admonition type="note" >}}
+The `consul` and `etcd` backends are deprecated as of Mimir 3.0. If you're currently using `consul` or `etcd` for the HA tracker, refer to the migration guide for instructions on migrating to `memberlist`.
+{{< /admonition >}}
 
 To migrate from Consul or etcd to memberlist without downtime, see [Migrate HA tracker from Consul or etcd to memberlist](migrate-ha-tracker-to-memberlist/).
 
 The following CLI flags (and their respective YAML configuration options) are available for configuring the HA tracker KV store:
 
-- `-distributor.ha-tracker.store`: The backend storage to use, which is `memberlist`. The `consul` and `etcd` backends are deprecated.
-- `-memberlist.*`: The memberlist client configuration. It's common and used by other Mimir components as well.
-- [deprecated]`-distributor.ha-tracker.consul.*`: The Consul client configuration. Only use this if you have defined `consul` as your backend storage.
-- [deprecated]`-distributor.ha-tracker.etcd.*`: The etcd client configuration. Only use this if you have defined `etcd` as your backend storage.
+- `-distributor.ha-tracker.store`: Backend storage to use (default: `memberlist`).
+- `-memberlist.*`: Memberlist client configuration. This is shared by multiple components.
+
+The memberlist configuration is typically shared across multiple Mimir components (distributors, ingesters, etc.), so if you already have memberlist configured for hash ring synchronization, no additional configuration is required for the HA tracker.
 
 #### Configure expected label names for each Prometheus cluster and replica
 
@@ -138,12 +142,22 @@ The following configuration example snippet enables the HA tracker for all tenan
 ```yaml
 limits:
   accept_ha_samples: true
+
 distributor:
   ha_tracker:
     enable_ha_tracker: true
     kvstore:
-      [store: <string> | default = "consul"]
-      [consul | etcd: <config>]
+      store: memberlist
+
+memberlist:
+  # Memberlist configuration (typically shared with other components)
+  join_members:
+    - <IP_OR_DNS:PORT>
+    - <IP_OR_DNS:PORT>
 ```
 
+{{< admonition type="note" >}}
+If memberlist is already configured for other Mimir components, such as the hash ring, the HA tracker automatically uses that configuration. In most deployments, you don't need any additional memberlist configuration.
+{{< /admonition >}}
+
 For more information, see [distributor](../configuration-parameters/#distributor). The HA tracker flags are prefixed with `-distributor.ha-tracker.*`.
diff --git a/docs/sources/mimir/manage/mimir-runbooks/_index.md b/docs/sources/mimir/manage/mimir-runbooks/_index.md
@@ -1244,14 +1244,14 @@ When using Memberlist as KV store for hash rings, all read and update operations
 How it **works**:
 
 - Consul is typically used to store the hash ring state.
-- Etcd is typically used to store by the HA tracker (distributor) to deduplicate samples.
+- Memberlist (default since Mimir 3.0) or etcd/Consul (deprecated) are used by the HA tracker (distributor) to deduplicate samples.
 - If an instance is failing operations on the **hash ring**, either the instance can't update the heartbeat in the ring or is failing to receive ring updates.
 - If an instance is failing operations on the **HA tracker** backend, either the instance can't update the authoritative replica or is failing to receive updates.
 
 How to **investigate**:
 
-- Ensure Consul/Etcd is up and running.
-- Investigate the logs of the affected instance to find the specific error occurring when talking to Consul/Etcd.
+- Ensure the KV store backend (Consul, etcd, or memberlist) is up and running.
+- Investigate the logs of the affected instance to find the specific error occurring when talking to the KV store backend.
 
 ### MimirReachingTCPConnectionsLimit
 
diff --git a/docs/sources/mimir/manage/secure/securing-communications-with-tls.md b/docs/sources/mimir/manage/secure/securing-communications-with-tls.md
@@ -143,8 +143,8 @@ The following Grafana Mimir components support TLS for inter-communication, whic
 - Distributor gRPC client used to forward series matching a configured set to a dedicated remote endpoint: `-distributor.forwarding.grpc-client.*`
 - Alertmanager gRPC client used to connect to other Alertmanager instances: `-alertmanager.alertmanager-client.*`
 - gRPC client used by distributors, queriers, and rulers to connect to ingesters: `-ingester.client.*`
-- etcd client used by all Mimir components to connect to etcd, which is required only if you're running the hash ring or HA tracker on the etcd backend: `-<prefix>.etcd.*`
-- Memberlist client used by all Mimir components to gossip the hash ring, which is required only if you're running the hash ring on memberlist: `-memberlist.`
+- etcd client used by all Mimir components to connect to etcd (deprecated for HA tracker as of Mimir 3.0; use memberlist instead): `-<prefix>.etcd.*`
+- Memberlist client used by all Mimir components to gossip the hash ring and HA tracker: `-memberlist.`
 - Memcached client used by all Mimir components: `-blocks-storage.bucket-store.chunks-cache.memcached.*`, `-blocks-storage.bucket-store.index-cache.memcached.*`, `-blocks-storage.bucket-store.metadata-cache.memcached.*`, `-query-frontend.results-cache.memcached.*`, `-ruler-storage.cache.memcached.*`
 
 Each of the components listed above support the following TLS configuration options, which are shown with their corresponding flag suffixes:
diff --git a/docs/sources/mimir/set-up/jsonnet/deploy.md b/docs/sources/mimir/set-up/jsonnet/deploy.md
@@ -105,3 +105,7 @@ You can use [Tanka](https://tanka.dev/) and [jsonnet-bundler](https://github.com
    The generated Kubernetes manifests create resources in the `default` namespace.
    To use a different namespace, change the `namespace` configuration option in the `environments/default/main.jsonnet` file, and re-generate the Kubernetes manifests.
    {{< /admonition >}}
+
+   {{< admonition type="note" >}}
+   Starting from Mimir 3.0, a Jsonnet-based installation uses `memberlist` for the HA tracker by default. No external dependencies are required. If you're upgrading from a previous version that used etcd or Consul for the HA tracker, refer to the migration guide.
+   {{< /admonition >}}
