| 
 | 1 | +---  | 
 | 2 | +aliases:  | 
 | 3 | +  - ../../operators-guide/deploy-grafana-mimir/jsonnet/configure-ingest-storage/  | 
 | 4 | +description: Learn how to configure Grafana Mimir ingest storage architecture when using Jsonnet.  | 
 | 5 | +menuTitle: Configure ingest storage architecture  | 
 | 6 | +title: Configure ingest storage architecture  | 
 | 7 | +weight: 20  | 
 | 8 | +---  | 
 | 9 | + | 
 | 10 | +# Configure ingest storage architecture  | 
 | 11 | + | 
 | 12 | +The [ingest storage](https://grafana.com/docs/mimir/<MIMIR_VERSION>/get-started/about-grafana-mimir-architecture/) is the next generation architecture of Grafana Mimir. It decouples the Mimir read and write paths using Apache Kafka or a Kafka-compatible backend.  | 
 | 13 | + | 
 | 14 | +To enable ingest storage, set the following Jsonnet:  | 
 | 15 | + | 
 | 16 | +```jsonnet  | 
 | 17 | +{  | 
 | 18 | +  _config+:: {  | 
 | 19 | +    ingest_storage_enabled: true,  | 
 | 20 | +  }  | 
 | 21 | +}  | 
 | 22 | +```  | 
 | 23 | + | 
 | 24 | +{{< admonition type="note" >}}  | 
 | 25 | +The ingest storage architecture requires a production-grade Apache Kafka cluster or Kafka-compatible backend.  | 
 | 26 | +{{< /admonition >}}  | 
 | 27 | + | 
 | 28 | +{{< admonition type="warning" >}}  | 
 | 29 | +The ingest storage architecture requires the ruler remote evaluation to be enabled for the Mimir instance.  | 
 | 30 | +Refer to the [Configure the Grafana Mimir ruler with Jsonnet](https://grafana.com/docs/mimir/<MIMIR_VERSION>/set-up/jsonnet/configure-ruler/) documentation for details about the ruler's operational modes.  | 
 | 31 | +{{< /admonition >}}  | 
 | 32 | + | 
 | 33 | +## Kafka connection options  | 
 | 34 | + | 
 | 35 | +The jsonnet configures some Kafka connection parameters by default:  | 
 | 36 | + | 
 | 37 | +- **Kafka topic**: `ingest`  | 
 | 38 | +- **Auto-created partitions**: 1000 (when topic doesn't exist)  | 
 | 39 | +- **Kafka addresses**: `kafka.<namespace>.svc.<cluster_domain>:9092`  | 
 | 40 | + | 
 | 41 | +To customize the Kafka connection configuration, override the default settings:  | 
 | 42 | + | 
 | 43 | +```jsonnet  | 
 | 44 | +{  | 
 | 45 | +  _config+:: {  | 
 | 46 | +    ingest_storage_enabled: true,  | 
 | 47 | +  },  | 
 | 48 | +
  | 
 | 49 | +  // Override producer and consumer addresses for external Kafka  | 
 | 50 | +  ingest_storage_kafka_producer_address:: 'kafka-broker-1:9092,kafka-broker-2:9092',  | 
 | 51 | +  ingest_storage_kafka_consumer_address:: 'kafka-broker-1:9092,kafka-broker-2:9092',  | 
 | 52 | +
  | 
 | 53 | +  // Override Kafka client configuration  | 
 | 54 | +  ingest_storage_kafka_client_args+:: {  | 
 | 55 | +    'ingest-storage.kafka.topic': 'mimir-ingest',  | 
 | 56 | +    'ingest-storage.kafka.auto-create-topic-default-partitions': 500,  | 
 | 57 | +  },  | 
 | 58 | +}  | 
 | 59 | +```  | 
 | 60 | + | 
 | 61 | +### Kafka partition sizing  | 
 | 62 | + | 
 | 63 | +The configured topic must have at least as many partitions as the number of ingesters in one zone.  | 
 | 64 | + | 
 | 65 | +The [Configure the Grafana Mimir Kafka backend](https://grafana.com/docs/mimir/<MIMIR_VERSION>/configure/configure-kafka-backend/) documentation provides more details about Kafka configurations.  | 
 | 66 | + | 
 | 67 | +## Migration considerations  | 
 | 68 | + | 
 | 69 | +{{< admonition type="warning" >}}  | 
 | 70 | +Migration of a live Grafana Mimir cluster from the classic architecture to the ingest storage is a complex multi-step process, that requires careful planning and execution.  | 
 | 71 | + | 
 | 72 | +For use-cases that allow some downtime, it's simpler to deploy a new Grafana Mimir cluster, that uses ingest storage, and migrate the data from the old cluster to the new one.  | 
 | 73 | +{{< /admonition >}}  | 
0 commit comments