Skip to content

Commit 6fe9064

Browse files
mjsaxmjsax
committed
MINOR: improve KS docs about recommended configs (#18516)
Reviewers: Bill Bejeck <[email protected]>
1 parent 8035097 commit 6fe9064

File tree

1 file changed

+38
-25
lines changed

1 file changed

+38
-25
lines changed

‎docs/streams/developer-guide/config-streams.html

+38-25
Original file line numberDiff line numberDiff line change
@@ -63,8 +63,10 @@
6363
</li>
6464
<li><a class="reference internal" href="#recommended-configuration-parameters-for-resiliency" id="id21">Recommended configuration parameters for resiliency</a>
6565
<ul>
66-
<li><a class="reference internal" href="#acks" id="id22">acks</a></li>
67-
<li><a class="reference internal" href="#id2" id="id23">replication.factor</a></li>
66+
<li><a class="reference internal" href="#acks-short" id="id22">acks</a></li>
67+
<li><a class="reference internal" href="#replication-factor-short" id="id23">replication.factor</a></li>
68+
<li><a class="reference internal" href="#min-isr-short" id="id44">min.insync.replicas</a></li>
69+
<li><a class="reference internal" href="#num-standby-replicas-short" id="id45">num.standby.replicas</a></li>
6870
</ul>
6971
</li>
7072
<li><a class="reference internal" href="#optional-configuration-parameters" id="id6">Optional configuration parameters</a>
@@ -155,7 +157,7 @@ <h4><a class="toc-backref" href="#id4">application.id</a><a class="headerlink" h
155157
<div class="section" id="bootstrap-servers">
156158
<h4><a class="toc-backref" href="#id5">bootstrap.servers</a><a class="headerlink" href="#bootstrap-servers" title="Permalink to this headline"></a></h4>
157159
<blockquote>
158-
<div><p>(Required) The Kafka bootstrap servers. This is the same <a class="reference external" href="http://kafka.apache.org/documentation.html#producerconfigs">setting</a> that is used by the underlying producer and consumer clients to connect to the Kafka cluster.
160+
<div><p>(Required) The Kafka bootstrap servers. This is the same <a class="reference external" href="/{{version}}/documentation.html#producerconfigs">setting</a> that is used by the underlying producer and consumer clients to connect to the Kafka cluster.
159161
Example: <code class="docutils literal"><span class="pre">&quot;kafka-broker1:9092,kafka-broker2:9092&quot;</span></code>.</p>
160162
</div></blockquote>
161163
</div>
@@ -172,59 +174,70 @@ <h3><a class="toc-backref" href="#id21">Recommended configuration parameters for
172174
</tr>
173175
</thead>
174176
<tbody valign="top">
175-
<tr class="row-odd"><td>acks</td>
176-
<td>Producer</td>
177-
<td><code class="docutils literal"><span class="pre">acks=1</span></code></td>
178-
<td><code class="docutils literal"><span class="pre">acks=all</span></code></td>
177+
<tr class="row-odd">
178+
<td>acks</td>
179+
<td>Producer (for version &lt;=2.8)</td>
180+
<td><code class="docutils literal"><span class="pre">acks="1")</span></code></td>
181+
<td><code class="docutils literal"><span class="pre">acks="all"</span></code></td>
179182
</tr>
180-
<tr class="row-even"><td>replication.factor (for broker version 2.3 or older)/td>
183+
<tr class="row-even">
184+
<td>replication.factor (for broker version 2.3 or older)/td>
181185
<td>Streams</td>
182186
<td><code class="docutils literal"><span class="pre">-1</span></code></td>
183-
<td><code class="docutils literal"><span class="pre">3</span></code></td>
187+
<td><code class="docutils literal"><span class="pre">3</span></code> (broker 2.4+: ensure broker config <code>default.replication.factor=3</code>)</td>
184188
</tr>
185-
<tr class="row-odd"><td>min.insync.replicas</td>
189+
<tr class="row-odd">
190+
<td>min.insync.replicas</td>
186191
<td>Broker</td>
187192
<td><code class="docutils literal"><span class="pre">1</span></code></td>
188193
<td><code class="docutils literal"><span class="pre">2</span></code></td>
189194
</tr>
190-
<tr class="row-even"><td>num.standby.replicas</td>
195+
<tr class="row-even">
196+
<td>num.standby.replicas</td>
191197
<td>Streams</td>
192198
<td><code class="docutils literal"><span class="pre">0</span></code></td>
193199
<td><code class="docutils literal"><span class="pre">1</span></code></td>
194200
</tr>
195201
</tbody>
196202
</table>
197-
<p>Increasing the replication factor to 3 ensures that the internal Kafka Streams topic can tolerate up to 2 broker failures. Changing the acks setting to &#8220;all&#8221;
198-
guarantees that a record will not be lost as long as one replica is alive. The tradeoff from moving to the default values to the recommended ones is
203+
<p>Increasing the replication factor to 3 ensures that the internal Kafka Streams topic can tolerate up to 2 broker failures.
204+
The tradeoff from moving to the default values to the recommended ones is
199205
that some performance and more storage space (3x with the replication factor of 3) are sacrificed for more resiliency.</p>
200-
<div class="section" id="acks">
206+
<div class="section" id="acks-short">
201207
<h4><a class="toc-backref" href="#id22">acks</a><a class="headerlink" href="#acks" title="Permalink to this headline"></a></h4>
202208
<blockquote>
203209
<div><p>The number of acknowledgments that the leader must have received before considering a request complete. This controls
204210
the durability of records that are sent. The possible values are:</p>
205211
<ul class="simple">
206-
<li><code class="docutils literal"><span class="pre">acks=0</span></code> The producer does not wait for acknowledgment from the server and the record is immediately added to the socket buffer and considered sent. No guarantee can be made that the server has received the record in this case, and the producer won&#8217;t generally know of any failures. The offset returned for each record will always be set to <code class="docutils literal"><span class="pre">-1</span></code>.</li>
207-
<li><code class="docutils literal"><span class="pre">acks=1</span></code> The leader writes the record to its local log and responds without waiting for full acknowledgement from all followers. If the leader immediately fails after acknowledging the record, but before the followers have replicated it, then the record will be lost.</li>
208-
<li><code class="docutils literal"><span class="pre">acks=all</span></code> The leader waits for the full set of in-sync replicas to acknowledge the record. This guarantees that the record will not be lost if there is at least one in-sync replica alive. This is the strongest available guarantee.</li>
212+
<li><code class="docutils literal"><span class="pre">acks="0"</span></code> The producer does not wait for acknowledgment from the server and the record is immediately added to the socket buffer and considered sent. No guarantee can be made that the server has received the record in this case, and the producer won&#8217;t generally know of any failures. The offset returned for each record will always be set to <code class="docutils literal"><span class="pre">-1</span></code>.</li>
213+
<li><code class="docutils literal"><span class="pre">acks="1"</span></code> The leader writes the record to its local log and responds without waiting for full acknowledgement from all followers. If the leader immediately fails after acknowledging the record, but before the followers have replicated it, then the record will be lost.</li>
214+
<li><code class="docutils literal"><span class="pre">acks="all"</span></code> (default since 3.0 release) The leader waits for the full set of in-sync replicas to acknowledge the record. This guarantees that the record will not be lost if there is at least one in-sync replica alive. This is the strongest available guarantee.</li>
209215
</ul>
210216
<p>For more information, see the <a class="reference external" href="https://kafka.apache.org/documentation/#producerconfigs">Kafka Producer documentation</a>.</p>
211217
</div></blockquote>
212218
</div>
213-
<div class="section" id="id2">
214-
<h4><a class="toc-backref" href="#id23">replication.factor</a><a class="headerlink" href="#id2" title="Permalink to this headline"></a></h4>
219+
<div class="section" id="replication-factor-short">
220+
<h4><a class="toc-backref" href="#id23">replication.factor</a><a class="headerlink" href="#id23" title="Permalink to this headline"></a></h4>
215221
<blockquote>
216222
<div>See the <a class="reference internal" href="#replication-factor-parm"><span class="std std-ref">description here</span></a>.</div></blockquote>
217223
</div>
218-
<div class="section" id="i32">
219-
<h4><a class="toc-backref" href="#id23">num.standby.replicas</a><a class="headerlink" href="#id2" title="Permalink to this headline"></a></h4>
224+
<div class="section" id="min-isr-short">
225+
<h4><a class="toc-backref" href="#id44">min.insync.replicas</a><a class="headerlink" href="#id44" title="Permalink to this headline"></a></h4>
226+
<p>The minimum number of in-sync replicas available for replication if the producer is configured with <code>acks="all"</code>
227+
(see <a href="/{{version}}/documentation/#topicconfigs_min.insync.replicas">topic configs</a>).
228+
</p>
229+
</div>
230+
<div class="section" id="num-standby-replicas-short">
231+
<h4><a class="toc-backref" href="#id45">num.standby.replicas</a><a class="headerlink" href="#id45" title="Permalink to this headline"></a></h4>
220232
<blockquote>
221233
<div>See the <a class="reference internal" href="#streams-developer-guide-standby-replicas"><span class="std std-ref">description here</span></a>.</div></blockquote>
222234
</div>
223235
<pre class="line-numbers"><code class="language-java">Properties streamsSettings = new Properties();
224236
// for broker version 2.3 or older
225237
//streamsSettings.put(StreamsConfig.REPLICATION_FACTOR_CONFIG, 3);
238+
// for version 2.8 or older
239+
//streamsSettings.put(StreamsConfig.producerPrefix(ProducerConfig.ACKS_CONFIG), "all");
226240
streamsSettings.put(StreamsConfig.topicPrefix(TopicConfig.MIN_IN_SYNC_REPLICAS_CONFIG), 2);
227-
streamsSettings.put(StreamsConfig.producerPrefix(ProducerConfig.ACKS_CONFIG), "all");
228241
streamsSettings.put(StreamsConfig.NUM_STANDBY_REPLICAS_CONFIG, 1);</code></pre>
229242

230243
</div>
@@ -920,7 +933,7 @@ <h4><a class="toc-backref" href="#id40">log.summary.interval.ms</a><a class="hea
920933
for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
921934
</p>
922935
<p>
923-
Note that one warmup replica corresponds to one <a href="https://kafka.apache.org/34/documentation/streams/architecture#streams_architecture_tasks">Stream Task</a>. Furthermore, note that each warmup task can only be promoted to an active task during
936+
Note that one warmup replica corresponds to one <a href="/{{version}}/documentation/streams/architecture#streams_architecture_tasks">Stream Task</a>. Furthermore, note that each warmup task can only be promoted to an active task during
924937
a rebalance (normally during a so-called probing rebalance, which occur at a frequency specified by the
925938
<code class="docutils literal"><span class="pre">probing.rebalance.interval.ms</span></code> config). This means that the
926939
maximum rate at which active tasks can be migrated from one Kafka Streams instance to another instance can be determined by
@@ -1250,8 +1263,8 @@ <h4><a class="toc-backref" href="#id17">Naming</a><a class="headerlink" href="#n
12501263
<div class="section" id="default-values">
12511264
<h4><a class="toc-backref" href="#id18">Default Values</a><a class="headerlink" href="#default-values" title="Permalink to this headline"></a></h4>
12521265
<p>Kafka Streams uses different default values for some of the underlying client configs, which are summarized below. For detailed descriptions
1253-
of these configs, see <a class="reference external" href="https://kafka.apache.org/documentation.html#producerconfigs">Producer Configs</a>
1254-
and <a class="reference external" href="https://kafka.apache.org/documentation.html#consumerconfigs">Consumer Configs</a>.</p>
1266+
of these configs, see <a class="reference external" href="/{{version}}/documentation.html#producerconfigs">Producer Configs</a>
1267+
and <a class="reference external" href="/{{version}}/documentation.html#consumerconfigs">Consumer Configs</a>.</p>
12551268
<table border="1" class="non-scrolling-table docutils">
12561269
<thead valign="bottom">
12571270
<tr class="row-odd"><th class="head">Parameter Name</th>

0 commit comments

Comments
 (0)