From 64d4e128c9ab8a537371c2f41cb473e83787ebc9 Mon Sep 17 00:00:00 2001 From: Shwetha Rao Date: Fri, 9 May 2025 15:40:00 +0530 Subject: [PATCH 1/3] first commit --- modules/learn/pages/clusters-and-availability/rebalance.adoc | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/modules/learn/pages/clusters-and-availability/rebalance.adoc b/modules/learn/pages/clusters-and-availability/rebalance.adoc index 55894d78ea..6594dd7978 100644 --- a/modules/learn/pages/clusters-and-availability/rebalance.adoc +++ b/modules/learn/pages/clusters-and-availability/rebalance.adoc @@ -228,6 +228,10 @@ This setting has no effect if you have enabled file-based index rebalancing. Users with Full Admin or Cluster Admin roles can change the batch size using the REST API. See xref:rest-api:rest-modify-index-batch-size.adoc[Modify Index Batch Size]. +[#smart-batching] +==== Smart Batching + + [#rebalancing-the-search-service] === Search Service From 3c1cefbd5c10c9b845203507974d1f15c3774cc8 Mon Sep 17 00:00:00 2001 From: Shwetha Rao Date: Fri, 9 May 2025 16:35:30 +0530 Subject: [PATCH 2/3] smart-batching --- .../clusters-and-availability/rebalance.adoc | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/modules/learn/pages/clusters-and-availability/rebalance.adoc b/modules/learn/pages/clusters-and-availability/rebalance.adoc index 6594dd7978..ad6117ece0 100644 --- a/modules/learn/pages/clusters-and-availability/rebalance.adoc +++ b/modules/learn/pages/clusters-and-availability/rebalance.adoc @@ -231,6 +231,22 @@ See xref:rest-api:rest-modify-index-batch-size.adoc[Modify Index Batch Size]. [#smart-batching] ==== Smart Batching +During a rebalance process, indexes are relocated and rebuilt in batches. +The default batch size is 3 to set a limit on the amount of work attempted concurrently. + +In Couchbase Server 7.1 and later versions, you can use smart batching with one or more of the following purposes: + +* Reduces further time and resources required to move the index metadata. +* Rebuilds indexes at their new locations during rebalance. +* Increases rebalance performance by increasing pipeline parallelism by starting the next batch when the previos batch is only partially completed. +* Allows the increase of overall concurrency by using administrator-determined modifications of the batch size. +* Optimizes the sharing of data streams where index relocation takes place. +* Repairs, prior to index relocation, index replicas lost due to node failure. +* When appropriate, reassigns scheduled index-rebuilding from one batch to the next, to balance the workloads. + +If at least one node in the cluster is running a Server version 7.1+ then most of the smart-batching features apply to the cluster where some of the nodes are running an earlier version. + +You can change the default batch size of 3 by the Full Admin and Cluster Admin roles using the REST API. For information, see xref:rest-api:rest-modify-index-batch-size.adoc[Modify Index Batch Size]. [#rebalancing-the-search-service] === Search Service From 1756d916097050698e729afd1d6712ddf96a60e5 Mon Sep 17 00:00:00 2001 From: Shwetha Rao Date: Fri, 9 May 2025 17:03:36 +0530 Subject: [PATCH 3/3] added-file-based-rebalance-note --- .../pages/clusters-and-availability/failover.adoc | 6 +++--- .../pages/clusters-and-availability/rebalance.adoc | 14 +++++++++----- .../pages/manage-settings/general-settings.adoc | 5 ++++- modules/rest-api/pages/post-settings-indexes.adoc | 2 ++ .../pages/rest-modify-index-batch-size.adoc | 2 ++ 5 files changed, 20 insertions(+), 9 deletions(-) diff --git a/modules/learn/pages/clusters-and-availability/failover.adoc b/modules/learn/pages/clusters-and-availability/failover.adoc index 3d9d014e92..ed721adde0 100644 --- a/modules/learn/pages/clusters-and-availability/failover.adoc +++ b/modules/learn/pages/clusters-and-availability/failover.adoc @@ -27,9 +27,9 @@ Graceful failover _must_ be manually initiated. Hard failover _can_ be manually Hard failover can also be initiated automatically by Couchbase Server: this is known as _automatic_ failover. The Cluster Manager detects the unavailability of a node, and duly initiates a _hard_ failover, without administrator intervention. - -Note that when a node is failed over (as opposed to _removed_), some replica vBuckets are lost from the surviving nodes; since some are promoted to _active_ status, and are not replaced with new replica-copies. -By contrast, xref:learn:clusters-and-availability/removal.adoc[removal] creates new copies of those replica vBuckets that would otherwise be lost. This maintains the cluster's previous level of data-availability; but results in greater competition for memory resources, across the surviving nodes. +NOTE: When a node is failed over (as opposed to _removed_), some replica vBuckets are lost from the surviving nodes; since some are promoted to _active_ status, and are not replaced with new replica-copies. +By contrast, xref:learn:clusters-and-availability/removal.adoc[removal] creates new copies of those replica vBuckets that would otherwise be lost. +This maintains the cluster's previous level of data-availability; but results in greater competition for memory resources, across the surviving nodes. Ideally, after any failover, xref:learn:clusters-and-availability/rebalance.adoc[rebalance] should be performed. This is especially important when a Data Service node has been failed over, since the rebalance will ensure an optimal ratio of active to replica vBuckets across all the remaining Data Service nodes. diff --git a/modules/learn/pages/clusters-and-availability/rebalance.adoc b/modules/learn/pages/clusters-and-availability/rebalance.adoc index ad6117ece0..96699e6ad1 100644 --- a/modules/learn/pages/clusters-and-availability/rebalance.adoc +++ b/modules/learn/pages/clusters-and-availability/rebalance.adoc @@ -145,6 +145,8 @@ The Index Service maintains a cluster-wide set of index definitions and metadata The rebalance process takes into account the nodes' CPU, RAM, and disk bandwidth to limit its effect on database performance. +NOTE: Shard Based Rebalance and Rebalance Based on File Transfer are synonyms for File-based Rebalance. + [#index-rebalance-methods] ==== Index Rebalance Methods @@ -231,22 +233,24 @@ See xref:rest-api:rest-modify-index-batch-size.adoc[Modify Index Batch Size]. [#smart-batching] ==== Smart Batching -During a rebalance process, indexes are relocated and rebuilt in batches. -The default batch size is 3 to set a limit on the amount of work attempted concurrently. +During a Rebalance operation, Couchbase Server breaks the rebuilding process into batches to limit the performance impacts. + +The default batch size is set to 3 applying a limit on the amount of work attempted concurrently. In Couchbase Server 7.1 and later versions, you can use smart batching with one or more of the following purposes: * Reduces further time and resources required to move the index metadata. * Rebuilds indexes at their new locations during rebalance. -* Increases rebalance performance by increasing pipeline parallelism by starting the next batch when the previos batch is only partially completed. +* Increases rebalance performance by increasing pipeline parallelism by starting the next batch when the previous batch is only partially completed. * Allows the increase of overall concurrency by using administrator-determined modifications of the batch size. * Optimizes the sharing of data streams where index relocation takes place. * Repairs, prior to index relocation, index replicas lost due to node failure. * When appropriate, reassigns scheduled index-rebuilding from one batch to the next, to balance the workloads. -If at least one node in the cluster is running a Server version 7.1+ then most of the smart-batching features apply to the cluster where some of the nodes are running an earlier version. +If at least one node in the cluster is running a Server version 7.1+ then most of the smart batching features apply to the cluster where some of the nodes are running an earlier version. -You can change the default batch size of 3 by the Full Admin and Cluster Admin roles using the REST API. For information, see xref:rest-api:rest-modify-index-batch-size.adoc[Modify Index Batch Size]. +You can change the default batch size of 3 by the Full Admin and Cluster Admin roles using the REST API. +For information, see xref:rest-api:rest-modify-index-batch-size.adoc[Modify Index Batch Size]. [#rebalancing-the-search-service] === Search Service diff --git a/modules/manage/pages/manage-settings/general-settings.adoc b/modules/manage/pages/manage-settings/general-settings.adoc index 0c534e5a32..abe4e5a316 100644 --- a/modules/manage/pages/manage-settings/general-settings.adoc +++ b/modules/manage/pages/manage-settings/general-settings.adoc @@ -313,12 +313,15 @@ During an upgrade, mixed mode clusters with nodes that support bloom filters wil [#file-based-index-rebalance] * *File Transfer Based Rebalance*: -Controls whether Couchbase Server rebuilds indexes or copies them between nodes. +Controls whether Couchbase Server rebuilds indexes or copies them between nodes. +Use the **Enable File Transfer Based Rebalance** checkbox to enable or disable the File Transfer Based Rebalance from the UI. The default cleared setting has an Index Server node rebuild any newly assigned index during a rebalance. You cannot enable file-based rebalance when you have enabled Memory Optimized Index Storage. When you select this option, Couchbase Server copies the index files from one Index Server node to another during a rebalance instead of rebuilding them. See xref:learn:clusters-and-availability/rebalance.adoc#index-rebalance-methods[Index Rebalance Methods]. + +NOTE: Shard Based Rebalance and Rebalance Based on File Transfer are synonyms for File-based Rebalance. ++ You can disable this feature from the UI or via REST API. To learn about disabling this feature via REST API, see xref:rest-api:post-settings-indexes.adoc#disable-file-transfer-based-rebalance[Curl Command to Disable the File Transfer Based Rebalance]. This feature is disabled by default. + WARNING: Disabling this feature slows down the Rebalance operation. diff --git a/modules/rest-api/pages/post-settings-indexes.adoc b/modules/rest-api/pages/post-settings-indexes.adoc index bb6e2baf85..88a826b79d 100644 --- a/modules/rest-api/pages/post-settings-indexes.adoc +++ b/modules/rest-api/pages/post-settings-indexes.adoc @@ -161,6 +161,8 @@ include::example$post-settings-indexes.jsonc[] The following command disables the File Transfer Based Rebalance (`enableShardAffinity`) feature in the xref:manage:manage-settings/general-settings.adoc#index-storage-mode[Index Storage Mode]. +NOTE: Shard Based Rebalance and Rebalance Based on File Transfer are synonyms for File-based Rebalance. + [source#example-curl,bash] ---- include::example$post-settings-indexes.sh[tag=disable-ftb-rebalance] diff --git a/modules/rest-api/pages/rest-modify-index-batch-size.adoc b/modules/rest-api/pages/rest-modify-index-batch-size.adoc index 2f39e7943b..6091e644d4 100644 --- a/modules/rest-api/pages/rest-modify-index-batch-size.adoc +++ b/modules/rest-api/pages/rest-modify-index-batch-size.adoc @@ -5,6 +5,8 @@ {description} +NOTE: Shard Based Rebalance and Rebalance Based on File Transfer are synonyms for File-based Rebalance. + == HTTP Method and URI ----