diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 8133ad9abe..9cc098dd8a 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -82,6 +82,10 @@ include::third-party:partial$nav.adoc[] **** xref:learn:clusters-and-availability/xdcr-filtering.adoc[XDCR Advanced Filtering] **** xref:learn:clusters-and-availability/xdcr-conflict-resolution.adoc[XDCR Conflict Resolution] **** xref:learn:clusters-and-availability/xdcr-with-scopes-and-collections.adoc[XDCR with Scopes and Collections] + **** xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning] + **** xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging] + ***** xref:learn:clusters-and-availability/xdcr-viewing-conflict-logs.adoc[Viewing Conflict Logs] + **** xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway] *** xref:learn:clusters-and-availability/groups.adoc[Server Group Awareness] * xref:learn:security/security-overview.adoc[Security] ** xref:learn:security/authentication.adoc[Authentication] diff --git a/modules/introduction/partials/new-features-80.adoc b/modules/introduction/partials/new-features-80.adoc index d875f9a19f..f4bb1ded02 100644 --- a/modules/introduction/partials/new-features-80.adoc +++ b/modules/introduction/partials/new-features-80.adoc @@ -113,6 +113,11 @@ Additional information sent by clients at connection time can be found in the lo https://jira.issues.couchbase.com/browse/MB-61048[MB-61048]:: Once faulty remote cluster credentials are fixed, XDCR will now be able to more quickly restart replications that depend on the repaired references. +https://jira.issues.couchbase.com/browse/MB-58989[MB-58989]:: +XDCR detects and logs concurrent conflicts that occur in different clusters but on the same document version due to independent modifications by locally connected applications during Active-Active replication. +The conflict logs are for your information only. +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging Feature]. + [#section-new-feature-800-query-service] === Query Service diff --git a/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-after-upgrade.png b/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-after-upgrade.png new file mode 100644 index 0000000000..ccb0c7d8b7 Binary files /dev/null and b/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-after-upgrade.png differ diff --git a/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-before-upgrade.png b/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-before-upgrade.png new file mode 100644 index 0000000000..9f49db8e86 Binary files /dev/null and b/modules/learn/assets/images/clusters-and-availability/xdcr-active-sgw-before-upgrade.png differ diff --git a/modules/learn/pages/clusters-and-availability/conflictlogging-examples.adoc b/modules/learn/pages/clusters-and-availability/conflictlogging-examples.adoc new file mode 100644 index 0000000000..c070848d8a --- /dev/null +++ b/modules/learn/pages/clusters-and-availability/conflictlogging-examples.adoc @@ -0,0 +1,147 @@ +[#conflictlogging-key] +==== conflictLogging key + +`conflictLogging` is the replication setting key. +The types of values for `conflictLogging` are as follows: + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#enabled-for-all-conflicts[Enabled for all conflicts] + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#enabled-with-special-logging-rules[Enabled with special logging rules] + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#disabled-with-stored-logging-rules[Disabled with stored logging rules] + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#disabled-conflict-logging[Disabled] + +NOTE: The conflict logs are not replicated because XDCR adds a hidden system xattribute flag called `_xdcr_conflict` into each of the conflict logs (xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Records]) stored in conflict collection. +This attribute prevents conflict logs from replicating to the target. + +`conflictLogging` details, value format, and examples are as follows. + +[#enabled-for-all-conflicts] +===== Enabled for all conflicts + +To log all conflicts to the same collection for the entire replication: + +. Enable conflict logging by setting `disabled` to `false`. + +. Designate a default conflict collection by specifying the conflict log bucket and the conflict log collection in `bucket` and `collection`. + +NOTE: You can designate any regular collection as the conflict collection, including collections that are a part of the replication. + +*Format:* +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": false, + "bucket": [bucketName], + "collection": [scope].[collection], + }' +---- + +*Example:* Conflict logging is enabled, and the default conflict collection is `conflictlogsbucket.myappscope.log1collection`. +All conflicts in the replication will be logged to the default conflict collection. + +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": false, + "bucket": "conflictlogsbucket", + "collection": "myappscope.log1collection" + }' +---- + +[#enabled-with-special-logging-rules] +===== Enabled with special logging rules + +To specify different conflict collection rules for different scopes and collections in your replication, define `loggingRules` in the `conflictLogging` settings. + +You can use `loggingRules` to specify, for any scope or collection in the replication, whether the conflicts from that scope or collection must be logged to a different conflict collection or not. +If you do not specify in the logging rules, then by default, the conflicts from all collections are logged to the default conflict collection. + +*Format:* +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": false, + "bucket": [bucketName], + "collection": [scope].[collection], + "loggingRules": {[logging_rules]}, + }' +---- + +*Example:* Conflict logging is enabled. The default conflict collection is the same as in the previous example, however, `loggingRules` is added. +The `conflictLogging` setting defines the following rules: + +* Conflict logs for the collection `agents.partners` must be logged to a different collection called `conflictlogsbucket2.myappscope.partnerlogs` + +* Conflicts must not be logged in the collection `agents.retailers`. + +* Conflict logs for the `inventory` scope must be logged to a collection called `conflictlogsbucket2.myappscope.inventorylogs`. + +[source,json] +---- +--data-urlencode conflictLogging= '{ +"disabled": false, +"bucket": "conflictlogsbucket1", +"collection": "myappscope.log1collection" +"loggingRules": { + "agents.partners": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.partnerlogs" + }, + "agents.retailers": null, + "inventory": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.inventorylogs" + } + } +}' +---- + +[#disabled-with-stored-logging-rules] +===== Disabled with stored logging rules + +You can configure `conflictLogging` (specify the default conflict collection and the `loggingRules`) but not enable `conflictLogging` by setting `disabled` to `true`. + +NOTE: You can update the replication and set the `conflictLogging` status `disabled` to `true` or `false` at any time. + +When you set `"disabled": true`, it also disables conflict logging, but all of the rules are retained for the collection. + +*Format:* + +[source,json] +---- +--data-urlencode conflictLogging= '{ + "bucket": [bucketName], + "collection": [scope].[collection], + "loggingRules": {[logging_rules]}, + "disabled": true, +}' +---- + +*Example:* Conflict logging is disabled, but the default conflict collection and the logging rules are retained. + +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": true, + "bucket": "conflictlogsbucket1", + "collection": "myappscope.log1collection", + "loggingRules": { + "agents.partners": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.partnerlogs" + }, + "agents.retailers": null, + "inventory": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.inventorylogs" + } + } +}' +---- + +[#disabled-conflict-logging] +===== Disabled + +To disable conflict logging, set `--data-urlencode conflictLogging='{}'`. diff --git a/modules/learn/pages/clusters-and-availability/old-xdcr-conflict-logging-feature.adoc b/modules/learn/pages/clusters-and-availability/old-xdcr-conflict-logging-feature.adoc new file mode 100644 index 0000000000..131474a80f --- /dev/null +++ b/modules/learn/pages/clusters-and-availability/old-xdcr-conflict-logging-feature.adoc @@ -0,0 +1,1073 @@ += XDCR Conflict Logging +:description: pass:q[XDCR detects and logs concurrent conflicts that occur in different clusters but on the same document version due to independent modifications by locally connected applications during Active-Active replication. You can manually review and resolve these conflicts to prevent data loss.] + +[abstract] +{description} + +During XDCR Active-Active replication, a conflict can occur if different applications connected locally to two different clusters, independently modify the same version of the document at the same time. +These simultaneous modifications happen within the convergence window*, so the changes made in one document are not reflected in the history of the other. + +XDCR detects and logs these mutations, which you can manually review, upsert, and continue with the resolution process. + +_*A convergence window is a period during which multiple actors, such as users, applications, or clusters, can modify the same document in an active-active topology, before the changes have fully converged across all clusters._ + +== Adaptibility of XDCR Conflict Logging + +* Conflicts are expected to be rare during XDCR replication. Due to any reason, if there is an upsurge of conflicts, XDCR skips logging conflicts and prioritises replication. + +* Conflict logging is asynchronous and attempted on the best-effort basis, which makes it the lowest operational priority. +If any error occurs, such as low memory quota or a lag in storing conflict logs, XDCR may pause or stop the logging process, reports the error type for the pipeline, and continues with replication so there is no High Availability issue. + +* xref:learn:clusters-and-availability/xdcr-conflict-resolution.adoc[Conflict Resolution] process remains unchanged. + +[#overview-of-xdcr-conflict-logging] +== Overview of the XDCR Conflict Logging Process + +A conflict logging process has the following steps: + +. xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#xdcr-conflict-logging-prerequisites[Prerequisites]: Enable conflict logging by configuring and deploying all replication clusters on Server 8.0.0 or a later version, xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#create-conflict-collection[creating a conflict collection], and configuring xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.#replication-settings-for-conflict-logging[replication settings]. +Also, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#deploy-xdcr-conflict-logging[Deploying XDCR Active-Active Conflict Logging Setup]. + +. xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#xdcr-conflict-detection-logging[Conflict Detection and Logging]: Start the XDCR conflict resolution process. +If the same version of the document is modified in the source and target clusters by locally connected different applications at around the same time, XDCR detects these conflicts using Hybrid Logical Vectors (HLVs). +XDCR logs the conflict details such as document ID, document contents, and conflicting document histories, into a collection designated for conflict logging. + +. xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-access-and-review[Conflict Access and Review]: You can access and review the logged conflicts. +Then manually resolve the conflicts, upsert the documents, and contiue the xref:learn:clusters-and-availability/xdcr-conflict-resolution.adoc[replication process] without data loss. + +You can configure an XDCR Active-Active setup for logging conflicts through one of the following methods: + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#greenfield-deployment-conflict-logging[Greenfield Deployment]: Set up a new XDCR active-active setup compatible for conflict logging. + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#upgrade-xdcr-setup-conflict-logging[Upgrading an Existing XDCR Active-Passive Setup]: Configure an existing XDCR active-passive setup into an XDCR active-active setup compatible for conflict logging. + +[#conflict-collection] +== Conflict Collection + +Conflict collection is a regular collection created by you, which stores the conflict log messages, the metadata of conflicting documents, and the conflicting document bodies from the affected clusters of the XDCR topology. + +* A conflict collection is a regular collection, so you can use regular RBAC to control standard access and programmatic access to the documents in the collection. + +* Supports enabling auditing for investigations if a collection or scope associated with the conflict collection is dropped. +For information about enabling auditing and enabling event-auditing, see xref:manage:manage-security/manage-auditing.adoc[Manage Auditing]. +For more information about Event Auditing, see xref:audit-event-reference/audit-event-reference.adoc[Audit Event Reference]. + +* You can store conflict collection in the source bucket or in a separate collection in a bucket that's not used in any XDCR replications. + +* You can use a conflict collection for a single or multiple XDCR replications while the document keys are unique. + +* XDCR can write into conflict collections associated with the source cluster, however, XDCR cannot modify the conflict collections in any target clusters. + +[IMPORTANT] +==== +* Use the conflict collection only for storing the conflict data to prevent applications from writing into this collection, and causing data inconsistency and feature issues. + +* External documents in the collection can interfere with event retrieval because of the dependency on xref:8.0@server:howtos/kv-operations.adoc#scoped-kv-operations[KV Range Scan]. +==== + +[#create-conflict-collection] +=== Create Conflict Collection + +When xref:manage:manage-buckets/create-bucket.adoc[creating a bucket] and a collection (see and xref:manage:manage-scopes-and-collections/manage-scopes-and-collections.adoc[Manage Scopes and Collections]), designate them as conflict bucket and conflict collection for logging conflicts by using: + +** xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging +[Replication Settings for XDCR Conflict Logging], to set from UI. + +** xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication], to set through REST API. + +NOTE: To configure the conflict collection, use the same collection namespace specified in the replication setting, xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#use-conflictlogging-and-loggingrules[Use conflictLogging Key and loggingRules Value in XDCR]. + +[#replication-settings-for-conflict-logging] +== Replication Settings for Conflict Logging + +To enable XDCR conflict logging, you must configure the replication settings for logging conflicts. +The replication settings are as follows: + +[#enable-hlv-conflict-logging] +=== Enable Hybrid Logical Vectors (HLVs) + +To enable XDCR conflict logging, as a prerequisite, Hybrid Logical Vectors (HLVs) must be enabled on all buckets of all clusters participating in the replication topology. + +XDCR uses HLV metadata to detect conflicts from different clusters and stores the document ID, document contents, and conflicting document histories into a pre-defined xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#create-conflict-collection[Conflict Collection] created by you. + +To enable the bucket property `enableCrossClusterVersioning`: + +* Using REST API, see xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing]. + +* From the UI, see xref:manage:manage-buckets/edit-bucket.adoc#edit-bucket-for-eccv[Edit a Bucket to Enable Cross Cluster Versioning]. + +[IMPORTANT] +==== +* Once `enableCrossClusterVersioning` is enabled, it cannot be disabled. +As a result, HLV cannot be disabled. + +* `enableCrossClusterVersioning` can be enabled only when editing a bucket. + +* In the future, to add a new cluster to the existing replication topology, enable HLV on each new cluster to activate conflict logging. Because only the documents mutated after enabling the `enableCrossClusterVersioning` property on both the source and target clusters, are considered for conflict logging. +==== + +Couchbase recommends to monitor bucket memory quota and storage usage after enabling HLV to ensure adequate resources are available. + +For more information about the bucket property `enableCrossClusterVersioning` and HLV, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + +[#use-conflictlogging-and-loggingrules] +=== Use conflictLogging Key and loggingRules Value in XDCR + +To log conflicts, create an XDCR replication by using the following replication settings, which accepts a key and a corresponding JSON object as the value. + +* `conflictLogging` is the replication setting key. +* `loggingRules` is the JSON object as the value. + +These replication settings are a part of the advanced settings of the XDCR replication. See xref:rest-api:rest-xdcr-adv-settings.adoc[Manage Advanced Settings]. + +To configure the key and value: + +* From UI, see xref:manage:manage-xdcr/create-xdcr-replication.adoc[Create a Replication]. For reference, see xref:manage:manage-xdcr/replicate-using-scopes-and-collections.adoc[Replicate Using Scopes and Collections]. + +* Using REST API, see xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication]. + +NOTE: By default, conflicts are logged in the conflict collection you create. You can use `loggingRules` to override this default setting. + +[#conflictlogging-key] +==== conflictLogging key + +`conflictLogging` is the replication setting key and its details are as follows. + +|=== +| Value | Parameter/Example | Description + +| *Disabled* +|`{}` +| An empty JSON object value must be specified to disable the Conflict Logging feature. + +| *Enabled for all conflicts* + +| Parameter: + +`{ + “bucket”: [bucketName], + “collection”: [scope].[collection]”, +}` + +Example: + +`{ + “bucket”: “bucket1.prime”, + “collection”: “scope2.collection3”, +} +{ + “bucket”: “b2.prime”, + “collection”: “_default.col4”, +}` + +| The `bucket` key in the embedded object indicates the bucket in which the conflict collection is located. Periods (`.`) are allowed in bucket names. The `collection` key in the embedded object indicates the scope and collection namespace for the conflict collection. This is required to enable the Conflict Logging feature. + +| *Enabled with special logging rules* + +| Parameter: + +`{ + “bucket”: [bucketName], + “collection”: [scope].[collection]”, + “loggingRules”: {}, +}` + +| The `loggingRules` key is optional. If you use the `loggingRules` key, then the value must be a non-empty JSON object with specific key-value pairs, where each pair is a specific rule. The mutations that cause conflicts are compared with the rules. If a conflict causing mutation does not belong to a rule from `loggingRules`, that conflict is not logged. + +For example explanation, see xref:xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#example-special-loggingrules[Example for Special Logging Rules]. + +| *Disabled with stored logging rules* + +| +`{ + “bucket”: [bucketName], + “collection”: [scope].[collection]”, + “loggingRules”: {...}, + “disabled”: true, +}` + +| This is similar in function to the *Disabled* potential value. +However, this value provides the additional benefit of saving the logging rules on disk, which enables the restoration of the rules from the UI. +|=== + +[#example-special-loggingrules] +==== Example for Special Logging Rules + +*Example*: + +---- +{ + "bucket": "bucket1", + "collection": "scope2.collection3", + "loggingRules": { + "s1": {}, + "s1.privateCol": null, + "privateScope": null, + "privateScope.specialCol": {}, + "rbacScope": { + "bucket": "customBucket", + "collection": "customScope.customCollection" + }, + "rbacScope.rbacCollection": { + "bucket": "customBucket", + "collection": "specialScope.specialCollection" + } + } +} +---- + +*Explanation*: + +* Default logging of conflicts happens in `scope2.collection3` of `bucket1`. + +* You can override the defaults by providing special logging rules using `loggingRules`. + +** All conflicts from `s1` except mutations that belong in `s1.privateCol` will be logged in the conflict collection `scope2.collection3` of `bucket1`. + +** None of the conflicts from `privateScope` will be logged. However, documents from a special collection `privateScope.specialCol` in the same `privateScope` will be logged. + +** All conflicts from `rbacScope` will be logged in a separate bucket `customBucket` and collection `customScope.customCollection`. + +** If a conflict is caused by a mutation from `rbacScope.rbacCollection`, then that conflict is logged in `specialScope.specialCollection`. + + +[#logging-rules] +==== loggingRules + +This section explains the mapping of conflict logs to their source. The value of `loggingRules` is a JSON object composed of any number of the following Key-Value (KV) formats. + +[NOTE] +==== +* The keys and their priorities of each KV pair are specified in the format similar to xref:manage:manage-xdcr/replicate-using-scopes-and-collections.adoc##rules-for-explicit-mappings[Rules for Explicit Mappings] and also have the same priority as listed in the mapping rules. + +* Similar to explicit mapping mode, if a source scope or collection is not specified, then the conflict from that source is ignored. + +* Conflict logging rules can be saved without enabling the Conflict Logging setting, however, the rules will come into effect only when the Conflict Logging setting is enabled. +==== + +|=== +| Key | Value | Description + +| [scope] +| `{}` + +Empty JSON object. +| All mutations from the specified scope will be logged in the conflict collection defined by bucket, scope, and collection in the *ConflictLogging* key. + +| [scope] +| `null` +| All mutations from the specified scope will not be logged. + +| [scope.collection] +| `{}` +| All mutations from the specified collection will be logged in the conflict collection defined by bucket, scope, and collection in the *ConflictLogging* key. + +| [scope.collection] +| `null` +| All mutations from the specified collection will not be logged. + +| [scope] +|`{ + “bucket”: [otherBucket], + “collection”: [otherScope].[otherCollection]” + }` +| All mutations from the chosen scope will be logged in the specific conflict collection defined by otherBucket, otherScope, and otherCollection; instead of the original location mentioned in the *ConflictLogging* key. + +| [scope.collection] +|`{ + “bucket”: [otherBucket], + “collection”: [otherScope].[otherCollection]” + }` +| All mutations from the chosen scope will be logged in the specific conflict collection defined by bucket, scope, and collection; instead of the original location mentioned in the *ConflictLogging* key. +|=== + +[#tunable-replication-settings] +==== Tunable Replication Settings + +By default, the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-logger[Conflict Logger] is tuned to handle a low percentage of conflicts. +Use the following settings to tune the Conflict Logger. + +|=== +| Setting | Description | Default + +| `conflictLogging` +| The main setting which when non-empty and valid enables logging of conflicts. + +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflictlogging-key[conflictLogging Key]. +| + +| `cLogSetTimeoutMs` +| Sets the timeout for SetMeta operation for conflict writing. +| 5000ms + +| `cLogPoolGetTimeoutMs` +| Timeout for getting connection from connection pool +| 5000ms + +| `cLogNetworkRetryCount` +| Number of attempts for a single doc when there is a network error +| 5 + +| `cLogNetworkRetryIntervalMs` +| Time interval to sleep between attempts to write the conflict doc on network error +| 2000ms + +| `cLogWorkerCount` +| Number of workers used by the conflict logger, which actually write conflict documents. +| 20 + +| `cLogQueueCapacity` +| Queue capacity for the conflict logger. +| 6000 + +| `cLogMaxErrorCount` +| Max number of errors encountered by conflict logger in a “Error Window” after which the conflict logger will be hibernated. + +For information about hibernation, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#reliability-components-logger[Reliability Components]. +| 10 + +| `cLogErrorTimeWindowMs` +| This is the time window in which the number of errors in the conflict logger will be checked. +| 120000ms + +| `cLogReattemptDurationMs` +| This is the duration for which the conflict logger will be hibernated and after this the logger will be active again. + +For information about hibernation, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#reliability-components-logger[Reliability Components]. +| 600000ms +|=== + +[#deploy-xdcr-conflict-logging] +== Deploying XDCR Active-Active Conflict Logging Setup + +Deploy XDCR active-active setup and enable logging of conflicts using one of the following procedures: + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#greenfield-deployment-conflict-logging[Greenfield Deployment]: New installation of Couchbase Server 8.0.0 (or a later version) clusters. Then enable the Conflict Logging feature for each necessary bucket. + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#upgrade-xdcr-setup-conflict-logging[Upgrade]: Upgrade an XDCR setup from a Server version earlier than 8.0.0 to the Server version 8.0.0 (or a later version). +Then enable the Conflict Logging feature for each necessary bucket. + +[#xdcr-conflict-logging-prerequisites] +=== Prerequisites for XDCR Active-Active Conflict Logging + +To use the XDCR Conflict Logging feature, it must be enabled. +The prerequisites for enabling XDCR’s conflict logging capability are as follows: + +. *Bi-directional replication setup*: All clusters in the XDCR topology must be running Couchbase Server 8.0.0 or a later version. +XDCR supports conflict logging only in bi-directional replications. +In these replications, mutations from any cluster can be replicated to any other cluster. ++ +[#presence-of-conflict-collection] +. *Presence of Conflict Collection*: Ensure you have created a collection in each participating clusters and designated these collections for logging conflicts. ++ +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#create-conflict-collection[Create Conflict Collection]. + +. *Hybrid Logical Vectors (HLVs) compatibility*: Set the bucket property `enableCrossClusterVersioning` to `true` on all the buckets in the XDCR topology clusters to enable HLV. ++ +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#enable-hlv-conflict-logging[Enable Hybrid Logical Vectors (HLVs)]. + +[#greenfield-deployment-conflict-logging] +=== Greenfield Deployment + +To configure a new active-active XDCR setup for logging conflicts, do the following: + +. Create two clusters, A and B, with all the nodes of both the clusters on Server 8.0.0 (or a later) version. ++ +IMPORTANT: Couchbase doesn’t recommend replicating data with Server versions earlier than 8.0.0 because conflicts won’t be logged and HLV metadata won't update properly. +This can result in data loss and undetected conflicts. + +. Create buckets, for example, B1 and B2 in Cluster A and Cluster B respectively, between which XDCR will be set up. +Now, do the following: ++ +NOTE: If the replication topology has more than two buckets, you must enable ECCV for all those buckets. + +.. Enable the `enableCrossClusterVersioning` (ECCV) setting on B1. All mutations in B1 from this point forward will include a new metadata Hybrid Logical Vector (HLV). Refer to this time as T1. ++ +[NOTE] +==== +* Once enabled, `enableCrossClusterVersioning` cannot be disabled. +* For information about enabling ECCV, see xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing] using REST API and xref:manage:manage-buckets/edit-bucket.adoc#edit-bucket-for-eccv[Edit a Bucket to Enable Cross Cluster Versioning] from the UI. +==== +.. Enable the ECCV setting on B2. All mutations in B2 from this point forward will include a new metadata HLV. Refer to this time as T2. + +. Create bucket(s) and collection(s) for logging conflicts on both Cluster A and Cluster B using the following: +** xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#create-conflict-collection[Create Conflict Collection]. +** xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging +[Replication Settings for XDCR Conflict Logging] from UI. +** xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication] using REST API. ++ +NOTE: Ensure to set a size enough to accommodate the expected volume of conflicts. + +. Create XDCR from B1 to B2 using xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging +[Replication Settings for XDCR Conflict Logging] from UI or see xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication] using REST API. Select the target(s) in A, where the conflicts from A, detected by XDCR, must be logged. + +. Create XDCR from B2 to B1 using xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging +[Replication Settings for XDCR Conflict Logging] from UI or see xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication] using REST API. Select the target(s) in B, where the conflicts from B, detected by XDCR, must be logged. + +. Fine tune the Conflict Logger on both clusters, A and B, depending on the percentage of expected conflicts, using xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#tunable-replication-settings[Tunable Replication Settings]. + +After the configuration, this setup is ready to process application traffic on both B1 and B2. +When a document mutated at time t1 (in B1) and another document mutated at time t2 (in B2) are detected to be in conflict, then: + +* Both documents are logged along with a conflict metadata document in respective conflict buckets. + +* Logging occurs only if t1 > T1 and t2 > T2. + +[#upgrade-xdcr-setup-conflict-logging] +=== Upgrading an Existing XDCR Active-Passive Setup + +Convert an existing XDCR active-passive setup into XDCR active-active setup for logging conflicts. + +For illustration, there are two clusters, A and B. +Cluster A is active in the XDCR setup with bucket B1 connected to passive cluster B in the XDCR setup with bucket B2. +Currently, the application traffic will be only on bucket B1 of Cluster A. + +. Upgrade both clusters A and B with all the nodes of the clusters to Server 8.0.0 (or a later) version. ++ +NOTE: Couchbase doesn’t recommend replicating data on Server versions earlier than 8.0.0 because conflicts won’t be logged and HLV metadata won't update properly. +This can result in data loss and undetected conflicts. + +. On B1 and B2, enable the `enableCrossClusterVersioning` (ECCV) setting. +All mutations in B1 and B2 from this point forward will include a new metadata Hybrid Logical Vector (HLV). +Refer to this time as T1 and T2 for B1 and B2 respectively. ++ +[NOTE] +==== +* Once enabled, `enableCrossClusterVersioning` cannot be disabled. +* If there are more than two buckets in the replication topology, you must enable ECCV for all those buckets. +* For information about enabling ECCV, see xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing] using REST API and xref:manage:manage-buckets/edit-bucket.adoc#edit-bucket-for-eccv[Edit a Bucket to Enable Cross Cluster Versioning] from the UI. +==== + +. Create bucket(s) and collection(s) for logging conflicts on both Cluster A and Cluster B using the following: +** xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#create-conflict-collection[Create Conflict Collection]. +** xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging +[Replication Settings for XDCR Conflict Logging] from UI. +** xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication] using REST API. ++ +NOTE: Ensure to set a size enough to accommodate the expected volume of conflicts. + +. Update the existing XDCR replication setting from B1 to B2 by configuring xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging +[Replication Settings for XDCR Conflict Logging] from UI or see xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication] using REST API. +Select the target(s) in A, where th conflicts from A, detected by XDCR, must be logged. + +. Create XDCR from B2 to B1 using xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging +[Replication Settings for XDCR Conflict Logging] from UI. Select the target(s) in B, where the conflicts from B, detected by XDCR, must be logged. + +. Fine tune the Conflict Logger on both clusters, A and B, depending on the percentage of expected conflicts, using xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#tunable-replication-settings[Tunable Replication Settings]. + +. Enable application traffic on cluster B. + +After the configuration, this setup is ready to process application traffic on both B1 and B2. +When a document mutated at time t1 (in B1) and another document mutated at time t2 (in B2) are detected to be in conflict, then: + +* Both documents are logged along with a conflict metadata document in respective conflict buckets. + +* Logging occurs only if t1 > T1 and t2 > T2. + +[#xdcr-conflict-detection-logging] +== Conflict Detection and Logging + +During XDCR replication, conflicts are supposed to be uncommon. +If there are any, then XDCR detects conflicts by comparing the HLV (Hybrid Logical Vector) metadata of the source and target documents. + +After detecting conflicts, XDCR defines each logging event as a conflict record in a xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#create-conflict-collection[Conflict Collection]. + +IMPORTANT: If logging fails, XDCR continues with conflict resolution to ensure High Availability. + +[#conflict-record] +=== Conflict Record + +A conflict record consists of these three elements: + +* *Conflict Record Document (CRD):* It is a JSON record document that contains source and target document key-value pairs. CRD consists of the following elements: +** Metadata document. +** Document key: `crd__`. +** Source document key and target document key. + +* *Source document:* It is a copy of the source document that caused the conflict. This logged source document consists of the following elements: +** Document body and xattrs of the source bucket from the conflict. +** Document key: `src__`. +* *Target document:* It is a copy of the target document that caused the conflict. This logged target document consists of the following elements: +** Document body and xattrs of the target bucket from the conflict. +** Document key: `tgt__`. + +The generated `` ensures that each conflict for the same document is unique and logs each conflict for each document key. + +XDCR adds a hidden system xattribute flag called `_xdcr_conflict` into each of the three documents in a conflict record to prevent the documents from replicating to the target. + +[#conflict-record-document-format] +=== Conflict Record Document Format + +This is an example of Conflict Record Document. + +[source,json] +---- +{ + "docId": "ae03ecde-203c-4a3b-a6d0-36b60f627c8a", + "id": "crd_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "replId": "d976db26a5ce512a243db65a3026073f/bucket1/bucket1-nhuUKw2mOVPRiDvcoF4hJg==", + "srcDoc": { + "bucketUUID": "fada12469f1027ecb078d2c19a2802fb", + "cas": 1749453999768928300, + "clusterUUID": "1bc375a26b7f822ebe91288c58a6c614", + "collection": "_default", + "expiry": 0, + "flags": 0, + "id": "src_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "isDeleted": false, + "nodeId": "127.0.0.1", + "revSeqno": 2, + "scope": "_default", + "xattrs": { + "_mou": "", + "_sync": "", + "_vv": "" + } + }, + "tgtDoc": { + "bucketUUID": "6cd3a35ba2d337d84d4fcd30c3a46fb9", + "cas": 1749453999840231400, + "clusterUUID": "d976db26a5ce512a243db65a3026073f", + "collection": "_default", + "expiry": 0, + "flags": 0, + "id": "tgt_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "isDeleted": false, + "nodeId": "18.206.194.145", + "revSeqno": 2, + "scope": "_default", + "xattrs": { + "_mou": "", + "_sync": "", + "_vv": "{\"cvCas\":\"0x00003dca244f4718\",\"src\":\"ntR6bcR9VHcjcPpQ5osEiw\",\"ver\":\"0x00003dca244f4718\"}" + } + }, + "timestamp": "2025-06-09T07:26:40.005Z" +} +---- + +Some of the key-value pairs in CRD document are as follows: + +|=== +| Key | Value + +| `timestamp` +| Exact date and time of the event in UTC iso-8601 format: +YYYY-MM-DDThh:mm:ss:SSSZ + +The timestamp will be the wall clock of the node that noted the conflict. + +| `id`/`srcDoc.id`/`tgtDoc.id` +| CRD document ID prefixed with `crd_`. + +A copy of conflict source document ID prefixed with `src_`. + +A copy of conflict target document ID prefixed with `tgt_`. + +| `docId` +| document id of the actual document in conflict. + +| `replId` +| Replication ID. + +| `srcDoc` and `tgtDoc` +| Document ID of the “copy” of source document to look up in the conflict collection and the “copy” of the target document to look up in the conflict collection. + +| `nodeId` +| Node identification, which can be the hostname or IP address and port. + +| `bucketUUID` +| Source/target bucket unique ID. + +| `cas` +| Source/target document CAS. + +| `clusterUUID` +| Source/target cluster unique ID. + +| `isDeleted` +| True or false for is the source/target document deleted? + +| `scope` and `collection` +| Name of the scope and collection where the conflict source/target documents exist. + +| `revSeqno` +| Source/target document’s revision sequence number. + +| `expiry` +| Source/target document’s expiry time. + +| `flags` +| Source/target document’s flag. + +| `xattrs` +| System xattributes. + +| `_vv` +| Document version vector. +HLV metadata. +|=== + +[#conflict-logger] +=== Conflict Logger + +The Logger in XDCR records conflict events detected during a replication. +It operates as an asynchronous component designed to minimize the impact on replication performance and system resources. + +Conflict logging is attempted on the best-effort basis, which makes it the lowest operational priority. +If any error occurs, such as low memory quota or a lag in storing conflict logs, XDCR may pause or stop the logging process, reports the error type for the replication pipeline, and continues with replication so there is no High Availability issue. +See xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-logging-errors[Conflict Logging Error Scenarios] + +By default, the Conflict Logger is tuned to handle a low percentage of conflicts. Use xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#tunable-replication-settings[Tunable Replication Settings] to change the default settings of the Conflict Logger. + +`xdcr_clog_status` is a Prometheus stat, which indicates the status of the Conflict Logger. +The status can be one of the following: + +* Hibernated: The logger is temporarily disabled due to persistent errors or resource constraints. ++ +NOTE: Hibernation prevents the degradation of the replication performance. Logging is re-enabled after a set interval, or after the errors are manually resolved and the logger is manually restarted. + +* Running: The logger is actively processing conflict events. + +* DNE: The logger does not exist. + +[#conflict-access-and-review] +== Conflict Access and Review + +XDCR conflict logs are your application documents, so they're added in regular collections called xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#create-conflict-collection[Conflict Collection] designated by you. + +You can access, view, and process these documents programmatically, similar to your application documents, using the available utilities, features, and options. + +For example, you can use RBAC to manage access to conflict collections. +You can also use Couchbase services and tools such as Eventing and Kafka Connect to process conflict log documents. + +For information about the Conflict Record Document, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Record]. + +[#retrieve-and-clean-up-conflict-record] +=== Retrieve and Clean Up Conflict Record + +You can monitor and manage the conflict records xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#using-kv-range-scan-clog[Using KV Range Scan]. + +The following is a high-level procedure about using conflict logs from a conflict collection: + +. Connect an SDK to the conflict collection. + +. Issue a KV range start with the following parameters: + +** Start: `crd_` (the prefix for conflict record document) +** End: Not specified + +. KV range scan returns an ID, which you use for the scan. + +. Issue a KV range scan `continue` command using the ID repeatedly. + +. Each `continue` command returns a single CRD document. + +. Examine the CRD document. Within the document, perform GET on source and target document ID. Open the document ID that matches your search with the following format. + +** `src__` for source conflict document. +** `tgt__` for target conflict document. + +. Resolve conflicts manually and write back to the source bucket. + +. Optional step: Clean up by starting a transaction. + +.. Delete the Conflict Record Document (CRD). +.. Delete the conflict source document. +.. Delete the conflict target document. + +[#using-query-clog] +==== Using Query + +KV range scan can work only using Data Service capability. +You can also use query instead of KV range scan. +However, this potentially requires creation of secondary indexes. +Additionally, it requires Index service and Query service to be activated in the cluster. +To access, view, and retrieve conflict logs using query, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#using-sdk-sqlpp-for-conflict-logs[Using Couchbase SDK with SQL++ for Conflict Logs]. + +[#using-sdk-sqlpp-for-conflict-logs] +=== Using Couchbase SDK with SQL++ for Conflict Logs + +The most common way to view these documents is by using the Couchbase SDK with SQL++. + +The following are a few examples. +In the examples, the conflict logs are stored in the collection `conflictlogs.bucket1.app1logs`. + +[#sqlpp-examples-clog] +==== SQL++ Examples + +For queries, you can create a custom index or a primary index. + +* Create a primary index as follows. ++ +[source,sqlpp] +---- +CREATE PRIMARY INDEX `#primary` ON `conflictlogs`.`bucket1`.`app1logs` +---- + +* Create a custom index as follows. ++ +You can use the `WHERE` clause with `SUBSTR(META().id,0,3) = “crd”` by making it an indexed field instead of `WHERE META().id LIKE "crd%"`. ++ +For example: ++ +[source,sqlpp] +---- +CREATE INDEX `idx_crd_timestamp_docId` ON `conflictlogs`.`bucket1`.`applogs`(SUBSTR(META().id,0,3), `timestamp`,`docId`) +---- ++ +NOTE: When creating custom indexes, `META().id` is already a part of the index key. +When using `LIKE` in the queries, underscore is a wildcard. +If you use underscore in comparison strings, ensure you use the escape character, so that it's used as an exact comparison instead of a wildcard comparison. +For example, `WHERE META().id LIKE “crd\\_%”`. + +*Query example 1:* List all `docId` that had conflicts between two time periods. + +[source,sqlpp] +---- +SELECT META().id, b.docId, b.timestamp +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META().id LIKE "crd%" and b.timestamp between "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +*Query example 2:* Number of conflicts logged between two time periods. + +[source,sqlpp] +---- +SELECT count(*) AS num_conflicts +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META().id LIKE "crd%" and b.timestamp between "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +*Query example 3:* Number of different `docIds` that had conflicts between two time periods. + +[source,sqlpp] +---- +SELECT count (distinct b.docId) as num_different_docIds +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META().id LIKE "crd%" and b.timestamp between "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +*Query example 4:* List all `docIds` and the number of times each `docId` had a logged conflict between two time periods. + +[source,sqlpp] +---- +SELECT b.docId AS DOCID, COUNT(1) AS num_times_seen +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META(b).id LIKE "crd%" AND b.timestamp BETWEEN "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +GROUP BY b.docId +ORDER BY num_times_seen DESC LIMIT 20 +---- + +The query result is as follows: + +---- +[ + { + "DOCID": "4d9d18e7-5fa2-4eae-b32b-60d3d369da94", + "num_times_seen": 2 + }, + { + "DOCID": "ae03ecde-203c-4a3b-a6d0-36b60f627c8a", + "num_times_seen": 1 + } +] +---- + +*Query example 5:* List all `docIds` and fields from the `src` and `tgt` documents that were in conflict, to view only the `docId` that had conflict with select fields from the `src` and `tgt` documents. + +[source,sqlpp] +---- +SELECT b.docId, b.srcDoc.id AS srcDocId, b.tgtDoc.id AS tgtDocId, +(SELECT b2.country, b2.note +FROM `conflictlogs`.`bucket1`.`app1logs` AS b2 USE KEYS b.srcDoc.id)[0] AS src_doc_fields, +(SELECT b3.country, b3.note +FROM `conflictlogs`.`bucket1`.`app1logs` AS b3 USE KEYS b.tgtDoc.id)[0] AS tgt_doc_fields +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META(b).id LIKE "crd%" AND b.timestamp BETWEEN "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +The query result is as follows: +---- +[ + { + "docId": "ae03ecde-203c-4a3b-a6d0-36b60f627c8a", + "srcDocId": "src_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "tgtDocId": "tgt_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "src_doc_fields": { + "country": "UK", + “note”: “need review” + }, + "tgt_doc_fields": { + "country": "CA", + “note”: “ok” + } + } +] +---- + +NOTE: As the conflict log documents won’t be replicated, if you need to replicate them, copy them into a different document. + +[#using-eventing-for-conflict-logs] +=== Using Eventing for Conflict Logs + +The conflict logs and conflict documents are not replicated as XDCR’s special attribute (system xattrs called `_xdcr_conflict`) prevents replication: + +---- +"xattrs": { + "_xdcr_conflict": true + } +---- + +To force replicate the conflict logs and conflict documents, do the following: + +. Use Eventing to copy the document to a new collection along with other modifications. + +. Replicate the new document from the new collection. + +You cannot update system xattrs, however, following is an example of an Eventing function you can deploy to combine the `crd_`, `src_`, and `tgt_` documents into one new document that does not have the system attribute. +Here, it is assumed that the documents are small. +If the documents are large, then you can copy only the relevant fields from the `src` and `tgt` documents to a single document. + +[#eventing-example-clog] +==== Eventing example + +This is an Eventing function example to copy the conflict log documents into a single new document. + +Eventing bucket alias: `src` for `conflictlogs._default.bucket1logs` with read only access. + +Eventing bucket alias: `copy` for `conflictlogs._default.bucket1logs_copy` with read/write access. + +[source,javascript] +---- +function OnUpdate(doc, meta, xattrs) { + if (meta.id.startsWith("crd_") === true) + { + log("crd doc created/updated", meta.id); + + //get the conflict source and target document id's + var suffix = (meta.id).substring(4); + var srcId = "src_" + suffix; + var tgtId = "tgt_" + suffix; + + //read the conflicting source and target documents + var src_doc = src[srcId]; + var tgt_doc = src[tgtId]; + + //add the conflicting source and target documents to the crd doc + doc["src_doc"] = src_doc; + doc["tgt_doc"] = tgt_doc; + + //copy the resulting new crd doc to a different collection + copy[meta.id] = doc; + } +} +---- + +Resultant document example: + +NOTE: See xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record-document-format[Conflict Record Document Format] to understand the document format. + +---- +{ + "timestamp": "2025-05-20T19:25:55.308Z", + "id": "crd_1747769155308581666_c0468a42632daabce649b1fb7e174a530946c04b1dde75218290c4885ad11669", + "docId": "ASHUNK100", + "replId": "1bc375a26b7f822ebe91288c58a6c614/bucket1/bucket1-PN8-oFADADCOe6S6A8YuIw==", + "srcDoc": { + "id": "src_1747769155308581666_c0468a42632daabce649b1fb7e174a530946c04b1dde75218290c4885ad11669", + "nodeId": "127.0.0.1", + "bucketUUID": "6cd3a35ba2d337d84d4fcd30c3a46fb9", + "clusterUUID": "d976db26a5ce512a243db65a3026073f", + "isDeleted": false, + "collection": "_default", + "scope": "_default", + "expiry": 0, + "flags": 33554438, + "cas": 1747769124557291500, + "revSeqno": 4, + "xattrs": { + "_vv": "{\"cvCas\":\"0x00002a1777524118\",\"src\":\"ntR6bcR9VHcjcPpQ5osEiw\",\"ver\":\"0x00002a1777524118\",\"pv\":[\"000018fe76524118@8DdrZBqHaACpmDeYJIv+kg\"]}", + "_sync": "", + "_mou": "" + } + }, + "tgtDoc": { + "id": "tgt_1747769155308581666_c0468a42632daabce649b1fb7e174a530946c04b1dde75218290c4885ad11669", + "nodeId": "34.213.22.102", + "bucketUUID": "fada12469f1027ecb078d2c19a2802fb", + "clusterUUID": "1bc375a26b7f822ebe91288c58a6c614", + "isDeleted": false, + "collection": "_default", + "scope": "_default", + "expiry": 0, + "flags": 33554438, + "cas": 1747769104359030800, + "revSeqno": 4, + "xattrs": { + "_vv": "{\"cvCas\":\"0x000018fe76524118\",\"src\":\"8DdrZBqHaACpmDeYJIv+kg\",\"ver\":\"0x000018fe76524118\"}", + "_sync": "", + "_mou": "" + } + }, + "src_doc": { + "author": "Unknown", + "create_date_utc": "2025-05-20T07:56:26", + "res_from": "WEST", + "res_name": "Harry West", + "res_note": "visitor -- HJV NOTE EAST", + "reserved": true, + "title": "Ashmole 782", + "type": "book", + "unix_time": "1747768801", + "update_date_utc": "2025-05-20T19:20:01" + }, + "tgt_doc": { + "author": "Unknown", + "create_date_utc": "2025-05-20T07:56:26", + "res_from": "WEST", + "res_name": "Harry West", + "res_note": "visitor -- HJV NOTE WEST", + "reserved": true, + "title": "Ashmole 782", + "type": "book", + "unix_time": "1747768801", + "update_date_utc": "2025-05-20T19:20:01" + } +} +---- + +[#using-kv-range-scan-clog] +=== Using KV Range Scan + +To use only KV, use Couchbase SDK with KV APIs including xref:8.0@server:howtos/kv-operations.adoc#kv-range-scan[KV Range Scan]. + +For example, to get a list of all `crd` prefix documents (CRD documents) using the KV Range Scan xref:8.0@server:howtos/kv-operations.adoc#kv-range-scan-prefix[PrefixScan], use KV `collection.get` to get the conflicting `src` and `tgt` docs. +You can also use KV sub-document API `lookupIn` to retrieve only the selected fields. +After processing the documents, you can remove the conflict log documents using the KV `collection.remove`. + +For more information about KV Range Scan operations and data operations, see xref:8.0@server:howtos/kv-operations.adoc[Data Operations]. For more information about data operations on sub-documents, see xref:8.0@server:howtos/subdocument-operations.adoc[Sub-Document Operations]. + +[#xdcr-kv-range-scan-examples] +==== KV Range Scan examples + +The following example uses Couchbase Python SDK. + +[source,python] +---- +from datetime import timedelta +import json + +from couchbase.auth import PasswordAuthenticator +from couchbase.cluster import Cluster +from couchbase.collection import (GetOptions, RemoveOptions) +from couchbase.collection import DeltaValue, SignedInt64 +from couchbase.exceptions import ( + CouchbaseException, + PathNotFoundException, + SubdocPathMismatchException) +from couchbase.options import ScanOptions +from couchbase.kv_range_scan import PrefixScan +import couchbase.subdocument as SD + +# connection info +connection_string = "couchbase://localhost" +username = "username1" +password = "Some!password" + +# conflict logs collection location +bucket_name = "conflictlogs" +scope_name = "bucket1" +collection_name = "app1logs" + +cluster = Cluster( + connection_string, + authenticator=PasswordAuthenticator(username, password)); + +cluster.wait_until_ready(timedelta(seconds=5)) +cb = cluster.bucket(bucket_name); +logs_coll = cb.scope(scope_name).collection(collection_name) + +# Using KV PrefixScan to get list of crd doc ids +# Get crd document ids only +crd_list = logs_coll.scan(PrefixScan('crd'), ScanOptions(ids_only=True)) + +# For each crd document, get the docId, timestamp, srcDoc.id, tgtDoc.id +# Then, get the src and tgt docs from the conflict log collection +for crd in crd_list: + crdlookup = logs_coll.lookup_in(crd.id, + [ SD.get("docId"), + SD.get("timestamp"), + SD.get("srcDoc.id"), + SD.get("tgtDoc.id") ]) + + doc_id = crdlookup.content_as[str](0) + timestamp = crdlookup.content_as[str](1) + src_doc_id = crdlookup.content_as[str](2) + tgt_doc_id = crdlookup.content_as[str](3) + + # Get the src and tgt documents in conflict + src_doc = logs_coll.get(src_doc_id); + tgt_doc = logs_coll.get(tgt_doc_id); + + # Print the info as json + output = { "docId": doc_id, "log_timestamp": timestamp, + "src_doc": src_doc.content_as[dict], + "tgt_doc": tgt_doc.content_as[dict] } + print (json.dumps(output)) +---- + +[replication-continuation] +== Replication Continuation + +After you resolve conflicts and xref:n1ql:n1ql-language-reference/upsert.adoc[upsert] the documents, you can continue the replication process without data loss, through xref:learn:clusters-and-availability/xdcr-conflict-resolution.adoc#revision-id-based-conflict-resolution[Sequence number-based Conflict Resolution] or xref:learn:clusters-and-availability/xdcr-conflict-resolution.adoc#timestamp-based-conflict-resolution[Timestamp-based Conflict Resolution]. + +[#disable-conflict-logging] +== Disable XDCR Conflict Logging + +This section explains disabling XDCR Conflict Logging feature. + +* *HLV*: The bucket property `enableCrossClusterVersioning` cannot be disabled once it is enabled. +For more information, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. +Enabling HLV depends on `enableCrossClusterVersioning`, so HLV also cannot be disabled after it is enabled. + +* *Setting the replication setting off*: Set the `conflictLogging` parameter to `{}` or `{ "disabled": true }` because an empty JSON object value must be specified to disable the Conflict Logging capability of XDCR. ++ +For configuration information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#use-conflictlogging-and-loggingrules[Use conflictLogging and loggingRules]. +When it is disabled, any conflict documents previously created by XDCR in the conflict collections remain in place. + +* *Deleting a Replication*: When you delete a replication, conflict record documents in the conflict collection remain until you manually delete them. +As you can change the conflict collection multiple times during the lifecycle of a replication, XDCR does not automatically clean up previously created conflict records. + +[#conflict-logging-errors] +== Conflict Logging Error Scenarios + +This section explains errors and their causes that may occur during the conflict logging process. + +[#error-source-side-network-partition] +=== Source-Side Network Partition + +A network partition on the source side can prevent XDCR from reading or writing to the conflict collection if the transaction mode is enabled. +The following scenarios may occur: + +* *Replication Creation/Change/Deletion*: If you create a replication and specify a conflict collection, the operation fails during a network partition. +However, by design, replication of conflict collection is not supported by XDCR. +So, replication creation would fail under these conditions regardless. ++ +Any replication change during a network partition is not recommended, as metakv may also encounter issues. +Problems with accessing the conflict collection during a network partition may not introduce new issues but may prevent some conflicts from being logged. + +* *Live Replication*: Standard XDCR replication is generally not affected by a source-side network partition. +However, certain features, such as Peer-to-Peer protocols used for checkpoint migration may be impacted. + +[#error-conflict-collection-quota] +=== Conflict Collection Usage Quota + +If the bucket containing the conflict collection reaches its quota and cannot accept more data, for example, if the Resident Ratio guardrail is triggered, XDCR cannot log conflicts but the replication continues. +This approach is intended to prioritize continued replication and maintain High Availability, even if some conflict events are not logged. + +[#error-handling-conflict-logging] +=== Error Handling + +In all error scenarios described above, XDCR makes a best-effort attempt to record the conflict document. +If logging fails, XDCR advances the sequence number to continue processing replication. diff --git a/modules/learn/pages/clusters-and-availability/xdcr-active-active-sgw.adoc b/modules/learn/pages/clusters-and-availability/xdcr-active-active-sgw.adoc new file mode 100644 index 0000000000..d192c23041 --- /dev/null +++ b/modules/learn/pages/clusters-and-availability/xdcr-active-active-sgw.adoc @@ -0,0 +1,99 @@ += XDCR Active-Active with Sync Gateway +:description: pass:q[You can use XDCR with Sync Gateway mobile clusters in a bi-directional, active-active replication, but you must make sure that both the Server and the Sync Gateway versions support this option. Otherwise, using XDCR with Sync Gateway buckets in a bi-directional replication can cause data corruption.] + +[abstract] +{description} + +[#xdcr-active-active-sgw-intro] +== Introduction + +NOTE: *To set up XDCR bi-directional replication with Sync Gateway (SGW), you need to have at least a Server 7.6.6 version and SGW 4.0.0 version. +However, Sync Gateway 4.0.0 is a future release version. + +In the versions earlier than Server 7.6.6 and Sync Gateway (SGW) 4.0.0*, only an active-passive setup was supported with both XDCR and SGW. +XDCR Active-Active replication with Sync Gateway for XDCR-Mobile interoperability configuration was introduced in the Server 7.6.6 version, where you can configure an active-active XDCR setup with Sync Gateway (SGW) and mobile applications both on the XDCR source and target clusters. + +[IMPORTANT] +==== +Here are a few limitations to the _XDCR Active-Active with Sync Gateway_ feature. + +* If you use the _user created extended attributes (user xattrs)_ in your documents, and you have more than 10 user xattrs in a document, then you cannot use the feature _XDCR Active-Active with Sync Gateway_. +This is due to an internal limitation of managing extended attributes in a document. +If you try to use the feature _XDCR Active-Active with Sync Gateway_ when you have more than 10 user xattrs in your document, the XDCR replication **silently skips** replicating that document. +As a result, the data in the replication-skipped document will not be consistent between the target and source clusters. +The only way you will know this skip occured is because the Prometheus stat `subdoc_cmd_docs_skipped` will be incremented and the document will _not_ be consistent between the target and source. +* If you use Eventing service functions that update documents in XDCR-replicated buckets (Eventing source bucket mutations), ensure your functions do not cause continuous replication loops. +In bi-directional active-active XDCR environments, Eventing functions that trigger document updates can lead to "ping-pong" replication unless you implement logic to prevent infinite loops. +Always add safeguards to avoid redundant updates and unwanted replication behavior in bi-directional setups. For more information, see xref:sync-gateway:xdcr-active-active-eventing.adoc[XDCR Active-Active and Eventing]. +==== + +You can configure XDCR Active-Active with Sync Gateway for XDCR-Mobile interoperability using one of the following methods: + +* xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc#xdcr-active-active-sgw-greenfield-deployment[Greenfield deployment]: Set up a new active-active configuration with both XDCR and SGW. +* xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc#xdcr-active-active-sgw-upgrade[Upgrading an existing setup]: Convert an existing active-passive XDCR-SGW configuration to an active-active XDCR-SGW setup. + +NOTE: When using the feature _XDCR Active-Active with Sync Gateway_, where Sync Gateway version is 4.0* or a later version and Server is 7.6.6 or a later version, the replication target XDCR inbound user must have the RBAC roles, xref:learn:security/roles.adoc#xdcr-inbound[XDCR Inbound] role and xref:learn:security/roles.adoc#data-writer[Data Writer] role. + +[#xdcr-active-active-sgw-prerequisites] +== Prerequisites + +Set the bucket property `enableCrossClusterVersioning` to use the setting `mobile=Active` during the processes xref:manage:manage-xdcr/create-xdcr-replication.adoc[Create a Replication] in xref:manage:manage-xdcr/xdcr-management-overview.adoc[Manage XDCR]. +To enable the bucket property `enableCrossClusterVersioning` using the REST API, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#modify-enablecrossclusterversioning[Modify the bucket property enableCrossClusterVersioning] or xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing]. + +[#xdcr-active-active-sgw-greenfield-deployment] +== Greenfield Deployment + +To configure a new active-active XDCR with Sync Gateway setup, do the following: + +. Create two clusters on Server 7.6.6 or a later version with _all_ the nodes of the clusters. For example, cluster A and cluster B (or you can upgrade the existing Server clusters to 7.6.6 or a later version). +. Create buckets, for example, B1 and B2 in cluster A and cluster B respectively, between which XDCR will be set up. Now, do the following: +.. Enable the ECCV setting on B1. All the mutations in B1 will have a new metadata called HLV. +.. Enable the ECCV setting on B2. All the mutations in B2 will have a new metadata called HLV. ++ +NOTE: ECCV refers to the bucket property `enableCrossClusterVersioning`. If there are more than two buckets in the replication topology, you must enable ECCV for all those buckets. ++ +. Create an XDCR from B1 to B2 by setting `mobile=Active`. Also, create an XDCR from B2 to B1 by setting `mobile=Active`. ++ +For information about creating an XDCR by setting `mobile=Active` through the REST API, see xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication]. ++ +For information about creating an XDCR by setting `mobile=Active` from the UI, see xref:manage:manage-xdcr/create-xdcr-replication.adoc#create-an-xdcr-replication-with-the-ui[Create an XDCR Replication with the UI]. +. Configure SGW 4.0.0* version on each cluster, cluster A and cluster B. + +This setup can handle application traffic on both buckets B1 and B2 of the respective clusters along with SGW import into both the buckets simultaneously. + +[#xdcr-active-active-sgw-upgrade] +== Upgrading an existing setup + +You can convert an existing active-passive XDCR-Sync Gateway (SGW) setup into an active-active XDCR-Sync Gateway setup. + +For illustration, there are two clusters, A and B. An SGW is connected to cluster A and this cluster is active. +Cluster B is passive with XDCR setup from bucket B1 in cluster A to bucket B2 in cluster B. +The current application traffic should be only on bucket B1 of cluster A. + +.Replication before upgrade: XDCR Active-Passive with SGW +image::clusters-and-availability/xdcr-active-sgw-before-upgrade.png[,720,align=left] + +. Upgrade both clusters A and B with _all_ the nodes of the clusters to Server 7.6.6 or a later version. +. Enable ECCV on bucket B1. All the mutations in B1, after this point of time, will have a new metadata called HLV. ++ +NOTE: ECCV refers to the bucket property `enableCrossClusterVersioning`. If there are more than two buckets in the replication topology, you must enable ECCV for all those buckets. ++ +. Enable ECCV on bucket B2. All the mutations in B2, after this point of time, will have a new metadata called HLV. ++ +NOTE: If there are more than two buckets in the replication topology, you must enable ECCV for all those buckets. ++ +. Update the replication settings to `mobile=Active` of the already existing XDCR from B1 to B2. ++ +You can use the REST API or the XDCR UI to update an existing replication. For information about using the REST API to modify the replication settings for an existing replication, see xref:rest-api:rest-xdcr-adv-settings.adoc#change-existing-replication-with-mobile-active[Change Settings for an Existing Replication to Set mobile=Active] in xref:rest-api:rest-xdcr-adv-settings.adoc[Managing Advanced Settings]. ++ +. Create an XDCR from B2 to B1 with the replication settings as `mobile=Active`. +. Upgrade SGW on cluster A to the version 4.0.0*. +. Connect SGW version 4.0.0* to cluster B. +. Enable application active traffic on cluster B. + +This setup can handle application traffic on both buckets B1 and B2 of the respective clusters along with SGW import into both the buckets simultaneously. + +This is an illustration of the final configuration: + +.Replication after upgrade: XDCR Active-Active with SGW +image::clusters-and-availability/xdcr-active-sgw-after-upgrade.png[,720,align=left] diff --git a/modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc b/modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc new file mode 100644 index 0000000000..a637daa8d6 --- /dev/null +++ b/modules/learn/pages/clusters-and-availability/xdcr-conflict-logging-feature.adoc @@ -0,0 +1,541 @@ += XDCR Conflict Logging +:description: pass:q[XDCR detects and logs concurrent conflicts that occur in different clusters but on the same document version due to independent modifications by locally connected applications during Active-Active replication. The conflict logs are for your information only. The best practice in Active-Active systems is that application environments must be designed to avoid conflicts.] +:page-toclevels: 3 + +[abstract] +{description} + +During XDCR Active-Active replication, a conflict can occur if different applications connected locally to two different clusters, independently modify the same version of the document at around the same time. +These simultaneous modifications happen within the convergence window*, so the changes made in one document are not reflected in the history of the other. + +If conflict logging is enabled, XDCR logs these conflict events in a conflict log collection, specified by you, for your information. +The conflict logging does not change conflict resolution. + +XDCR performs conflict resolution and replicates the documents whether conflicts are logged or not. +Conflict logging is informational only, so that you can review the conflict log message documents and the copies of the documents that were in conflict, to help you design application systems that avoid conflicts. + +_*A convergence window is a period during which multiple actors, such as users, applications, or clusters, can modify the same document in an active-active topology, before the changes have fully converged across all clusters._ + +== Adaptibility of XDCR Conflict Logging + +* Conflict logging is asynchronous and done on a best-effort basis, meaning, not all conflicts may be logged. + +* Conflicts are expected to be rare during XDCR replication. +If there is an upsurge of conflicts, so that logging conflicts may cause replication issues, XDCR stops logging conflicts temporarily to prioritize replication. + +* If any error occurs, due to resource issues or a delay in writing to the conflict log collection, XDCR may pause or stop the logging process, report the error type for the pipeline, and continue with replication so that there is no High Availability issue. + +* xref:learn:clusters-and-availability/xdcr-conflict-resolution.adoc[Conflict Resolution] process remains unchanged. + +[#using-of-xdcr-conflict-logging] +== Using the XDCR Conflict Logging Feature + +To log conflicts during an XDCR, follow these steps: + +. As a prerequisite, to enable the conflict logging feature, xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[Enable the Cross Cluster Versioning] setting on each bucket of the XDCR topology. +Do this by setting the bucket property `enableCrossClusterVersioning` to `true` on all the buckets in the XDCR topology clusters. +As a result, xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#hlv-data-maintained-in-xattr[Hybrid Logical Vector (HLV)] is enabled and stored in all documents of the replication. ++ +To enable the bucket property `enableCrossClusterVersioning`: + +* Using REST API, see xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing]. + +* From the UI, see xref:manage:manage-buckets/edit-bucket.adoc#edit-bucket-for-eccv[Edit a Bucket to Enable Cross Cluster Versioning]. ++ +[NOTE] +==== +* Once `enableCrossClusterVersioning` is enabled, it cannot be disabled. +As a result, HLV also cannot be disabled. + +* When you add a new cluster to the existing replication topology, set `enableCrossClusterVersioning` to `true` in each bucket of the new cluster, to activate conflict logging. Because only the documents mutated after enabling the `enableCrossClusterVersioning` property on both the source and target clusters, are considered for conflict logging. +==== + +. When creating buckets and collections, designate at least one bucket as a conflict bucket and one collection in it as a xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-collection[conflict collection], on every cluster, for logging conflicts. +Use the following: + +** xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging +[Replication Settings for XDCR Conflict Logging], to set from UI. + +** xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication] by xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Configuring the conflictLogging Settings], to set through REST API. + +. xref:learn:clusters-and-availability/xdcr-viewing-conflict-logs.adoc[Viewing Conflict Logs]: When XDCR detects conflicts, it logs xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Record] into the conflict collection specified by you. +You can access and review the logged conflicts. + +[#conflict-collection] +== Conflict Collection + +A conflict collection is a regular collection that you create to store the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Record] (Conflict Record Document, and a copy of the source and target documents responsible for the conflict) from the affected clusters of the XDCR topology. + +For information about configuring a collection to log conflicts, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Configuring the conflictLogging Settings]. + +* As conflict collection is a regular collection, you can use regular RBAC to control standard access and programmatic access to the documents in the collection. + +* You can store conflict collection in the source bucket or in a separate collection in a bucket that's not used in any XDCR replications. + +* XDCR can write into conflict collections associated with the source cluster, however, XDCR cannot modify the conflict collections in any target clusters. + +NOTE: XDCR adds a hidden system xattribute flag called `_xdcr_conflict` into each of the three documents in a xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Record] stored in conflict collection, which prevents these documents (conflict logs) from replicating to the target. + +[#conflict-record] +=== Conflict Record + +A conflict record consists of the following three documents: + +* *Conflict Record Document (CRD):* It's a JSON record document that contains source and target document key-value pairs. +CRD consists of the following elements: +** Metadata document. +** Document key: `crd__`. +** Source document key and target document key. + +* *Source document:* It's a copy of the source document that caused the conflict. +This logged source document consists of the following elements: +** Document body and xattrs of the source bucket from the conflict. +** Document key: `src__`. +* *Target document:* It's a copy of the target document that caused the conflict. +This logged target document consists of the following elements: +** Document body and xattrs of the target bucket from the conflict. +** Document key: `tgt__`. + +The generated `` ensures that each conflict for the same document is unique and logs each conflict for each document key. + +XDCR adds a hidden system xattribute flag called `_xdcr_conflict` into each of the three documents in a conflict record to prevent the documents from replicating to the target. + +To view and access the conflict logs, see xref:learn:clusters-and-availability/xdcr-viewing-conflict-logs.adoc[Viewing Conflict Logs]. + +[#conflict-record-document-format] +=== Conflict Record Document Format + +This is an example of the Conflict Record Document. + +[source,json] +---- +{ + "docId": "ae03ecde-203c-4a3b-a6d0-36b60f627c8a", + "id": "crd_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "replId": "d976db26a5ce512a243db65a3026073f/bucket1/bucket1-nhuUKw2mOVPRiDvcoF4hJg==", + "srcDoc": { + "bucketUUID": "fada12469f1027ecb078d2c19a2802fb", + "cas": 1749453999768928300, + "clusterUUID": "1bc375a26b7f822ebe91288c58a6c614", + "collection": "_default", + "expiry": 0, + "flags": 0, + "id": "src_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "isDeleted": false, + "nodeId": "127.0.0.1", + "revSeqno": 2, + "scope": "_default", + "xattrs": { + "_mou": "", + "_sync": "", + "_vv": "" + } + }, + "tgtDoc": { + "bucketUUID": "6cd3a35ba2d337d84d4fcd30c3a46fb9", + "cas": 1749453999840231400, + "clusterUUID": "d976db26a5ce512a243db65a3026073f", + "collection": "_default", + "expiry": 0, + "flags": 0, + "id": "tgt_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "isDeleted": false, + "nodeId": "18.206.194.145", + "revSeqno": 2, + "scope": "_default", + "xattrs": { + "_mou": "", + "_sync": "", + "_vv": "{\"cvCas\":\"0x00003dca244f4718\",\"src\":\"ntR6bcR9VHcjcPpQ5osEiw\",\"ver\":\"0x00003dca244f4718\"}" + } + }, + "timestamp": "2025-06-09T07:26:40.005Z" +} +---- + +Some of the key-value pairs in CRD are as follows: + +|=== +| Key | Value + +| `timestamp` +| Exact date and time of the event in UTC iso-8601 format: +YYYY-MM-DDThh:mm:ss:SSSZ + +The timestamp will be the wall clock of the node that noted the conflict. + +| `id`/`srcDoc.id`/`tgtDoc.id` +| CRD document ID prefixed with `crd_`. + +A copy of conflict source document ID prefixed with `src_`. + +A copy of conflict target document ID prefixed with `tgt_`. + +| `docId` +| document id of the actual document in conflict. + +| `replId` +| Replication ID. + +| `srcDoc` and `tgtDoc` +| Document ID of the “copy” of source document to look up in the conflict collection and the “copy” of the target document to look up in the conflict collection. + +| `nodeId` +| Node identification, which can be the hostname or IP address and port. + +| `bucketUUID` +| Source/target bucket unique ID. + +| `cas` +| Source/target document CAS. + +| `clusterUUID` +| Source/target cluster unique ID. + +| `isDeleted` +| True or false for - is the source/target document deleted? + +| `scope` and `collection` +| Name of the scope and collection where the conflict source/target documents exist. + +| `revSeqno` +| Source/target document’s revision sequence number. + +| `expiry` +| Source/target document’s expiry time. + +| `flags` +| Source/target document’s flag. + +| `xattrs` +| System xattributes. + +| `_vv` +| Document version vector. +HLV metadata. +|=== + +[#conflict-logger] +=== Logger + +The Logger in XDCR records conflict events detected during a replication. +It operates as an asynchronous component designed to minimize the impact on replication performance and system resources. + +Conflict logging is attempted on the best-effort basis. +If any error occurs, due to resource issues or a delay in writing to the conflict log collection, XDCR may pause or stop the logging process, report the error type for the pipeline, and continue with replication so that there is no High Availability issue. + +By default, the Conflict Logger is tuned to handle a low percentage of conflicts. + +`xdcr_clog_status` is a Prometheus stat, which indicates the status of the Conflict Logger. +The status can be one of the following: + +* Hibernated: The logger is temporarily disabled due to persistent errors or resource constraints. ++ +NOTE: Hibernation prevents the degradation of the replication performance. Logging is re-enabled after a set interval, or after the errors are manually resolved and the logger is manually restarted. + +* Running: The logger is actively processing conflict events. + +* DNE: The logger does not exist. + +[#configure-conflictlogging-settings] +== Configuring the conflictLogging Setting + +You can configure a xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#conflict-collection[conflict collection] by using the `conflictLogging` key. + +The `conflictLogging` key is an advanced XDCR setting that enables or disables conflict logging and specifies where conflict records must be stored. +It allows you to define a default conflict collection and, optionally, detailed logging rules (through `loggingRules`) for different scopes and collections. +By configuring `conflictLogging`, you control whether conflicts are logged, and if so, in which bucket and collection the conflict records are saved. +For more information, see the examples and formats below. + +To log conflicts, when creating an XDCR replication use the following settings, which accepts a key and a corresponding JSON object as the value. + +* xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#conflictlogging-key[`conflictLogging`] is the key. +* xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#logging-rules[`loggingRules`] is a JSON object as the value. + +The conflict logging settings are a part of the xref:rest-api:rest-xdcr-adv-settings.adoc[Manage Advanced Settings] in XDCR replication. + +To configure the key and value: + +* From UI, see xref:manage:manage-xdcr/create-xdcr-replication.adoc[Create a Replication]. For reference, see xref:manage:manage-xdcr/replicate-using-scopes-and-collections.adoc[Replicate Using Scopes and Collections]. + +* Using REST API, when xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication], you need to configure the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflictlogging-key[conflictLogging key] and if necessary, the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#logging-rules[loggingRules]. + +NOTE: By default, conflicts are logged in the conflict collection you create. You can use `loggingRules` to override this default setting. + +[#conflictlogging-key] +=== conflictLogging key + +The types of values for the `conflictLogging` key are as follows: + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#enabled-for-all-conflicts[Enabled for all conflicts] + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#enabled-with-special-logging-rules[Enabled with special logging rules] + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#disabled-with-stored-logging-rules[Disabled with stored logging rules] + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#disabled-conflict-logging[Disabled] + +NOTE: The conflict logs are not replicated because XDCR adds a hidden system xattribute flag called `_xdcr_conflict` into each of the conflict logs (xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Records]) stored in conflict collection. +This attribute prevents conflict logs from replicating to the target. + +The `conflictLogging` setting details, value format, and examples are as follows. + +[#enabled-for-all-conflicts] +==== Enabled for all conflicts + +To log all conflicts to the same collection for the entire replication: + +. Enable conflict logging by setting `disabled` to `false`. + +. Designate a default conflict collection by specifying the conflict log bucket and the conflict log collection in `bucket` and `collection`. + +NOTE: You can designate any regular collection as the conflict collection, including collections that are a part of the replication. + +*Format:* +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": false, + "bucket": [bucketName], + "collection": [scope].[collection], + }' +---- + +*Example:* Conflict logging is enabled, and the default conflict collection is `conflictlogsbucket.myappscope.log1collection`. +All conflicts in the replication will be logged to the default conflict collection. + +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": false, + "bucket": "conflictlogsbucket", + "collection": "myappscope.log1collection" + }' +---- + +[#enabled-with-special-logging-rules] +==== Enabled with special logging rules + +To specify different conflict collection rules for different scopes and collections in your replication, define `loggingRules` in the `conflictLogging` settings. + +You can use `loggingRules` to specify, for any scope or collection in the replication, whether the conflicts from that scope or collection must be logged to a different conflict collection or not. +If you do not specify in the logging rules, then by default, the conflicts from all collections are logged to the default conflict collection. + +*Format:* +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": false, + "bucket": [bucketName], + "collection": [scope].[collection], + "loggingRules": {[logging_rules]}, + }' +---- + +*Example:* Conflict logging is enabled. The default conflict collection is the same as in the previous example, however, `loggingRules` is added. +The `conflictLogging` setting defines the following rules: + +* Conflict logs for the collection `agents.partners` must be logged to a different collection called `conflictlogsbucket2.myappscope.partnerlogs` + +* Conflicts must not be logged in the collection `agents.retailers`. + +* Conflict logs for the `inventory` scope must be logged to a collection called `conflictlogsbucket2.myappscope.inventorylogs`. + +[source,json] +---- +--data-urlencode conflictLogging= '{ +"disabled": false, +"bucket": "conflictlogsbucket1", +"collection": "myappscope.log1collection" +"loggingRules": { + "agents.partners": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.partnerlogs" + }, + "agents.retailers": null, + "inventory": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.inventorylogs" + } + } +}' +---- + +[#disabled-with-stored-logging-rules] +==== Disabled with stored logging rules + +You can configure `conflictLogging` (specify the default conflict collection and the `loggingRules`) but not enable `conflictLogging` by setting `disabled` to `true`. + +NOTE: You can update the replication and set the `conflictLogging` status `disabled` to `true` or `false` at any time. + +When you set `"disabled": true`, it also disables conflict logging, but all of the rules are retained for the collection. + +*Format:* + +[source,json] +---- +--data-urlencode conflictLogging= '{ + "bucket": [bucketName], + "collection": [scope].[collection], + "loggingRules": {[logging_rules]}, + "disabled": true, +}' +---- + +*Example:* Conflict logging is disabled, but the default conflict collection and the logging rules are retained. + +[source,json] +---- +--data-urlencode conflictLogging= '{ + "disabled": true, + "bucket": "conflictlogsbucket1", + "collection": "myappscope.log1collection", + "loggingRules": { + "agents.partners": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.partnerlogs" + }, + "agents.retailers": null, + "inventory": { + "bucket": "conflictlogsbucket2", + "collection": "myappscope.inventorylogs" + } + } +}' +---- + +[#disabled-conflict-logging] +==== Disabled + +To disable conflict logging, set `--data-urlencode conflictLogging='{}'`. + +[#logging-rules] +=== loggingRules + +Use `loggingRules` to override the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#enabled-for-all-conflicts[default conflict logging settings]. + +[NOTE] +==== +* The keys and their priorities of each KV pair are specified in the format similar to xref:manage:manage-xdcr/replicate-using-scopes-and-collections.adoc##rules-for-explicit-mappings[Rules for Explicit Mappings] and also have the same priority as listed in the mapping rules. + +* Similar to explicit mapping mode, if a source scope or collection is not specified, then the conflict from that source is ignored. + +* Conflict logging rules can be saved without enabling the Conflict Logging setting, however, the rules will come into effect only when the Conflict Logging setting is enabled. +==== + +IMPORTANT: If logging fails, XDCR continues with conflict resolution to ensure High Availability. + +The following section explains the mapping of conflict logs to their source. +The value of `loggingRules` is a JSON object composed of any number of the following Key-Value (KV) formats. + +|=== +| Key | Value | Description + +| [scope] +| `{}` + +Empty JSON object. +| All mutations from the specified scope will be logged in the conflict collection defined by bucket, scope, and collection in the *ConflictLogging* key. + +| [scope] +| `null` +| All mutations from the specified scope will not be logged. + +| [scope.collection] +| `{}` +| All mutations from the specified collection will be logged in the conflict collection defined by bucket, scope, and collection in the *ConflictLogging* key. + +| [scope.collection] +| `null` +| All mutations from the specified collection will not be logged. + +| [scope] +|`{ + “bucket”: [otherBucket], + “collection”: [otherScope].[otherCollection]” + }` +| All mutations from the chosen scope will be logged in the specific conflict collection defined by otherBucket, otherScope, and otherCollection; instead of the original location mentioned in the *ConflictLogging* key. + +| [scope.collection] +|`{ + “bucket”: [otherBucket], + “collection”: [otherScope].[otherCollection]” + }` +| All mutations from the chosen scope will be logged in the specific conflict collection defined by bucket, scope, and collection; instead of the original location mentioned in the *ConflictLogging* key. +|=== + +[#tunable-replication-settings] +== Tunable Replication Settings + +By default, the xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-logger[Conflict Logger] is tuned to handle a low percentage of conflicts. + +For retrieving the tunable settings using the REST API, in a conflict logging enabled replication, see xref:rest-api:rest-xdcr-adv-settings.adoc#rest-api-conflict-logging-replication-tunables[Retrieve the tunable settings for Conflict Logging in a replication]. + +The details of tuning the Conflict Logger is as follows: + +|=== +| Setting | Description | Default + +| `conflictLogging` +| The main setting which when non-empty and valid enables logging of conflicts. + +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflictlogging-key[conflictLogging Key]. +| + +| `cLogSetTimeoutMs` +| Sets the timeout for SetMeta operation for conflict writing. +| 5000ms + +| `cLogPoolGetTimeoutMs` +| Timeout for getting connection from connection pool +| 5000ms + +| `cLogNetworkRetryCount` +| Number of attempts for a single doc when there is a network error +| 5 + +| `cLogNetworkRetryIntervalMs` +| Time interval to sleep between attempts to write the conflict doc on network error +| 2000ms + +| `cLogWorkerCount` +| Number of workers used by the conflict logger, which actually write conflict documents. +| 20 + +| `cLogQueueCapacity` +| Queue capacity for the conflict logger. +| 6000 + +| `cLogMaxErrorCount` +| Max number of errors encountered by conflict logger in a “Error Window” after which the conflict logger will be hibernated. + +For information about hibernation, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#reliability-components-logger[Reliability Components]. +| 10 + +| `cLogErrorTimeWindowMs` +| This is the time window in which the number of errors in the conflict logger will be checked. +| 120000ms + +| `cLogReattemptDurationMs` +| This is the duration for which the conflict logger will be hibernated and after this the logger will be active again. + +For information about hibernation, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#reliability-components-logger[Reliability Components]. +| 600000ms +|=== + +[#disable-conflict-logging] +== Disable Conflict Logging + +This section explains disabling XDCR Conflict Logging feature. + +* *HLV*: The bucket property `enableCrossClusterVersioning` cannot be disabled once it is enabled. +For more information, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. +Enabling HLV depends on `enableCrossClusterVersioning`, so HLV also cannot be disabled after it is enabled. + +* *Setting the replication setting off*: Set the `conflictLogging` parameter to `{}` or `{ "disabled": true }` because an empty JSON object value must be specified to disable the conflict logging capability of XDCR. ++ +For configuration information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#use-conflictlogging-and-loggingrules[Use conflictLogging and loggingRules]. +When it is disabled, any conflict documents previously created by XDCR in the conflict collections remain in place. + +* *Deleting a Replication*: When you delete a replication, conflict record documents in the conflict collection remain until you manually delete them. +As you can change the conflict collection multiple times during the lifecycle of a replication, XDCR does not automatically clean up previously created conflict records. diff --git a/modules/learn/pages/clusters-and-availability/xdcr-conflict-resolution.adoc b/modules/learn/pages/clusters-and-availability/xdcr-conflict-resolution.adoc index 96ec6df573..9c5c644916 100644 --- a/modules/learn/pages/clusters-and-availability/xdcr-conflict-resolution.adoc +++ b/modules/learn/pages/clusters-and-availability/xdcr-conflict-resolution.adoc @@ -6,37 +6,21 @@ {description} [#conflicts_and_their_resolution] -== Conflicts and Their Resolution +== Conflict Resolution -A _conflict_ is caused when the source and target copies of an XDCR-replicated document are updated independently of and dissimilarly to one another, each by a local application. -The conflict must be _resolved_, by determining which of the variants should prevail; and then correspondingly saving both documents in identical form. -XDCR provides an automated _conflict resolution_ process. +When a source document is modified, XDCR determines whether this revision of the document must be applied to the target. +This process is called conflict resolution, which is a fully automated process. +XDCR supports the following two alternative conflict resolution policies: -Two, alternative conflict resolution policies are supported: _sequence-number-based_ (which is the default), and _timestamp-based_. -Note that _timestamp-based_ conflict resolution is only available in the Enterprise Edition of Couchbase Server. - -[#the_conflict_resolution_process] -== The Conflict Resolution Process - -When a source document is modified, XDCR determines whether this revision of the document should be applied to the target. -For documents above 256 bytes in size, XDCR fetches metadata from the target cluster before replicating. -The target metadata for the document is compared with the source metadata for the document, in order to choose which document should prevail (the exact subset of metadata used in this comparison depends on the source bucket's _conflict resolution policy_). -If the source document prevails, it is replicated to the target; if the target document prevails, the source document is not replicated. - -Once a replicated document reaches the target, the target cluster also performs a metadata comparison as described, in order to confirm that the document from the source cluster should indeed prevail. If this is confirmed, the document from the source cluster is applied to the target cluster, and the target cluster's previous version of the document is discarded. - -As a performance optimization, XDCR makes no metadata comparison on the source for documents of 256 bytes or less, thus making unnecessary a metadata fetch from the target cluster: instead, the document is replicated immediately to the target, and metadata comparison is performed there. - -If a document is deleted on the source, XDCR makes no metadata comparison on the source before replication. - -Once configured, conflict resolution is a fully automated process, requiring no manual intervention. +* Sequence number-based conflict resolution (This is the default policy). +* Timestamp-based conflict resolution. [#revision-id-based-conflict-resolution] == Conflict Resolution Based on Sequence Number Conflicts can be resolved by referring to documents' _sequence numbers_. Sequence numbers are maintained per document, and are incremented on every document-update. -A document's sequence number is stored as part of its _metadata_: specifically, as the value of the `rev` key (see xref:manage:manage-ui/manage-ui.adoc#console-documents[Documents], for details on how to inspect metadata). +A document's sequence number is stored as a part of its _metadata_: specifically, as the value of the `rev` key (see xref:manage:manage-ui/manage-ui.adoc#console-documents[Documents], for details on how to inspect metadata). The sequence numbers of source and target documents are compared; and the document with the higher sequence number prevails. If both documents have the same sequence number, the conflict is resolved by comparing the following metadata-elements, in the order shown: @@ -44,6 +28,9 @@ If both documents have the same sequence number, the conflict is resolved by com . Expiration (TTL) value . Document flags +When Cross Cluster Versioning is enabled, the Hybrid Logical Vector (HLV) metadata in the source and target documents' xattrs is also used in the conflict resolution processing. +For more information about the `enableCrossClusterVersioning` property and the HLV metadata, see xref:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + [#timestamp-based-conflict-resolution] == Timestamp-Based Conflict Resolution @@ -57,6 +44,8 @@ If both document-versions have the same timestamp-value, the conflict is resolve . Expiration (TTL) value . Document flags +When Cross Cluster Versioning is enabled, the Hybrid Logical Vector (HLV) metadata in the source and target documents' xattrs is also used in the conflict resolution processing. For more information about the `enableCrossClusterVersioning` property and the HLV metadata, see xref:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + [#time-synchronization] === Time Synchronization @@ -84,7 +73,7 @@ Each mutation has its own HLC timestamp. [#ensuring_safe_failover] === Ensuring Safe Failover -When failover (say, from data center A to data center B) is required, timestamp-based conflict resolution requires that applications redirect traffic to data center B only after the greater of the following two time-periods has elapsed: +When failover of an application is required (say, from data center A to data center B), timestamp-based conflict resolution requires that applications redirect traffic to data center B only after the greater of the following two time-periods has elapsed: * The replication latency between data centers A and B. This provides sufficient time for any _in-flight_ mutations to be received by data center B prior to traffic redirection. @@ -98,8 +87,16 @@ When availability is restored to data center A, applications must wait for the s Conflict resolution policy is configured on a per-bucket basis at bucket creation time, it cannot be changed later. For more information, see xref:manage:manage-buckets/create-bucket.adoc[Create a Bucket]. -Choosing a conflict resolution method requires consideration of the logic of the applications that require the data. -This is illustrated by the following examples: + +[IMPORTANT] +==== +* You must select the same conflict resolution policy for all the buckets in the replication topology because you can create a replication between only those buckets that have the same conflict resolution policy. +* When creating a bucket, you must actively choose the conflict resolution policy. +If you do not choose a policy, the Sequence number-based conflict resolution policy is set as default. +* After the bucket is created, you cannot change the conflict resolution policy for that bucket. In general, the Timestamp-based conflict resolution policy is preferred as the logic is easier to understand, feasible with general use cases, and also preferred for working with the latest Server features. +==== + +The following examples illustrate how the two different conflict resolution policies apply: * _Sequence-Number-based_, whereby the document with the higher number of updates wins. A hit-counter, for a website, is stored as a document within Couchbase Server: a value within the document is incremented each time the website is accessed. @@ -114,9 +111,10 @@ Therefore, in this instance, timestamp-based conflict resolution should be used, [#aligning_source_and_target_policies] == Aligning Source and Target Policies -XDCR replications cannot be created between buckets with different conflict resolution policies: source and target buckets must always be configured with the same policy. +XDCR replications cannot be created between buckets with different conflict resolution policies. The source and target buckets must always be configured with the same conflict resolution policy. -When using XDCR with a source cluster running a pre-4.6 version of Couchbase Server, only conflict resolution based on _sequence numbers_ can be used. +When creating a bucket, you must actively choose the conflict resolution policy. +If you do not choose a policy, the Sequence number-based conflict resolution policy is set as default. After the bucket is created, you cannot change the conflict resolution policy for that bucket. In general, the Timestamp-based conflict resolution policy is preferred as the logic is easier to understand. [#monitoring-conflict-resolution] == Monitoring Conflict Resolution on the Target Cluster diff --git a/modules/learn/pages/clusters-and-availability/xdcr-enable-crossclusterversioning.adoc b/modules/learn/pages/clusters-and-availability/xdcr-enable-crossclusterversioning.adoc new file mode 100644 index 0000000000..09fff9f075 --- /dev/null +++ b/modules/learn/pages/clusters-and-availability/xdcr-enable-crossclusterversioning.adoc @@ -0,0 +1,73 @@ += XDCR enableCrossClusterVersioning +:description: pass:q[Enabling Cross Cluster Versioning allows XDCR to add metadata to each replicated document.] + +[abstract] +{description} + + +Enabling Cross Cluster Versioning for all buckets in the replication topology is a pre-requisite for some XDCR features. +The bucket property `enableCrossClusterVersioning` cannot be disabled once it is set to `true`. +Therefore, you must not enable `enableCrossClusterVersioning` casually. + +When you set the bucket property `enableCrossClusterVersioning` (ECCV) to `true`, for each document processed by XDCR, XDCR stores additional metadata for the document in the extended attributes. +This metadata is also called Hybrid Logical Vector (HLV), which is a set of Hybrid Logical Clock (HLC) information. + +[#hlv-data-maintained-in-xattr] +== Hybrid Logical Vector (HLV) data maintained in the extended attributes + +The new metadata, HLV, is stored as a system created extended attribute (xattrs) called `_vv` (`xattrs._vv`). The HLV metadata is also called Version Vectors. + +The HLV metadata takes up `109 + 40N` bytes of space per document, where N is the number of buckets mutating the document in the replication topology. + +NOTE: As long as your replication topology is constant, the size of the HLV metadata will grow to `109 + 40N` bytes and remain constant. +However, the HLV data accumulation occurs when the replication topology changes and the document copy goes through different clusters, for example, when being restored to a new cluster. + +To remove the accumulated or outdated HLV data, the HLV metadata is pruned periodically. +You can control the pruning frequency by setting the bucket property `versionPruningWindowHrs`. + +[#remove-hlv-metadata] +=== Remove HLV metadata + +After enabling, you cannot disable the bucket property `enableCrossClusterVersioning`. +You can, however, backup and restore the data to a bucket where `enableCrossClusterVersioning` is set to `false`, and remove the xattrs information added by XDCR using the option `cbbackupmgr restore --disable-hlv`. + +NOTE: To remove the on-going maintenance of HLV information, you must remove the xattrs information previously added to the documents. +To completely remove the effects of enabling HLV, only restoring the data to a bucket where `enableCrossClusterVersioning` is set to `false` is not enough. + +[#modify-enablecrossclusterversioning] +== Modify the bucket property enableCrossClusterVersioning + +To modify or enable the bucket property `enableCrossClusterVersioning`: + +* Using REST API, see xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing]. + +* From the UI, see xref:manage:manage-buckets/edit-bucket.adoc#edit-bucket-for-eccv[Edit a Bucket to Enable Cross Cluster Versioning]. + +NOTE: You cannot enable the bucket property `enableCrossClusterVersioning` while creating the bucket. + +[#version-pruning-window-hrs] +== versionPruningWindowHrs + +`versionPruningWindowHrs` is a bucket property, which controls the pruning frequency of the HLV metadata. +The default value of versionPruningWindowHrs is 720 hours (30 days), which means that any HLV data older than 720 hours is pruned to remove the outdated entries. + +NOTE: `versionPruningWindowHrs` must be set to the same value for all buckets in an XDCR replication topology. + +[#manage-version-pruning-window-hrs] +=== Manage the bucket property versionPruningWindowHrs + +For information about modifying the bucket property `versionPruningWindowHrs` through REST API, see xref:rest-api:rest-bucket-create.adoc#example-versionpruningwindowhrs-edit[Example: Specifying time value for versionPruningWindowHrs, when Editing]. + +[#features-requiring-crossclusterversioning-enabled] +== Features for which Cross Cluster Versioning must be enabled + +* *XDCR Active-Active with Sync Gateway*: The bi-directional, active-active replication with Sync Gateway 4.0* or a later version and XDCR requires enabling Cross Cluster Versioning. +For more information, including important limitations, see xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway]. ++ +NOTE: *To set up XDCR bi-directional replication with Sync Gateway (SGW), you need to have at least a Server 8.0.0 version and SGW 4.0.0 version. +However, Sync Gateway 4.0.0 is a future release version. + +* *XDCR Conflict Logging*: XDCR Conflict Logging requires enabling Cross Cluster Versioning. +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging Feature]. + +NOTE: Only the documents mutated after the `enableCrossClusterVersioning` property is enabled on both the source and target clusters, are considered for conflict logging. \ No newline at end of file diff --git a/modules/learn/pages/clusters-and-availability/xdcr-viewing-conflict-logs.adoc b/modules/learn/pages/clusters-and-availability/xdcr-viewing-conflict-logs.adoc new file mode 100644 index 0000000000..458b461a67 --- /dev/null +++ b/modules/learn/pages/clusters-and-availability/xdcr-viewing-conflict-logs.adoc @@ -0,0 +1,348 @@ += Viewing Conflict Logs +:description: pass:q[View and access conflict logs from the conflict collection.] +:page-toclevels: 3 + +[abstract] +{description} + +XDCR conflict logs are your application documents, so they're added in regular collections called xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#create-conflict-collection[Conflict Collection] designated by you. + +You can access, view, and process these documents programmatically, similar to your application documents, using the available utilities, features, and options. + +For example, you can use RBAC to manage access to conflict collections. +You can also use Couchbase services and tools such as Eventing and Kafka Connect to process conflict log documents. + +For information about the Conflict Record Document, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record[Conflict Record]. + +[#using-sdk-sqlpp-for-conflict-logs] +== Using Couchbase SDK with SQL++ for Conflict Logs + +The most common way to view these documents is by using the Couchbase SDK with SQL++. + +The following are a few examples. +In the examples, the conflict logs are stored in the collection `conflictlogs.bucket1.app1logs`. + +[#sqlpp-examples-clog] +=== SQL++ Examples + +For queries, you can create a custom index or a primary index. + +* Create a primary index as follows. ++ +[source,sqlpp] +---- +CREATE PRIMARY INDEX `#primary` ON `conflictlogs`.`bucket1`.`app1logs` +---- + +* Create a custom index as follows. ++ +You can use the `WHERE` clause with `SUBSTR(META().id,0,3) = “crd”` by making it an indexed field instead of `WHERE META().id LIKE "crd%"`. ++ +For example: ++ +[source,sqlpp] +---- +CREATE INDEX `idx_crd_timestamp_docId` ON `conflictlogs`.`bucket1`.`applogs`(SUBSTR(META().id,0,3), `timestamp`,`docId`) +---- ++ +NOTE: When creating custom indexes, `META().id` is already a part of the index key. +When using `LIKE` in the queries, underscore is a wildcard. +If you use underscore in comparison strings, ensure you use the escape character, so that it's used as an exact comparison instead of a wildcard comparison. +For example, `WHERE META().id LIKE “crd\\_%”`. + +*Query example 1:* List all `docId` that had conflicts between two time periods. + +[source,sqlpp] +---- +SELECT META().id, b.docId, b.timestamp +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META().id LIKE "crd%" and b.timestamp between "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +*Query example 2:* Number of conflicts logged between two time periods. + +[source,sqlpp] +---- +SELECT count(*) AS num_conflicts +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META().id LIKE "crd%" and b.timestamp between "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +*Query example 3:* Number of different `docIds` that had conflicts between two time periods. + +[source,sqlpp] +---- +SELECT count (distinct b.docId) as num_different_docIds +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META().id LIKE "crd%" and b.timestamp between "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +*Query example 4:* List all `docIds` and the number of times each `docId` had a logged conflict between two time periods. + +[source,sqlpp] +---- +SELECT b.docId AS DOCID, COUNT(1) AS num_times_seen +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META(b).id LIKE "crd%" AND b.timestamp BETWEEN "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +GROUP BY b.docId +ORDER BY num_times_seen DESC LIMIT 20 +---- + +The query result is as follows: + +---- +[ + { + "DOCID": "4d9d18e7-5fa2-4eae-b32b-60d3d369da94", + "num_times_seen": 2 + }, + { + "DOCID": "ae03ecde-203c-4a3b-a6d0-36b60f627c8a", + "num_times_seen": 1 + } +] +---- + +*Query example 5:* List all `docIds` and fields from the `src` and `tgt` documents that were in conflict, to view only the `docId` that had conflict with select fields from the `src` and `tgt` documents. + +[source,sqlpp] +---- +SELECT b.docId, b.srcDoc.id AS srcDocId, b.tgtDoc.id AS tgtDocId, +(SELECT b2.country, b2.note +FROM `conflictlogs`.`bucket1`.`app1logs` AS b2 USE KEYS b.srcDoc.id)[0] AS src_doc_fields, +(SELECT b3.country, b3.note +FROM `conflictlogs`.`bucket1`.`app1logs` AS b3 USE KEYS b.tgtDoc.id)[0] AS tgt_doc_fields +FROM `conflictlogs`.`bucket1`.`app1logs` AS b +WHERE META(b).id LIKE "crd%" AND b.timestamp BETWEEN "2025-06-04T18:15:16.363Z" AND "2025-06-09T07:26:40.326Z" +---- + +The query result is as follows: +---- +[ + { + "docId": "ae03ecde-203c-4a3b-a6d0-36b60f627c8a", + "srcDocId": "src_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "tgtDocId": "tgt_1749454000005461807_e2f416ed317beb4f16a3587656b3d3450e1bec893cabebd3b9bb857d63824da4", + "src_doc_fields": { + "country": "UK", + “note”: “need review” + }, + "tgt_doc_fields": { + "country": "CA", + “note”: “ok” + } + } +] +---- + +NOTE: As the conflict log documents won’t be replicated, if you need to replicate them, copy them into a different document. + +[#using-eventing-for-conflict-logs] +== Using Eventing for Conflict Logs + +The conflict logs and conflict documents are not replicated as XDCR’s special attribute (system xattrs called `_xdcr_conflict`) prevents replication: + +---- +"xattrs": { + "_xdcr_conflict": true + } +---- + +To force replicate the conflict logs and conflict documents, do the following: + +. Use Eventing to copy the document to a new collection along with other modifications. + +. Replicate the new document from the new collection. + +You cannot update system xattrs, however, following is an example of an Eventing function you can deploy to combine the `crd_`, `src_`, and `tgt_` documents into one new document that does not have the system attribute. +Here, it is assumed that the documents are small. +If the documents are large, then you can copy only the relevant fields from the `src` and `tgt` documents to a single document. + +[#eventing-example-clog] +=== Eventing example + +This is an Eventing function example to copy the conflict log documents into a single new document. + +Eventing bucket alias: `src` for `conflictlogs._default.bucket1logs` with read only access. + +Eventing bucket alias: `copy` for `conflictlogs._default.bucket1logs_copy` with read/write access. + +[source,javascript] +---- +function OnUpdate(doc, meta, xattrs) { + if (meta.id.startsWith("crd_") === true) + { + log("crd doc created/updated", meta.id); + + //get the conflict source and target document id's + var suffix = (meta.id).substring(4); + var srcId = "src_" + suffix; + var tgtId = "tgt_" + suffix; + + //read the conflicting source and target documents + var src_doc = src[srcId]; + var tgt_doc = src[tgtId]; + + //add the conflicting source and target documents to the crd doc + doc["src_doc"] = src_doc; + doc["tgt_doc"] = tgt_doc; + + //copy the resulting new crd doc to a different collection + copy[meta.id] = doc; + } +} +---- + +Resultant document example: + +NOTE: See xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#conflict-record-document-format[Conflict Record Document Format] to understand the document format. + +---- +{ + "timestamp": "2025-05-20T19:25:55.308Z", + "id": "crd_1747769155308581666_c0468a42632daabce649b1fb7e174a530946c04b1dde75218290c4885ad11669", + "docId": "ASHUNK100", + "replId": "1bc375a26b7f822ebe91288c58a6c614/bucket1/bucket1-PN8-oFADADCOe6S6A8YuIw==", + "srcDoc": { + "id": "src_1747769155308581666_c0468a42632daabce649b1fb7e174a530946c04b1dde75218290c4885ad11669", + "nodeId": "127.0.0.1", + "bucketUUID": "6cd3a35ba2d337d84d4fcd30c3a46fb9", + "clusterUUID": "d976db26a5ce512a243db65a3026073f", + "isDeleted": false, + "collection": "_default", + "scope": "_default", + "expiry": 0, + "flags": 33554438, + "cas": 1747769124557291500, + "revSeqno": 4, + "xattrs": { + "_vv": "{\"cvCas\":\"0x00002a1777524118\",\"src\":\"ntR6bcR9VHcjcPpQ5osEiw\",\"ver\":\"0x00002a1777524118\",\"pv\":[\"000018fe76524118@8DdrZBqHaACpmDeYJIv+kg\"]}", + "_sync": "", + "_mou": "" + } + }, + "tgtDoc": { + "id": "tgt_1747769155308581666_c0468a42632daabce649b1fb7e174a530946c04b1dde75218290c4885ad11669", + "nodeId": "34.213.22.102", + "bucketUUID": "fada12469f1027ecb078d2c19a2802fb", + "clusterUUID": "1bc375a26b7f822ebe91288c58a6c614", + "isDeleted": false, + "collection": "_default", + "scope": "_default", + "expiry": 0, + "flags": 33554438, + "cas": 1747769104359030800, + "revSeqno": 4, + "xattrs": { + "_vv": "{\"cvCas\":\"0x000018fe76524118\",\"src\":\"8DdrZBqHaACpmDeYJIv+kg\",\"ver\":\"0x000018fe76524118\"}", + "_sync": "", + "_mou": "" + } + }, + "src_doc": { + "author": "Unknown", + "create_date_utc": "2025-05-20T07:56:26", + "res_from": "WEST", + "res_name": "Harry West", + "res_note": "visitor -- HJV NOTE EAST", + "reserved": true, + "title": "Ashmole 782", + "type": "book", + "unix_time": "1747768801", + "update_date_utc": "2025-05-20T19:20:01" + }, + "tgt_doc": { + "author": "Unknown", + "create_date_utc": "2025-05-20T07:56:26", + "res_from": "WEST", + "res_name": "Harry West", + "res_note": "visitor -- HJV NOTE WEST", + "reserved": true, + "title": "Ashmole 782", + "type": "book", + "unix_time": "1747768801", + "update_date_utc": "2025-05-20T19:20:01" + } +} +---- + +[#using-kv-range-scan-clog] +== Using KV Range Scan + +To use only KV, use Couchbase SDK with KV APIs including xref:8.0@server:howtos/kv-operations.adoc#kv-range-scan[KV Range Scan]. + +For example, to get a list of all `crd` prefix documents (CRD documents) using the KV Range Scan xref:8.0@server:howtos/kv-operations.adoc#kv-range-scan-prefix[PrefixScan], use KV `collection.get` to get the conflicting `src` and `tgt` docs. +You can also use KV sub-document API `lookupIn` to retrieve only the selected fields. +After processing the documents, you can remove the conflict log documents using the KV `collection.remove`. + +For more information about KV Range Scan operations and data operations, see xref:8.0@server:howtos/kv-operations.adoc[Data Operations]. For more information about data operations on sub-documents, see xref:8.0@server:howtos/subdocument-operations.adoc[Sub-Document Operations]. + +[#xdcr-kv-range-scan-examples] +=== KV Range Scan examples + +The following example uses Couchbase Python SDK. + +[source,python] +---- +from datetime import timedelta +import json + +from couchbase.auth import PasswordAuthenticator +from couchbase.cluster import Cluster +from couchbase.collection import (GetOptions, RemoveOptions) +from couchbase.collection import DeltaValue, SignedInt64 +from couchbase.exceptions import ( + CouchbaseException, + PathNotFoundException, + SubdocPathMismatchException) +from couchbase.options import ScanOptions +from couchbase.kv_range_scan import PrefixScan +import couchbase.subdocument as SD + +# connection info +connection_string = "couchbase://localhost" +username = "username1" +password = "Some!password" + +# conflict logs collection location +bucket_name = "conflictlogs" +scope_name = "bucket1" +collection_name = "app1logs" + +cluster = Cluster( + connection_string, + authenticator=PasswordAuthenticator(username, password)); + +cluster.wait_until_ready(timedelta(seconds=5)) +cb = cluster.bucket(bucket_name); +logs_coll = cb.scope(scope_name).collection(collection_name) + +# Using KV PrefixScan to get list of crd doc ids +# Get crd document ids only +crd_list = logs_coll.scan(PrefixScan('crd'), ScanOptions(ids_only=True)) + +# For each crd document, get the docId, timestamp, srcDoc.id, tgtDoc.id +# Then, get the src and tgt docs from the conflict log collection +for crd in crd_list: + crdlookup = logs_coll.lookup_in(crd.id, + [ SD.get("docId"), + SD.get("timestamp"), + SD.get("srcDoc.id"), + SD.get("tgtDoc.id") ]) + + doc_id = crdlookup.content_as[str](0) + timestamp = crdlookup.content_as[str](1) + src_doc_id = crdlookup.content_as[str](2) + tgt_doc_id = crdlookup.content_as[str](3) + + # Get the src and tgt documents in conflict + src_doc = logs_coll.get(src_doc_id); + tgt_doc = logs_coll.get(tgt_doc_id); + + # Print the info as json + output = { "docId": doc_id, "log_timestamp": timestamp, + "src_doc": src_doc.content_as[dict], + "tgt_doc": tgt_doc.content_as[dict] } + print (json.dumps(output)) +---- + diff --git a/modules/manage/assets/images/manage-buckets/edit-bucket-with-eccv.png b/modules/manage/assets/images/manage-buckets/edit-bucket-with-eccv.png new file mode 100644 index 0000000000..200b2e3faa Binary files /dev/null and b/modules/manage/assets/images/manage-buckets/edit-bucket-with-eccv.png differ diff --git a/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-at-scope.png b/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-at-scope.png new file mode 100644 index 0000000000..15baa94f13 Binary files /dev/null and b/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-at-scope.png differ diff --git a/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-collection.png b/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-collection.png new file mode 100644 index 0000000000..1cf59abdce Binary files /dev/null and b/modules/manage/assets/images/manage-xdcr/customise-conflict-logging-collection.png differ diff --git a/modules/manage/assets/images/manage-xdcr/xdcr-add-replication-screen.png b/modules/manage/assets/images/manage-xdcr/xdcr-add-replication-screen.png index 82a4f0b688..e38a9212e8 100644 Binary files a/modules/manage/assets/images/manage-xdcr/xdcr-add-replication-screen.png and b/modules/manage/assets/images/manage-xdcr/xdcr-add-replication-screen.png differ diff --git a/modules/manage/assets/images/manage-xdcr/xdcr-outgoing-with-conflict-logging.png b/modules/manage/assets/images/manage-xdcr/xdcr-outgoing-with-conflict-logging.png new file mode 100644 index 0000000000..26c2c546a2 Binary files /dev/null and b/modules/manage/assets/images/manage-xdcr/xdcr-outgoing-with-conflict-logging.png differ diff --git a/modules/manage/pages/manage-buckets/create-bucket.adoc b/modules/manage/pages/manage-buckets/create-bucket.adoc index 7290327995..7049e2335c 100644 --- a/modules/manage/pages/manage-buckets/create-bucket.adoc +++ b/modules/manage/pages/manage-buckets/create-bucket.adoc @@ -175,6 +175,8 @@ For more information about flushing, see xref:manage-buckets/flush-bucket.adoc[F [#add-data-bucket-dialog-expanded] image::manage-buckets/addBucketWithMagmaOption.png[,400,align=center, alt="An image that displays the Add Data Bucket dialog, with a Couchbase Bucket Type and CouchStore Storage Backend selected. The Advanced bucket settings are expanded and to show the default selections for a Couchbase and Couchstore bucket."] +NOTE: XDCR Conflict Logging can be enabled only in the xref:manage:manage-buckets/edit-bucket.adoc[Edit a Bucket] mode. + [#memcached-bucket-settings] ==== Memcached Bucket Settings diff --git a/modules/manage/pages/manage-buckets/edit-bucket.adoc b/modules/manage/pages/manage-buckets/edit-bucket.adoc index ee736ae74c..480edbad6e 100644 --- a/modules/manage/pages/manage-buckets/edit-bucket.adoc +++ b/modules/manage/pages/manage-buckets/edit-bucket.adoc @@ -44,7 +44,25 @@ To display the user-interface for editing, left-click on the *Edit* button: image::manage-buckets/editBucketButton.png[,260,align=left] This displays the [.ui]*Edit Bucket Settings* dialog, which permits changes to be made to a subset of existing settings. -All the settings contained here are described in detail for the [.ui]*Add Data Bucket* dialog, on the page xref:manage-buckets/create-bucket.adoc[Create a Bucket] +All the settings contained here are described in detail for the [.ui]*Add Data Bucket* dialog, on the page xref:manage-buckets/create-bucket.adoc[Create a Bucket]. + +[#edit-bucket-for-eccv] +=== Edit a Bucket to Enable Cross Cluster Versioning + +[.ui]*Edit Bucket Settings* dialog displays the additional setting, *Enable Cross Cluster Versioning*. Use this setting to enable the xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning] bucket property. + +image::manage-buckets/edit-bucket-with-eccv.png[,399,align=center] + +When you enable the bucket setting [.ui]*Enable Cross Cluster Versioning*, for each document processed by XDCR, XDCR stores additional metadata for the document in the extended attributes. +This metadata is also called xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#hlv-data-maintained-in-xattr[Hybrid Logical Vector (HLV)]. + +NOTE: Once enabled, the Enable Cross Cluster Versioning bucket setting cannot be disabled. + +To use the following features, enable the bucket setting [.ui]*Enable Cross Cluster Versioning* on all buckets that are a part of the XDCR topology: + +* xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging]. + +* xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway]. == Making Changes diff --git a/modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc b/modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc index 75f93f1dcf..68a83a328e 100644 --- a/modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc +++ b/modules/manage/pages/manage-xdcr/create-xdcr-replication.adoc @@ -90,6 +90,8 @@ For more information , see xref:learn:data/scopes-and-collections.adoc#default-s An account of *Advanced Settings* is provided in xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-advanced-settings-pointer[Advanced Replication Settings with the UI], below. + The practical steps required for establishing filters are explained in xref:manage:manage-xdcr/filter-xdcr-replication.adoc[Filter a Replication]. ++ +Information about xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-ui-settings-for-conflict-logging[Replication Settings for XDCR Conflict Logging] is provided in a later section. . Enter appropriate information into the upper fields of the *Add Replication* screen. Specify `10.144.210.102` as the target cluster, and `travel-sample` as both source and target bucket. @@ -113,6 +115,10 @@ Note that by left-clicking on the row for the replication, additional controls c image::manage-xdcr/xdcr-outgoing-replications-with-replication-opened.png[,800,align=left] +When conflict logging is enabled, Outgoing Replications panel displays the Conflict Logging Rules panel, which shows the conflict collection mapping that you have chosen. Also, the bucket indicates Conflict Logging Enabled. + +image::manage-xdcr/xdcr-outgoing-with-conflict-logging.png[,800,align=left] + Use of the *Pause* control is described in xref:manage:manage-xdcr/pause-xdcr-replication.adoc[Pause a Replication]; use of the *Delete* control in xref:manage:manage-xdcr/delete-xdcr-replication.adoc[Delete a Replication]; and use of the *Edit* control in xref:manage:manage-xdcr/filter-xdcr-replication.adoc#editing-filters[Editing Filters]. [#monitor-current-replications] @@ -137,6 +143,38 @@ image::manage-xdcr/xdcr-advanced-settings.png[,400,align=left] The values displayed in the fields are defaults, which can be modified interactively, and saved: this may help in achieving optimal replication-performance. For details on the significance of each field, see the xref:xdcr-reference:xdcr-reference-intro.adoc[XDCR Reference]. +[#xdcr-ui-settings-for-conflict-logging] +=== Replication Settings for XDCR Conflict Logging + +The *XDCR Add Replication* page displays the following additional settings, which are specific to logging conflicts during an XDCR replication: + +* *Specify collections for storing conflict logs and documents*: Use this setting to enable choosing a conflict collection for storing conflict logs and documents. +This setting allows you to specify the bucket, scope, and collection where conflict logs and documents will be stored during XDCR conflict logging process. Conflict Logging Rules panel displays the mapping of conflict collection that you have chosen. + +* *Enable Conflict Logging*: Use this setting to enable or disable conflict logging for the replication. When enabled, XDCR starts logging conflicts in a specified bucket, scope, and collection. + +* *Default Conflict Log Collection*: Use this setting to specify the default bucket, scope, and collection for conflict logging. If a custom collection is not specified, the default collection is used. ++ +NOTE: By default, conflicts are logged in the bucket, scope, and collection of the *Default Conflict Log Collection*. Use *Customise Conflict Logging at scope level* and *Customise Conflict Logging at collection level* to override the default settings. + +* *Customise Conflict Logging at scope level*: Use this setting to specify the scope for a custom conflict collection. ++ +image::manage-xdcr/customise-conflict-logging-at-scope.png[,800,align=left] + +* *Customise Conflict Logging at collection level*: Use this setting to specify a custom collection to store conflict logs. You can choose a custom conflict collection that overrides the default conflict collection. Choose one of the following collection types: +** *Default collection*: To set the collection to the default collection of the bucket. +** *Parent collection*: To set the collection to that of the scope. +** *Custom collection*: To set the collection to a custom collection that you specify. +** *Do not log*: To disable conflict logging for this collection. ++ +image::manage-xdcr/customise-conflict-logging-collection.png[,800,align=left] + +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Configuring the conflictLogging Settings]. + +For information about XDCR Conflict Logging, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging]. + +For information about creating collections (including conflict collections), see xref:manage:manage-scopes-and-collections/manage-scopes-and-collections.adoc[Manage Scopes and Collections]. + [#error-notifications] === Error Notifications @@ -214,6 +252,19 @@ If successful, this provides the following response: For more information, see xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication]. For information on REST-driven configuration of the xref:manage:manage-xdcr/create-xdcr-replication.adoc#xdcr-advanced-settings-pointer[Advanced Settings] described above, see xref:rest-api:rest-xdcr-adv-settings.adoc[Managing Advanced Settings]. +[#create-an-xdcr-replication-with-mobile-as-active] +== Create an XDCR Replication with mobile=Active + +To create or update an XDCR replication with `mobile=Active`, do the following: + +* Create an XDCR replication with `mobile=Active` or update an existing replication. For information about _creating_ (new) an XDCR replication with `mobile=Active`, see xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc#xdcr-active-active-sgw-greenfield-deployment[Greenfield deployment], and for information about _updating_ an existing replication with `mobile=Active`, see xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc#xdcr-active-active-sgw-upgrade[Upgrade an existing setup]. +* Create or update an XDCR replication with `mobile=Active` option using the REST API, starting from Server 7.6.6 version. See xref:rest-api:rest-xdcr-create-replication.adoc[Creating a Replication]. +* Create or update a XDCR replication with `mobile=Active` option from the UI, starting from Server 7.6.6 version. See xref:manage:manage-xdcr/create-xdcr-replication.adoc#create-an-xdcr-replication-with-the-ui[Create an XDCR Replication with the UI]. + +The pre-requisite to use `mobile=Active` is to set the bucket property `enableCrossClusterVersioning`. For more information about the bucket property `enableCrossClusterVersioning`, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + +To enable the bucket property `enableCrossClusterVersioning` using REST API, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#modify-enablecrossclusterversioning[Modify the bucket property enableCrossClusterVersioning] or xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing]. + [#next-xdcr-steps-after-create-replication] == Next Steps diff --git a/modules/rest-api/pages/rest-bucket-create.adoc b/modules/rest-api/pages/rest-bucket-create.adoc index fbc196b313..8de8467b28 100644 --- a/modules/rest-api/pages/rest-bucket-create.adoc +++ b/modules/rest-api/pages/rest-bucket-create.adoc @@ -78,6 +78,7 @@ curl -X POST -u : -d allowedTimePeriod[toHour]= -d allowedTimePeriod[toMinute]= -d allowedTimePeriod[abortOutside]=[ true | false ] + -d versionPruningWindowHrs= ---- All parameters are described in the following subsections. @@ -1090,6 +1091,52 @@ This parameter is ignored if `autoCompactionDefined` is `false` (which is its de See the examples provided above, in xref:rest-api:rest-bucket-create.adoc#example-databasefragmentationthresholdpercentage-create[Example: Specifying a Data Fragmentation Threshold as a Percentage, when Creating] and xref:rest-api:rest-bucket-create.adoc#example-databasefragmentationthresholdpercentage-edit[Example: Specifying a Data Fragmentation Threshold as a Percentage, when Editing]. + +[#enablecrossclusterversioning] +=== enableCrossClusterVersioning + +Enabling Cross Cluster Versioning is a pre-requisite to a few XDCR features. +The bucket property `enableCrossClusterVersioning` can only be set to true after a bucket has been created. +When enabled, for each document processed by XDCR, XDCR stores additional metadata, called the Hybrid Logical Vector (HLV), in the document extended attributes (xattrs). +For more information, see xref:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + +See the example provided in xref:rest-api:rest-bucket-create.adoc#example-enablecrossclusterversioning-edit[Example: Turning on enableCrossClusterVersioning, when Editing] + +CAUTION: The default value is `false`. Do not change the value of this property unless instructed by a feature configuration. Once enabled, you cannot turn off the `enableCrossClusterVersioning` property. The only way for you to undo setting this value to `true` is to backup your data, create a new bucket, and restore the data, using the option `cbbackupmgr restore --disable-hlv` to remove the HLV info in the xattrs. + +[#example-enablecrossclusterversioning-edit] +==== Example: Turning on enableCrossClusterVersioning, when Editing + +The following example modifies the value of the bucket property `enableCrossClusterVersioning` to `true`. + +---- +curl -v -X POST http://localhost:8091/pools/default/buckets/testBucket \ +-u Administrator:password \ +-d enableCrossClusterVersioning=true +---- + +[#version-pruning-window-hrs-property] +=== versionPruningWindowHrs + +Controls the pruning frequency of the Hybrid Logical Vector (HLV) metadata. +The default value of versionPruningWindowHrs is 720 hours (30 days), which means that any HLV data older than 720 hours is pruned to remove the outdated entries. +For more information, see xref:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[versionPruningWindowHrs] in xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#version-pruning-window-hrs[XDCR enableCrossClusterVersioning]. + +See the example provided in xref:rest-api:rest-bucket-create.adoc#example-versionpruningwindowhrs-edit[Example: Specifying time value for versionPruningWindowHrs, when Editing] + +NOTE: `versionPruningWindowHrs` must be set to the same value for all buckets in an XDCR replication topology. + +[#example-versionpruningwindowhrs-edit] +==== Example: Specifying time value for versionPruningWindowHrs, when Editing + +The following example sets the time duration value for the bucket property `versionPruningWindowHrs`. + +---- +curl -v -X POST http://localhost:8091/pools/default/buckets/testBucket \ +-u Administrator:password \ +-d versionPruningWindowHrs=120 +---- + == Responses If bucket-creation is successful, HTTP response `202 Accepted` is returned, with empty content. @@ -1099,15 +1146,6 @@ If the bucket cannot created due to a missing or incorrect parameter, a `400` re If the URL is incorrectly specified a `404 (Object Not Found)` error is returned. Failure to authenticate gives `401 Unauthorized`. -[#notes] -== Notes - -Couchbase Server 7.6.2 added a property named `enableCrossClusterVersioning` to support a future feature. -Do not change the value of this property. - -CAUTION: Once enabled, you cannot turn off the `enableCrossClusterVersioning` property. -The only way for you to undo setting this value to `true` is to backup your data, create a new bucket, and reload the data into it. - == See Also A conceptual description of buckets is provided in xref:learn:buckets-memory-and-storage/buckets.adoc[Buckets]. diff --git a/modules/rest-api/pages/rest-xdcr-adv-settings.adoc b/modules/rest-api/pages/rest-xdcr-adv-settings.adoc index 45e1c8fa3b..4d2eb38ab5 100644 --- a/modules/rest-api/pages/rest-xdcr-adv-settings.adoc +++ b/modules/rest-api/pages/rest-xdcr-adv-settings.adoc @@ -94,30 +94,42 @@ If successful, the call returns an object similar to the following: ---- { + "casDriftThresholdSecs": 100, + "conflictLogging": null, "checkpointInterval": 600, "ckptSvcCacheEnabled": true, + "collectionsOSOMode": true, "compressionType": "Auto", + "dcpEnablePurgeRollback": false, "desiredLatency": 50, "docBatchSizeKb": 2048, "failureRestartInterval": 10, + "filterBinary": false, "filterBypassExpiry": false, "filterBypassUncommittedTxn": false, "filterDeletion": false, "filterExpiration": false, + "genericServicesLogLevel": { + < ... diagnostic items cut out due to length ... > + }, "goGC": 100, "goMaxProcs": 4, - "hlvPruningWindowSec": 259200, "jsFunctionTimeoutMs": 20000, "logLevel": "Info", "mergeFunctionMapping": {}, + "mobile": "Off", "networkUsageLimit": 0, "optimisticReplicationThreshold": 256, + "preCheckCasDriftThresholdHours": 8760, "preReplicateVBMasterCheck": true, "priority": "High", "replicateCkptIntervalMin": 20, + "retryOnErrExceptAuthErrMaxWaitSec": 360, "retryOnRemoteAuthErr": true, - "retryOnRemoteAuthErrMaxWaitSec": 3600, + "retryOnRemoteAuthErrMaxWaitSec": 360, + "skipReplSpecAutoGc": false, "sourceNozzlePerNode": 2, + "targetTopologyLogFrequency": 1800, "statsInterval": 1000, "targetNozzlePerNode": 2, "workerBatchSize": 500 @@ -133,6 +145,8 @@ Use the following call, which is explained in detail in xref:rest-api:rest-get-c curl -v -X GET http://localhost:8091/pools/default/tasks -u Administrator:password | jq '.' ---- +*Example*: Retrieve ongoing cluster tasks. + If successful, the call returns an array of ongoing cluster tasks, similar to the following: ---- @@ -195,6 +209,8 @@ If successful, the call returns an array of ongoing cluster tasks, similar to th ] ---- +*Example*: Retrieve second member of the array. + A `settings_URI` is provided for each of the ongoing replications, within the second member of the array. The `settings_URI` can be used to retrieve information on the corresponding replication. @@ -209,6 +225,8 @@ If the call is successful, an object containing the settings for the specified r ---- { + "casDriftThresholdSecs": 100, + "conflictLogging": null, "checkpointInterval": 600, "ckptSvcCacheEnabled": true, "colMappingRules": {}, @@ -217,34 +235,77 @@ If the call is successful, an object containing the settings for the specified r "collectionsMirroringMode": false, "collectionsOSOMode": true, "compressionType": "Auto", + "dcpEnablePurgeRollback": false, "desiredLatency": 50, "docBatchSizeKb": 2048, "failureRestartInterval": 10, + "filterBinary": false, "filterBypassExpiry": false, "filterBypassUncommittedTxn": false, "filterDeletion": false, "filterExpiration": false, "filterExpression": "", - "hlvPruningWindowSec": 259200, "jsFunctionTimeoutMs": 20000, "logLevel": "Info", "mergeFunctionMapping": {}, + "mobile": "Off", "networkUsageLimit": 0, "optimisticReplicationThreshold": 256, "pauseRequested": false, + "preCheckCasDriftThresholdHours": 8760, "preReplicateVBMasterCheck": true, "priority": "High", "replicateCkptIntervalMin": 20, + "retryOnErrExceptAuthErrMaxWaitSec": 360, "retryOnRemoteAuthErr": true, - "retryOnRemoteAuthErrMaxWaitSec": 3600, - "sourceNozzlePerNode": 2, + "retryOnRemoteAuthErrMaxWaitSec": 360, + "skipReplSpecAutoGc": false, "sourceNozzlePerNode": 2, "statsInterval": 1000, "targetNozzlePerNode": 2, + "targetTopologyLogFrequency": 1800, "type": "xmem", "workerBatchSize": 500 } ---- +[#rest-api-conflict-logging-replication-tunables] +*Example*: Retrieve the tunable settings for Conflict Logging in a replication. + +After you create a replication with conflict logging enabled, to view the replication settings, retrieve the replication's `settings_URI`. Here's an example. + +---- +curl -X GET -s -u Administrator:'password' http://localhost:8091/settings/replications/f6f3a51e988f305b16433237d02bec7d%2Fbucket3%2Fbucket3 | jq `.` +---- + +If the call is successful, an object is returned, which contains the replication settings for fine tuning the Conflict Logging feature: + +NOTE: For more information about fine tuning the replication settings in the Conflict Logging feature, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#tunable-replication-settings[Tunable Replication Settings]. + +---- +{ + "cLogErrorTimeWindowMs": 120000, + "cLogMaxErrorCount": 10, + "cLogNetworkRetryCount": 5, + "cLogNetworkRetryIntervalMs": 2000, + "cLogPoolGetTimeoutMs": 5000, + "cLogQueueCapacity": 6000, + "cLogReattemptDurationMs": 600000, + "cLogSetTimeoutMs": 5000, + "cLogWorkerCount": 20, + "casDriftThresholdSecs": 3900, + "checkpointInterval": 600, + "ckptSvcCacheEnabled": true, + "colMappingRules": {}, + "collectionsExplicitMapping": false, + "collectionsMigrationMode": false, + "collectionsMirroringMode": false, + "collectionsOSOMode": true, + "compressionType": "Auto", + "conflictLogging": {}, +... +} +---- + === Change a Setting for a Specific Replication The following example modifies the value of `checkpointInterval`, for a specific replication: @@ -259,6 +320,8 @@ If successful, the call returns an object containing all current settings for th ---- { + "casDriftThresholdSecs": 100, + "conflictLogging": null, "checkpointInterval": 700, "ckptSvcCacheEnabled": true, "colMappingRules": {}, @@ -267,34 +330,51 @@ If successful, the call returns an object containing all current settings for th "collectionsMirroringMode": false, "collectionsOSOMode": true, "compressionType": "Auto", + "dcpEnablePurgeRollback": false, "desiredLatency": 50, "docBatchSizeKb": 2048, "failureRestartInterval": 10, + "filterBinary": false, "filterBypassExpiry": false, "filterBypassUncommittedTxn": false, "filterDeletion": false, "filterExpiration": false, "filterExpression": "", - "hlvPruningWindowSec": 259200, "jsFunctionTimeoutMs": 20000, "logLevel": "Info", "mergeFunctionMapping": {}, + "mobile": "Off", "networkUsageLimit": 0, "optimisticReplicationThreshold": 256, "pauseRequested": false, + "preCheckCasDriftThresholdHours": 8760, "preReplicateVBMasterCheck": true, "priority": "High", "replicateCkptIntervalMin": 20, + "retryOnErrExceptAuthErrMaxWaitSec": 360, "retryOnRemoteAuthErr": true, - "retryOnRemoteAuthErrMaxWaitSec": 3600, + "retryOnRemoteAuthErrMaxWaitSec": 360, + "skipReplSpecAutoGc": false, "sourceNozzlePerNode": 2, "statsInterval": 1000, "targetNozzlePerNode": 2, + "targetTopologyLogFrequency": 1800, "type": "xmem", "workerBatchSize": 500 } ---- +[#change-existing-replication-with-mobile-active] +=== Change Settings for an Existing Replication to Set mobile=Active + +The following example modifies the value of `mobile` to `Active` for an existing replication: + +---- +curl -X POST -u Administrator:password http://localhost:8091/ -d mobile=Active | jq '.' +---- + +For information about _XDCR with Sync Gateway mobile clusters in a bi-directional, active-active replication_, see xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway]. + [#xdcr-advanced-settings-rest] == List of Advanced Settings @@ -345,6 +425,17 @@ For information, see the xref:xdcr-reference:xdcr-advanced-settings.adoc#table-o This setting can be established and retrieved either for an individual replication or globally. +| `conflictLogging` +| JSON Object +| Configuration settings for logging conflicts. + +This configuration setting defines objects/parameters and options used to control how conflicts are logged during an XDCR replication. + +Within the JSON Object, `conflictLogging` setting is enabled by default (`disabled=false`). + +This setting uses a JSON object named `loggingRules` to define its values. +For more information, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Configuring the conflictLogging Setting]. + | `desiredLatency` | Integer | Specifies the amount of time in milliseconds within which a high-priority replication's currently remaining changes should be reduced in number to zero, by ongoing transmission of the changes from source to target bucket. @@ -455,6 +546,11 @@ This setting can be established and retrieved either for an individual replicati | JSON Object | Each of the key-value pairs in the JSON object should map a collection-specifier (in the form `scope.collection`) to the name of a defined merge-function. +| `mobile` +| Active or Off +| Default: `Off`. +When set to `Active`, enables the setting _XDCR Active-Active with Sync Gateway 4.0+_ on the clusters of both sides of the replication. The default value `Off` indicates that the replication setup supports either _XDCR Active-Passive with Sync Gateway_ or _XDCR Active-Active without Sync Gateway_. For more information, see xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway]. + | `networkUsageLimit` | Integer | Default: 0. @@ -471,6 +567,8 @@ Documents with sizes less than this threshold (in bytes) will be replicated opti This setting can be established and retrieved either for an individual replication or globally. +XDCR optimistic replication is applicable only when the bucket property `enableCrossClusterVersioning` is disabled. For information about the property `enableCrossClusterVersioning`, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#version-pruning-window-hrs[XDCR enableCrossClusterVersioning]. + | `pauseRequested` | Boolean (true or false) | Default: false. diff --git a/modules/rest-api/pages/rest-xdcr-create-replication.adoc b/modules/rest-api/pages/rest-xdcr-create-replication.adoc index b386c64ab2..cd11d72d27 100644 --- a/modules/rest-api/pages/rest-xdcr-create-replication.adoc +++ b/modules/rest-api/pages/rest-xdcr-create-replication.adoc @@ -28,6 +28,7 @@ curl -v -X POST -u [admin]:[password] -d fromBucket=[bucket-name] -d toCluster=[cluster-name] -d toBucket=[bucket-name] + -d conflictLogging [JSON-Document] -d collectionsExplicitMapping [true | false] -d collectionsMigrationMode [true | false] -d colMappingRules [JSON-Document] @@ -52,6 +53,7 @@ curl -v -X POST -u [admin]:[password] -d statsInterval=[Integer] -d logLevel=[String] -d networkUsageLimit=[Integer] + -d mobile=[Off | Active] ---- The `type` value must be `xmem`; which is sometimes referred to as *Version 2*, and corresponds to the _Memcached Binary_ protocol, used in XDCR communications. @@ -59,6 +61,13 @@ The `type` value must be `xmem`; which is sometimes referred to as *Version 2*, The `replicationType` value is always `continuous`. This value must be specified. +The `conflictLogging` setting enables or disables logging conflicts for the replication, and has a JSON Object `loggingRules` as its value. +Within the JSON Object, `conflictLogging` setting is enabled by default (`disabled=false`). You can specify the target bucket, scope, and collection for logging conflicts, and source collections of the replication. +This helps to track and resolve document conflicts manually during replication. +`loggingRules` is a JSON object that defines the custom conflict logging rules for specific collections. +An empty JSON object value must be specified to disable the `conflictLogging` (Conflict Logging feature). +For configuration information and examples, see xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc#configure-conflictlogging-settings[Configuring the conflictLogging Settings]. + If the optional parameters `collectionsExplicitMapping`, `colMappingRules`, and `collectionsMigrationMode` are all _unspecified_, replication occurs between the specified `fromBucket` and `toBucket` according to whatever _implicit mappings_ can be determined at bucket-level: that is, if a collection in the source bucket has the same _keyspace_ as a collection in the target bucket (i.e., the name of a collection and the name of the scope within which it is located form the identical _scope-name.collection-name_ pattern, on both source and target), data is replicated between those collections; with the optional application of a filter expression. At a minimum, this ensures that data is replicated between the default collections. @@ -122,6 +131,11 @@ The value can be `High`, `Medium`, or `Low`. The default value is `High`. For information, see xref:learn:clusters-and-availability/xdcr-overview.adoc#xdcr-priority[XDCR Priority]. +Use the `mobile=[Off | Active]` flag to enable the setting _XDCR Active-Active with Sync Gateway 4.0+_ by changing the value to `Active` on the clusters of both sides of the replication. The default value is `Off` , which indicates that the setup supports either _XDCR Active-Passive with Sync Gateway_ or _XDCR Active-Active without Sync Gateway_. + +[NOTE] +To enable the setting `mobile=[Off | Active]`, ensure you have enabled the property `enableCrossClusterVersioning` on all the participating buckets, which is a prerequisite. For information about the bucket property `enableCrossClusterVersioning`, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc[XDCR enableCrossClusterVersioning]. + For information on all other flags, see xref:rest-api:rest-xdcr-adv-settings.adoc[Managing Advanced XDCR Settings]. [#responses] diff --git a/modules/xdcr-reference/pages/xdcr-advanced-settings.adoc b/modules/xdcr-reference/pages/xdcr-advanced-settings.adoc index a641283ca3..02a5d39829 100644 --- a/modules/xdcr-reference/pages/xdcr-advanced-settings.adoc +++ b/modules/xdcr-reference/pages/xdcr-advanced-settings.adoc @@ -95,6 +95,8 @@ For documents that are smaller, XDCR replicates the document to the target witho Note that a low setting risks increased latency, due to a higher number or metadata fetches; but may also reduce the number of required replications (due to source and target having identical copies of the document). A high setting lowers latency during replication, since the number of metadata fetches is reduced; but may also raise the replication-rate excessively, overwhelming either network or target cluster. +XDCR optimistic replication is applicable only when the `enableCrossClusterVersioning` property is disabled. For information about the property `enableCrossClusterVersioning`, see xref:learn:clusters-and-availability/xdcr-enable-crossclusterversioning.adoc#version-pruning-window-hrs[XDCR enableCrossClusterVersioning]. + | XDCR Statistics Collection Interval | Specifies, in seconds, how frequently XDCR Statistics are updated.