WPB-20728 background job for user group to channel association

changelog.d/0-release-notes/WPB-20728

@@ -0,0 +1,15 @@
+Background-worker configuration: required values when supplying your own Helm values
+
+Add the following fields under `background-worker`:
+
+- `config.domain`
+- `config.postgresql`
+- `config.cassandraBrig`
+- `config.cassandraGalley`
+- `secrets.pgPassword`
+
+Notes
+- `config.cassandra` (for gundeck) already exists; no change needed.
+- `config.backgroundJobs` and `config.postgresqlPool` have defaults; override only if needed.
+- `config.postgresMigration.conversation` defaults to `postgresql`; change only if migrating conversations to PostgreSQL.
+- `config.brig` and `config.gundeck` endpoints have in-cluster defaults; override only if your service DNS/ports differ.

changelog.d/2-features/WPB-20728

@@ -0,0 +1 @@
+Add users of user groups to a channel in asynchronous background worker job

charts/background-worker/README.md

@@ -1,5 +1,21 @@
-Note that background-worker depends on some provisioned storage, namely:
+Note that background-worker depends on some provisioned storage/services, namely:
 
 - rabbitmq
+- postgresql
+- cassandra (three clusters)
+
+PostgreSQL configuration
+- Set connection parameters under `config.postgresql` (libpq keywords: `host`, `port`, `user`, `dbname`, etc.).
+- Provide the password via `secrets.pgPassword`; it is mounted at `/etc/wire/background-worker/secrets/pgPassword` and referenced from the configmap.
+
+Cassandra configuration
+- Background-worker connects to three Cassandra clusters:
+  - `config.cassandra` (keyspace: `gundeck`) for the dead user notification watcher.
+  - `config.cassandraBrig` (keyspace: `brig`) for the user store.
+  - `config.cassandraGalley` (keyspace: `galley`) for conversation-related data access.
+- TLS may be configured via either a reference (`tlsCaSecretRef`) or inline CA (`tlsCa`) for each cluster. Secrets mount under:
+  - `/etc/wire/background-worker/cassandra-gundeck`
+  - `/etc/wire/background-worker/cassandra-brig`
+  - `/etc/wire/background-worker/cassandra-galley`
 
 These are dealt with independently from this chart.

charts/background-worker/templates/_helpers.tpl

@@ -8,18 +8,38 @@
   {{- (semverCompare ">= 1.24-0" (include "kubeVersion" .)) -}}
 {{- end -}}
 
-{{- define "useCassandraTLS" -}}
+{{- define "useGundeckCassandraTLS" -}}
 {{ or (hasKey .cassandra "tlsCa") (hasKey .cassandra "tlsCaSecretRef") }}
 {{- end -}}
 
-{{/* Return a Dict of TLS CA secret name and key
-This is used to switch between provided secret (e.g. by cert-manager) and
-created one (in case the CA is provided as PEM string.)
-*/}}
-{{- define "tlsSecretRef" -}}
+{{- define "useBrigCassandraTLS" -}}
+{{ or (hasKey .cassandraBrig "tlsCa") (hasKey .cassandraBrig "tlsCaSecretRef") }}
+{{- end -}}
+
+{{- define "useGalleyCassandraTLS" -}}
+{{ or (hasKey .cassandraGalley "tlsCa") (hasKey .cassandraGalley "tlsCaSecretRef") }}
+{{- end -}}
+
+{{- define "gundeckTlsSecretRef" -}}
 {{- if .cassandra.tlsCaSecretRef -}}
 {{ .cassandra.tlsCaSecretRef | toYaml }}
 {{- else }}
-{{- dict "name" "background-worker-cassandra" "key" "ca.pem" | toYaml -}}
+{{- dict "name" "background-worker-cassandra-gundeck" "key" "ca.pem" | toYaml -}}
+{{- end -}}
+{{- end -}}
+
+{{- define "brigTlsSecretRef" -}}
+{{- if .cassandraBrig.tlsCaSecretRef -}}
+{{ .cassandraBrig.tlsCaSecretRef | toYaml }}
+{{- else }}
+{{- dict "name" "background-worker-cassandra-brig" "key" "ca.pem" | toYaml -}}
+{{- end -}}
+{{- end -}}
+
+{{- define "galleyTlsSecretRef" -}}
+{{- if and .cassandraGalley .cassandraGalley.tlsCaSecretRef -}}
+{{ .cassandraGalley.tlsCaSecretRef | toYaml }}
+{{- else }}
+{{- dict "name" "background-worker-cassandra-galley" "key" "ca.pem" | toYaml -}}
 {{- end -}}
 {{- end -}}

charts/background-worker/templates/cassandra-secret.yaml

@@ -1,9 +1,9 @@
-{{/* Secret for the provided Cassandra TLS CA. */}}
+{{/* Secrets for provided Cassandra TLS CAs */}}
 {{- if not (empty .Values.config.cassandra.tlsCa) }}
 apiVersion: v1
 kind: Secret
 metadata:
-  name: background-worker-cassandra
+  name: background-worker-cassandra-gundeck
   labels:
     app: background-worker
     chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
@@ -13,3 +13,33 @@ type: Opaque
 data:
   ca.pem: {{ .Values.config.cassandra.tlsCa | b64enc | quote }}
 {{- end }}
+{{- if not (empty .Values.config.cassandraBrig.tlsCa) }}
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: background-worker-cassandra-brig
+  labels:
+    app: background-worker
+    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    release: "{{ .Release.Name }}"
+    heritage: "{{ .Release.Service }}"
+type: Opaque
+data:
+  ca.pem: {{ .Values.config.cassandraBrig.tlsCa | b64enc | quote }}
+{{- end }}
+{{- if and .Values.config.cassandraGalley (not (empty .Values.config.cassandraGalley.tlsCa)) }}
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: background-worker-cassandra-galley
+  labels:
+    app: background-worker
+    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    release: "{{ .Release.Name }}"
+    heritage: "{{ .Release.Service }}"
+type: Opaque
+data:
+  ca.pem: {{ .Values.config.cassandraGalley.tlsCa | b64enc | quote }}
+{{- end }}

charts/background-worker/templates/configmap.yaml

@@ -21,15 +21,43 @@ data:
       host: federator
       port: 8080
 
+    brig:
+      host: brig
+      port: 8080
+
+    gundeck:
+      host: gundeck
+      port: 8080
+
     cassandra:
       endpoint:
         host: {{ .cassandra.host }}
         port: 9042
       keyspace: gundeck
-      {{- if eq (include "useCassandraTLS" .) "true" }}
-      tlsCa: /etc/wire/background-worker/cassandra/{{- (include "tlsSecretRef" . | fromYaml).key }}
+      {{- if eq (include "useGundeckCassandraTLS" .) "true" }}
+      tlsCa: /etc/wire/background-worker/cassandra-gundeck/{{- (include "gundeckTlsSecretRef" . | fromYaml).key }}
+      {{- end }}
+
+    cassandraBrig:
+      endpoint:
+        host: {{ .cassandraBrig.host }}
+        port: 9042
+      keyspace: brig
+      {{- if eq (include "useBrigCassandraTLS" .) "true" }}
+      tlsCa: /etc/wire/background-worker/cassandra-brig/{{- (include "brigTlsSecretRef" . | fromYaml).key }}
       {{- end }}
 
+    cassandraGalley:
+      endpoint:
+        host: {{ .cassandraGalley.host }}
+        port: 9042
+      keyspace: galley
+      {{- if eq (include "useGalleyCassandraTLS" .) "true" }}
+      tlsCa: /etc/wire/background-worker/cassandra-galley/{{- (include "galleyTlsSecretRef" . | fromYaml).key }}
+      {{- end }}
+
+    domain: {{ .Values.domain }}
+
     {{- with .rabbitmq }}
     rabbitmq:
       host: {{ .host }}
@@ -48,4 +76,18 @@ data:
 
     backendNotificationPusher:
 {{toYaml .backendNotificationPusher | indent 6 }}
+    {{- with .backgroundJobs }}
+    backgroundJobs:
+{{ toYaml . | indent 6 }}
+    {{- end }}
+    postgresql:
+{{ toYaml .postgresql | indent 6 }}
+    {{- if hasKey $.Values.secrets "pgPassword" }}
+    postgresqlPassword: /etc/wire/background-worker/secrets/pgPassword
+    {{- end }}
+    postgresqlPool:
+{{ toYaml .postgresqlPool | nindent 6 }}
+    {{- if .postgresMigration }}
+    postgresMigration: {{- toYaml .postgresMigration | nindent 6 }}
+    {{- end }}
   {{- end }}

charts/background-worker/templates/deployment.yaml

@@ -39,10 +39,20 @@ spec:
         - name: "background-worker-secrets"
           secret:
             secretName: "background-worker"
-        {{- if eq (include "useCassandraTLS" .Values.config) "true" }}
-        - name: "background-worker-cassandra"
+        {{- if eq (include "useGundeckCassandraTLS" .Values.config) "true" }}
+        - name: "background-worker-cassandra-gundeck"
           secret:
-            secretName: {{ (include "tlsSecretRef" .Values.config | fromYaml).name }}
+            secretName: {{ (include "gundeckTlsSecretRef" .Values.config | fromYaml).name }}
+        {{- end }}
+        {{- if eq (include "useBrigCassandraTLS" .Values.config) "true" }}
+        - name: "background-worker-cassandra-brig"
+          secret:
+            secretName: {{ (include "brigTlsSecretRef" .Values.config | fromYaml).name }}
+        {{- end }}
+        {{- if eq (include "useGalleyCassandraTLS" .Values.config) "true" }}
+        - name: "background-worker-cassandra-galley"
+          secret:
+            secretName: {{ (include "galleyTlsSecretRef" .Values.config | fromYaml).name }}
         {{- end }}
         {{- if .Values.config.rabbitmq.tlsCaSecretRef }}
         - name: "rabbitmq-ca"
@@ -58,11 +68,21 @@ spec:
             {{- toYaml .Values.podSecurityContext | nindent 12 }}
         {{- end }}
           volumeMounts:
+          - name: "background-worker-secrets"
+            mountPath: "/etc/wire/background-worker/secrets"
           - name: "background-worker-config"
             mountPath: "/etc/wire/background-worker/conf"
-          {{- if eq (include "useCassandraTLS" .Values.config) "true" }}
-          - name: "background-worker-cassandra"
-            mountPath: "/etc/wire/background-worker/cassandra"
+          {{- if eq (include "useGundeckCassandraTLS" .Values.config) "true" }}
+          - name: "background-worker-cassandra-gundeck"
+            mountPath: "/etc/wire/background-worker/cassandra-gundeck"
+          {{- end }}
+          {{- if eq (include "useBrigCassandraTLS" .Values.config) "true" }}
+          - name: "background-worker-cassandra-brig"
+            mountPath: "/etc/wire/background-worker/cassandra-brig"
+          {{- end }}
+          {{- if eq (include "useGalleyCassandraTLS" .Values.config) "true" }}
+          - name: "background-worker-cassandra-galley"
+            mountPath: "/etc/wire/background-worker/cassandra-galley"
           {{- end }}
           {{- if .Values.config.rabbitmq.tlsCaSecretRef }}
           - name: "rabbitmq-ca"

charts/background-worker/templates/secret.yaml

@@ -15,4 +15,7 @@ data:
   {{- with .Values.secrets }}
   rabbitmqUsername: {{ .rabbitmq.username | b64enc | quote }}
   rabbitmqPassword: {{ .rabbitmq.password | b64enc | quote }}
+  {{- if .pgPassword }}
+  pgPassword: {{ .pgPassword | b64enc | quote }}
+  {{- end }}
   {{- end }}

charts/background-worker/values.yaml

@@ -19,6 +19,22 @@ config:
   logLevel: Info
   logFormat: StructuredJSON
   enableFederation: false # keep in sync with brig, cargohold and galley charts' config.enableFederation as well as wire-server chart's tags.federation
+  # Postgres connection settings
+  #
+  # Values are described in https://www.postgresql.org/docs/17/libpq-connect.html#LIBPQ-PARAMKEYWORDS
+  # To set the password via a background-worker secret see `secrets.pgPassword`.
+  #
+  # Below is an example configuration used in CI tests.
+  postgresql:
+    host: postgresql # DNS name without protocol
+    port: "5432"
+    user: wire-server
+    dbname: wire-server
+  postgresqlPool:
+    size: 100
+    acquisitionTimeout: 10s
+    agingTimeout: 1d
+    idlenessTimeout: 10m
   rabbitmq:
     host: rabbitmq
     port: 5672
@@ -29,15 +45,37 @@ config:
     # tlsCaSecretRef:
     #   name: <secret-name>
     #   key: <ca-attribute>
+  # Cassandra clusters used by background-worker
   cassandra:
     host: aws-cassandra
+  cassandraBrig:
+    host: aws-cassandra
+  cassandraGalley:
+    host: aws-cassandra
 
   backendNotificationPusher:
     pushBackoffMinWait: 10000 # in microseconds, so 10ms
     pushBackoffMaxWait: 300000000 # microseconds, so 300s
     remotesRefreshInterval: 300000000 # microseconds, so 300s
 
-secrets: {}
+  # Background jobs consumer configuration
+  backgroundJobs:
+    # Maximum number of in-flight jobs per process
+    concurrency: 8
+    # Per-attempt timeout in seconds
+    jobTimeout: 60
+    # Total attempts, including the first try
+    maxAttempts: 3
+
+  domain: example.org
+
+  # Controls where conversation data is stored/accessed
+  postgresMigration:
+    conversation: postgresql
+
+secrets:
+  {}
+  # pgPassword: <postgres-password>
 
 podSecurityContext:
   allowPrivilegeEscalation: false

charts/integration/templates/configmap.yaml

@@ -56,6 +56,11 @@ data:
     backgroundWorker:
       host: backgroundWorker.{{ .Release.Namespace }}.svc.cluster.local
       port: 8080
+      # Background jobs defaults for integration tests
+      backgroundJobs:
+        concurrency: 4
+        jobTimeout: 5
+        maxAttempts: 3
 
     stern:
       host: stern.{{ .Release.Namespace }}.svc.cluster.local

docs/src/developer/reference/config-options.md

@@ -1707,3 +1707,43 @@ gundeck:
   settings:
     cellsEventQueue: "cells_events"
 ```
+## Background worker: Background jobs
+
+The background worker consumes jobs from RabbitMQ to process tasks asynchronously. The following configuration controls the consumer’s behavior:
+
+Internal YAML file and Helm values (under `background-worker.config`):
+
+```yaml
+backgroundJobs:
+  # Maximum number of in-flight jobs per process
+  concurrency: 8
+  # Per-attempt timeout in seconds
+  jobTimeout: 60
+  # Total attempts including the first run
+  maxAttempts: 3
+```
+
+Notes:
+
+- `concurrency` controls the AMQP prefetch and caps parallel handler execution per process.
+- `jobTimeout` bounds each attempt; timed‑out attempts are retried until `maxAttempts` is reached.
+- `maxAttempts` is total tries (first run plus retries). On final failure, the job is dropped (NACK requeue=false) and counted in metrics.
+
+Additional background-worker configuration:
+
+```yaml
+# Cassandra clusters
+cassandra:
+  host: aws-cassandra
+cassandraBrig:
+  host: aws-cassandra
+cassandraGalley:
+  host: aws-cassandra
+
+# Conversation storage backend selection
+postgresMigration:
+  conversation: cassandra # or postgresql
+```
+
+- `cassandraGalley` configures the third Cassandra cluster used for conversation-related data; TLS may be configured via `tlsCa` or `tlsCaSecretRef` similarly to the other clusters.
+- `postgresMigration.conversation` selects the storage location for conversation data; aligns with galley’s option and defaults to `cassandra`.