diff --git a/doc/content/enterprise/kubernetes/azure/architecture/_index.md b/doc/content/enterprise/kubernetes/azure/architecture/_index.md
index 879994e09f..ea22b554b3 100644
--- a/doc/content/enterprise/kubernetes/azure/architecture/_index.md
+++ b/doc/content/enterprise/kubernetes/azure/architecture/_index.md
@@ -19,11 +19,15 @@ The NAT GW is assigned a Public IP Address Prefix while the LB is assigned a sta
## Data Storage
-{{% tts %}} relies on two databases: PostgreSQL and Redis. Azure offers these databases as managed services.
+{{% tts %}} relies on two databases: PostgreSQL and Redis.
-We use [Azure Database for PostgreSQL](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/overview) as the relational database.
+For PostgreSQL, we use [Azure Database for PostgreSQL](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/overview) as the relational database.
-Azure offers a managed [Azure Cache for Redis Database](https://azure.microsoft.com/en-us/products/cache) service, but due to it's version not being compatible with {{% tts %}} the Redis we use is in-cluster. The newest Redis version supported by Azure Cache is 6.0, while the minimum required version of Redis for {{% tts %}} is 6.2.
+For Redis, we use an in-cluster deployment instead of the [Azure Cache for Redis Database](https://azure.microsoft.com/en-us/products/cache) service due to incompatibility with {{% tts %}}. The newest Redis version supported by Azure Cache is 6.0, while the minimum required version of Redis for {{% tts %}} is 6.2.
+
+{{< note >}}
+Azure Managed Redis does support a higher version, but it's available only in clustered mode. {{% tts %}} does not work on clustered Redis.
+{{}}
User uploads of profile pictures and end device pictures are stored in public Azure Storage Containers (Optional).
diff --git a/doc/content/enterprise/kubernetes/azure/architecture/azure-infrastructure.svg b/doc/content/enterprise/kubernetes/azure/architecture/azure-infrastructure.svg
index c59183df2a..5c6ca866f8 100644
--- a/doc/content/enterprise/kubernetes/azure/architecture/azure-infrastructure.svg
+++ b/doc/content/enterprise/kubernetes/azure/architecture/azure-infrastructure.svg
@@ -1,4 +1,4 @@
-
\ No newline at end of file
+
\ No newline at end of file
diff --git a/doc/content/enterprise/kubernetes/generic/_index.md b/doc/content/enterprise/kubernetes/generic/_index.md
index 41665925f0..8052259471 100644
--- a/doc/content/enterprise/kubernetes/generic/_index.md
+++ b/doc/content/enterprise/kubernetes/generic/_index.md
@@ -12,7 +12,7 @@ This guide helps the user to install and configure {{% tts %}} on a Kubernetes c
-Operating The Things Stack on a Kubernetes cluster is only meant for advanced users with sufficient experience with the chosen infrastructure platform and with Kubernetes.
+{{< warning "Operating The Things Stack on a Kubernetes cluster is only meant for advanced users with sufficient experience with the chosen infrastructure platform and with Kubernetes." />}}
{{< warning >}} Until the release of v1.0.0, all versions are considered to be in the Alpha stage and are not suitable for production use. The Things Industries does not offer any guarantees on compatibility between the Alpha versions. {{}}
diff --git a/doc/content/enterprise/kubernetes/generic/configuration/_index.md b/doc/content/enterprise/kubernetes/generic/configuration/_index.md
index 760947c492..9d35ff220c 100644
--- a/doc/content/enterprise/kubernetes/generic/configuration/_index.md
+++ b/doc/content/enterprise/kubernetes/generic/configuration/_index.md
@@ -29,7 +29,7 @@ global:
adminUserID: # User ID for the Administrator of the tenant. Do not use `admin`.
adminEmail: # Email of the Administrator of the tenant.
blob:
- provider: # "aws", "azure", "gcp", "local" or empty string to disable blob usage
+ provider: # "aws", "azure", "gcp", "local" or empty string to disable blob usage.
aws: # Set only if provider is "aws".
region: # region
accessKeyID: # AWS access key ID
@@ -108,6 +108,8 @@ To set up the gateway controller, the following fields must be filled in `values
`global.ttgc.tls.secretName`| Kubernetes Secret name containing the TLS. Should be the same as `global.ingress.tls.secretName`.
`global.ttgc.address` | (Optional) The URL of the gateway controller. It not specified, it defaults to `gc.thethings.industries:443`.
+{{< note "In case you are using the cloud-managed The Things Gateway Controller (i.e. `gc.thethings.industries:443`), your network must be registered on The Things Gateway Controller, otherwise connections to the gateway controller will fail. Please [contact The Things Industries support](mailto:support@thethingsindustries.com) for registration." />}}
+
{{% tts %}} verifies the identity of each connected TTIGPro gateway for security reasons, therefore mutual TLS is necessary to be configured in the ingress controller. mTLS configuration depends on the chosen ingress controller and is left to the operator of the Kubernetes cluster. {{% tts %}} recognizes the following client certificate header names:
- `X-Forwarded-Client-Cert`
- `X-Forwarded-Tls-Client-Cert`
@@ -155,7 +157,7 @@ For more info check the [Traefik docs on the PassTLSClientCert middleware](https
- Apply the k8s manifest to the cluster:
```bash
- kubectl apply -f traefik-passtlsclientcert.yaml
+ kubectl apply -f traefik-tlsoption.yaml
```
{{< note "{{% tts %}} Helm chart uses wildcard domains in ingress routes. This is necessary for multi-tenant deployments as we don't know the names of the tenants in advance. Traefik does not support TLS options for wildcard domains, because it maps the TLS options based solely on the host name (the `Host` part of the ingress rule) and it needs a concrete domain to match ([check out more in the Traefik docs](https://doc.traefik.io/traefik/v2.3/routing/routers/#options)). To go around this limitation, a default TLS option can be used which is the fallback for any unmatched host." />}}
diff --git a/doc/content/enterprise/kubernetes/generic/database-migrations/_index.md b/doc/content/enterprise/kubernetes/generic/database-migrations/_index.md
index 01808efdd9..bbfb428a33 100644
--- a/doc/content/enterprise/kubernetes/generic/database-migrations/_index.md
+++ b/doc/content/enterprise/kubernetes/generic/database-migrations/_index.md
@@ -21,6 +21,7 @@ Set the respective Helm chart values to `true` to perform database migrations du
| ------- | ---------------------------- |
| IS | `is.database.migrate` |
| NOC | `noc.store.database.migrate` |
+| AS | `as.storage.migrate` |
{{< note >}}
For example to migrate Identity Server use the following config.
@@ -33,8 +34,6 @@ is:
{{}}
-Remember to unset the value after the upgrade is done.
-
## Manual Procedure
1. Delete current Jobs (if any).
diff --git a/doc/content/enterprise/kubernetes/generic/install-charts/_index.md b/doc/content/enterprise/kubernetes/generic/install-charts/_index.md
index 1997434c04..e9b0a6adef 100644
--- a/doc/content/enterprise/kubernetes/generic/install-charts/_index.md
+++ b/doc/content/enterprise/kubernetes/generic/install-charts/_index.md
@@ -45,7 +45,17 @@ Assuming that you created a deployment specific `values` file in the previous se
$ helm install oci://registry-1.docker.io/thethingsindustries/lorawan-stack-helm-chart --version --values .values.yaml
```
-4. Update the chart.
+4. Check if the chart is set up properly.
+
+Run the helm tests to see if {{% tts %}} is configured properly:
+
+```bash
+$ helm test
+```
+
+If any test fails, check the logs of the test pods to troubleshoot the issue.
+
+5. Update the chart.
To deploy updates to the chart.
@@ -57,7 +67,7 @@ $ helm upgrade oci://registry-1.docker.io/thethingsindustries/lorawan-sta
> Note: The `helm` CLI pulls a chart from the repository only if it's locally not present.
-5. Uninstall the Chart.
+6. Uninstall the Chart.
Once done, you can uninstall the chart as follows.
diff --git a/doc/content/enterprise/kubernetes/generic/prerequisites/_index.md b/doc/content/enterprise/kubernetes/generic/prerequisites/_index.md
index 07909110fe..9910f2aa59 100644
--- a/doc/content/enterprise/kubernetes/generic/prerequisites/_index.md
+++ b/doc/content/enterprise/kubernetes/generic/prerequisites/_index.md
@@ -39,8 +39,8 @@ Please [contact our sales team](mailto:sales@thethingsindustries.com) for access
{{% tts %}} on Kubernetes requires the following infrastructural services to run.
1. A Kubernetes cluster
-2. PostgreSQL compatible database
-3. Redis compatible database (Redis 6.2 or above is required)
+2. PostgreSQL 14 or above
+3. Redis 6.2 or above
4. Blob Storage
5. An ingress controller to handle the ingress routes
6. TLS Certificates
@@ -49,11 +49,11 @@ Please [contact our sales team](mailto:sales@thethingsindustries.com) for access
These components are highly specific to the specific infrastructure chosen by the operator and hence are out of scope of this documentation.
-However, the following is a guide of the general principles involved in setting up the infrastructure.
+The following is a guide of the general principles involved in setting up the infrastructure.
#### 1. Kubernetes Cluster
-{{% tts %}} requires a minimum kubernetes version of v1.21. However, we recommend using the highest available version.
+{{% tts %}} requires a minimum kubernetes version of v1.21. We recommend using the highest available version.
#### 2. Postgres Compatible Database
@@ -97,6 +97,12 @@ The Things Stack requires the following buckets.
- The contents of this bucket must be _private_ since they contain secrets.
- Enabling encryption and versioning is highly recommended.
+5. Plugins configuration
+
+- Once this bucket is setup, place an empty `plugins.yml` file at the root of the bucket.
+- The contents of this bucket must be _private_ since they contain secrets.
+- Enabling encryption and versioning is highly recommended.
+
##### Using Local Blob Storage
In the case of using a local blob, the following steps are necessary.
@@ -107,108 +113,27 @@ In the case of using a local blob, the following steps are necessary.
$ sudo chown -R 886:886
```
-2. Create the `edcs`, `interop`, `end_device_pictures` and `profile_pictures` folders at the root of the blob folder.
+2. Create the `edcs`, `interop`, `end_device_pictures`, `profile_pictures`, `plugins` folders at the root of the blob folder.
3. Create an empty `config.yml` in the `edcs` and `interop` folders.
+4. Create an empty `plugins.yml` in the `plugins` folder.
-##### Disabling Blob Storage
+#### 5. An ingress controller
-{{% tts %}} Helm Chart by default expects a blob storage configured but it is possible to use {{% tts %}} without it. You can disable the usage of blob by setting `global.interop.configSource` and `global.blob.provider` values to an empty string `""`.
+An ingress controller is needed to route the incoming traffic. {{% tts %}} Helm chart uses Kubernetes ingress resources for routing requests to the components of {{% tts %}}. This allows the users of {{% tts %}} Helm chart to configure an ingress controller of their choice. However, Kubernetes ingress routes support only L7 traffic (HTTP/gRPC). For this reason, the configuration for routing UDP Packet Forwarder traffic for gateways is not handled by the Helm chart.
-#### 5. An ingress controller
+{{% tts %}} needs several [port allocations with various protocols]({{< ref "/concepts/networking/#port-allocations" >}}). We recommend using an ingress controller that natively supports L4 and L7 protocols. Depending on your preferred setup for gateways, an ingress controller that supports only L7 protocols (such as Ingress-Nginx) can be used too, but the connecting gateways will be either limited to L7 protocols or more complex to setup and maintain if using L4 protocols.
-An ingress controller is needed to route the incoming traffic. Specify the ingress controller by setting the `global.ingress.controller` to the class name of the ingress controller deployed in the cluster. For TLS, make sure to set the `global.ingress.controller.tls.secretName`. The secret has to be accessible from the namespace where the {{% tts %}} Helm Chart is deployed. These ports are needed by {{% tts %}} and must be exposed:
-
-```yaml
-web:
- protocol: TCP
- port: 1885
- exposedPort: 80
-websecure:
- protocol: TCP
- port: 8885
- exposedPort: 443
-grpc:
- protocol: TCP
- port: 1884
- exposedPort: 1884
-grpcsecure:
- protocol: TCP
- port: 8884
- exposedPort: 8884
-# Gateway Connectivity
-gtwmqttv2:
- protocol: TCP
- port: 1881
- exposedPort: 1881
-gtwmqttv2secure:
- protocol: TCP
- port: 8881
- exposedPort: 8881
-gtwmqttv3:
- protocol: TCP
- port: 1882
- exposedPort: 1882
-gtwmqttv3secure:
- protocol: TCP
- port: 8882
- exposedPort: 8882
-semtechws:
- protocol: TCP
- port: 1887
- exposedPort: 1887
-semtechwssecure:
- protocol: TCP
- port: 8887
- exposedPort: 8887
-# Application MQTT
-appmqtt:
- protocol: TCP
- port: 1883
- exposedPort: 1883
-appmqttsecure:
- protocol: TCP
- port: 8883
- exposedPort: 8883
-# The Things Indoor Gateway Pro
-ttigw:
- protocol: "TCP"
- port: 1889
- exposedPort: 1889
-ttigwsecure:
- protocol: "TCP"
- port: 8889
- exposedPort: 8889
-# Interoperability. This part is optional. Only enable it if interoperability is needed.
-interop:
- protocol: TCP
- # Note: Change this to 1886 if using `server-only` mode.
- port: 8886
- exposedPort: 8886
-```
+Although we do support UDP Packet Forwarder as a gateway connection option, it requires [more configuration on your side]({{< ref "enterprise/kubernetes/generic/#udp-gateway-support" >}}). We recommend using LoRa Basics™ Station or The Things Industries Gateway protocols for connecting gateways.
-In case annotations are needed for certain protocols or for the {{% tts %}} services, these can be specified under `global.ingress.annotations` and `global.ingress.serviceAnnotations`. E.g. Traefik annotations can be specified as:
-```yaml
-ingress:
- controller: "traefik"
- tls:
- secretName: "ingress-tls-cert"
- annotations:
- grpc:
- traefik.ingress.kubernetes.io/router.entrypoints: grpcsecure
- traefik.ingress.kubernetes.io/router.tls: "true"
- http:
- traefik.ingress.kubernetes.io/router.entrypoints: websecure
- semtechws:
- traefik.ingress.kubernetes.io/router.entrypoints: semtechwssecure,semtechws
- traefik.ingress.kubernetes.io/router.tls: "true"
- serviceAnnotations:
- traefik.ingress.kubernetes.io/service.serversscheme: h2c
-```
+To configure the ingress controller for {{% tts %}}:
+1. Specify the ingress controller by setting the `global.ingress.controller` to the class name of the ingress controller deployed in the cluster. This will be used to set the ingress class name in the ingress routes that handle {{% tts %}} traffic.
+2. Specify the TLS secret by setting the `global.ingress.controller.tls.secretName`. The secret has to be accessible from the namespace where the {{% tts %}} Helm Chart is deployed. This will be used to terminate TLS for {{% tts %}} traffic
+3. Add annotations for the ingress routes if needed by setting `global.ingress.annotations.http`, `global.ingress.annotations.grpc`, `global.ingress.annotations.semtechws` or `global.ingress.annotations.ttigw`.
+4. Add ingress specific service annotations for {{% tts %}} services by setting `global.ingress.serviceAnnotations` if needed.
+5. Expose the ports used by {{% tts %}} in your ingress controller. A list of all the ports can be found [here]({{< ref "/concepts/networking/#port-allocations" >}}). For production environments, make sure to expose only TLS ports.
Examples of ingress controllers configurations can be found [here](https://www.thethingsindustries.com/docs/the-things-stack/host/kubernetes/generic/prerequisites/sample-ingress-controllers/).
-{{< note "{{% tts %}} Helm chart uses Kubernetes ingress rules for routing requests to the components of {{% tts %}}. This allows the users of {{% tts %}} Helm chart to configure an ingress controller of their choice. However, Kubernetes ingress routes support only L7 traffic (HTTP/gRPC). For this reason, UDP Packet Forwarder for gateways is not supported in the Helm chart for now." />}}
-
#### 6. TLS Certificates
The Things Stack expects a [Kubernetes TLS Secret](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) which contains the server leaf certificates.
diff --git a/doc/content/enterprise/kubernetes/generic/prerequisites/sample-ingress-controllers.md b/doc/content/enterprise/kubernetes/generic/prerequisites/sample-ingress-controllers.md
index 36880eec98..c019bf0ec1 100644
--- a/doc/content/enterprise/kubernetes/generic/prerequisites/sample-ingress-controllers.md
+++ b/doc/content/enterprise/kubernetes/generic/prerequisites/sample-ingress-controllers.md
@@ -8,44 +8,47 @@ aliases: [
]
---
-The following are examples of ingress controllers for {{% tts %}} deployment on Kubernetes.
+The following are examples of ingress controller configurations for {{% tts %}} deployment on Kubernetes.
## Traefik
-Example of a Traefik configuration provided through the values of an Traefik Helm chart. More info about the Helm chart
-can be found [here](https://github.com/traefik/traefik-helm-chart).
+Example of a Traefik configuration. More info about the Helm chart can be found [here](https://github.com/traefik/traefik-helm-chart).
+
+{{% tts %}} Helm chart ingress configuration:
+
+```yaml
+global:
+ ingress:
+ controller: "traefik"
+ tls:
+ secretName: "ingress-tls-cert"
+ annotations:
+ http:
+ traefik.ingress.kubernetes.io/router.entrypoints: web,websecure
+ traefik.ingress.kubernetes.io/router.tls: "true"
+ grpc:
+ traefik.ingress.kubernetes.io/router.entrypoints: grpc,grpcsecure
+ traefik.ingress.kubernetes.io/router.tls: "true"
+ semtechws:
+ traefik.ingress.kubernetes.io/router.entrypoints: semtechws,semtechwssecure
+ traefik.ingress.kubernetes.io/router.tls: "true"
+ serviceAnnotations:
+ traefik.ingress.kubernetes.io/service.serversscheme: h2c
+```
+
+Traefik Helm chart port mapping configuration:
```yaml
deployment:
kind: "Deployment"
replicas: 1
-ingressRoute:
- dashboard:
- enabled: false
-additionalArguments:
-- "--entrypoints.udp.udp.timeout=90s"
ports:
web:
- protocol: "TCP"
- port: 1885
- expose:
- default: true
- exposedPort: 80
- redirectTo:
- port: "websecure"
- websecure:
- protocol: "TCP"
- port: 8885
- expose:
- default: true
- exposedPort: 443
- traefik:
- protocol: "TCP"
- port: 9000
- expose:
- default: false
+ redirections:
+ to: websecure
+ scheme: https
grpc:
protocol: "TCP"
port: 1884
@@ -118,66 +121,114 @@ ports:
expose:
default: true
exposedPort: 8889
- interop:
- protocol: "TCP"
- port: 8886
+ udp:
+ protocol: "UDP"
+ port: 1700
expose:
default: true
- exposedPort: 8886
+ exposedPort: 1700
```
-## Ingress NGINX
+UDP Ingress Route for the UDP Packet Forwarder:
-Example of an Ingress NGINX configuration provided through the values of an Ingress NGINX Helm chart. More info about the
-Helm chart can be found [here](https://artifacthub.io/packages/helm/bitnami/nginx).
+{{< warning "The UDP Packet Forwarder requires session affinity to work properly. Traefik does not support session affinity for UDP. Session affinity can usually be set up on the Load Balancer of the cluster. We recommend using the newer {{% lbs %}} instead of the UDP Packet Forwarder." />}}
```yaml
-fullnameOverride: "nginx"
-namespaceOverride: "ingress-nginx"
-kind: Deployment
-replicaCount: '1'
-config:
- # redirect port 80 to 443 for HTTP to HTTPS.
- ssl-redirect: "true"
- upstream-keepalive-timeout: '90s'
-service:
- ports:
- http: 80
- https: 443
- extraPorts:
- - name: semtechws
- port: 1887
- targetPort: 1887
- nodePorts:
- tcp:
- # http
- "80": "1885"
- # https
- "443": "8885"
- # grpc
- "1884": "1884"
- # grpcsecure
- "8844": "8884"
- # gtwmqttv2
- "1881": "1881"
- # gtwmqttv2secure
- "8881": "8881"
- # gtwmqttv3
- "1882": "1882"
- # gtwmqttv3secure
- "8882": "8882"
- # semtechws
- "1887": "1887"
- # semtechwssecure
- "8887": "8887"
- # appmqtt
- "1883": "1883"
- # appmqttsecure
- "8883": "8883"
- # interop
- "8886": "8886"
- # ttigw
- "1889": "1889"
- # ttigwsecure
- "8889": "8889"
+apiVersion: traefik.containo.us/v1alpha1
+kind: IngressRouteUDP
+metadata:
+ name: tts-gs-udp-packet-forwarder
+ namespace: tts # Set this to the namespace where TTS is deployed.
+spec:
+ entryPoints:
+ - udp
+ routes:
+ - services:
+ - name: tts-gs # -gs
+ port: 1700
+```
+
+TCP Ingress Route for MQTT application integration:
+
+```yaml
+apiVersion: traefik.io/v1alpha1
+kind: IngressRouteTCP
+metadata:
+ name: tts-as-mqtt
+ namespace: tts # Set this to the namespace where TTS is deployed.
+spec:
+ entryPoints:
+ - appmqtt
+ routes:
+ - match: HostSNI(`*`)
+ services:
+ - name: tts-as # -as
+ port: 1883
+---
+apiVersion: traefik.io/v1alpha1
+kind: IngressRouteTCP
+metadata:
+ name: tts-app-mqtt-secure
+ namespace: tts # Set this to the namespace where TTS is deployed.
+spec:
+ entryPoints:
+ - appmqttsecure
+ routes:
+ - match: HostSNI(`*`)
+ services:
+ - name: tts-as # -as
+ port: 1883
+ tls:
+ secretName: ingress-tls-secret
+```
+
+## Ingress-Nginx Controller
+
+Example of an Ingress-Nginx Controller configuration. More info about the Helm chart can be found [here](https://kubernetes.github.io/ingress-nginx/).
+
+{{% tts %}} Helm chart ingress configuration:
+
+```yaml
+global:
+ ingress:
+ controller: "nginx"
+ tls:
+ secretName: "ingress-tls-cert"
+ annotations:
+ grpc:
+ nginx.ingress.kubernetes.io/backend-protocol: GRPC
+ nginx.ingress.kubernetes.io/use-http2: "true"
+ http:
+ nginx.ingress.kubernetes.io/backend-protocol: HTTP
+ nginx.ingress.kubernetes.io/use-http2: "true"
+ semtechws: {}
+ ttigw: {}
+ serviceAnnotations: {}
+```
+
+Ingress-Nginx Helm chart port mapping configuration:
+
+{{< note "When using Ingress-Nginx, enable the TLS ports in the Helm chart (v0.0.13 and above)." />}}
+
+```yaml
+controller:
+ config:
+ use-http2: true
+ strict-validate-path-type: false # This is needed to allow gRPC paths that contain dots.
+ default-ssl-certificate: / # Make sure to set this.
+tcp:
+ # Plain-text ports.
+ "1881": tts/the-things-stack-gs:1881 # /-gs:1881
+ "1882": tts/the-things-stack-gs:1882
+ "1883": tts/the-things-stack-as:1883
+ "1887": tts/the-things-stack-gs:1887
+ "1889": tts/the-things-stack-gs:1889
+ # TLS ports.
+ "8881": tts/the-things-stack-gs:8881
+ "8882": tts/the-things-stack-gs:8882
+ "8883": tts/the-things-stack-as:8883
+ "8887": tts/the-things-stack-gs:8887
+ "8889": tts/the-things-stack-gs:8889
+udp:
+ "1700": tts/the-things-stack-gs:1700
```