Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow REDIS_CLUSTER environment variable to use Redis in cluster mode #2377

Closed
wants to merge 2 commits into from

Conversation

n1mmy
Copy link

@n1mmy n1mmy commented Feb 8, 2025

Using a Redis cluster instead of a single host allows for a more high-availability deployment.

This patch is only allows a single seed host. A more full implementation would have some way to support multiple seed hosts (eg a comma separated list, or multiple environment variables). However, this is enough to get started and works well in environments that have a single name/IP that will connect to a redis cluster (eg a load balancer or a kubernetes service).

@n1mmy n1mmy force-pushed the master branch 3 times, most recently from 06183c8 to f756178 Compare February 8, 2025 16:22
@J0WI
Copy link
Contributor

J0WI commented Feb 13, 2025

There are several similar PRs: #2334, #2288 and #2236
Would you like to integrate them here?

@n1mmy
Copy link
Author

n1mmy commented Feb 19, 2025

@J0WI I can give it shot when I have some free development time in the next week or two. I don't have a great testing environment at the moment, but I'll figure something out.

Please don't wait for me though, if other things are ready to merge go ahead and merge them and I can update this PR for the new changes.

@n1mmy n1mmy marked this pull request as draft February 20, 2025 04:11
@n1mmy
Copy link
Author

n1mmy commented Feb 20, 2025

After looking at the other PRs in more detail, I realize this PR needs to update redis.config.php as well. I will update it when I am able. Please do not merge yet.

@n1mmy
Copy link
Author

n1mmy commented Feb 20, 2025

Nevermind. I found a better way to accomplish what I was trying to do without having to change the docker image.

In case anyone is looking for how to set up redis cluster mode, here's what I did:

  1. For your first run of the container to set up the config file, use the REDIS_HOST environment variable to get a starter redis config. For all subsequent runs, do not set REDIS_HOST so it does not create an incorrect session.ini file.

  2. Manually edit config/config.php and replace the redis section with redis.cluster section. For example:

  'redis.cluster' =>
  array (
    'seeds' =>
    array (
      0 => '10.103.186.138:6379',
    ),
    'password' => '',
    'port' => 6379,
    'dbindex' => 0,
    'timeout' => 1.5,
    'failover_mode' => 2,
  ),
  1. Bind mount a single .ini file into /usr/local/etc/php/conf.d. Do not bind mount a whole directory here, it would overshadow important files.

Here are the contents of my custom-session.ini file:

session.save_handler = rediscluster
session.save_path = "seed[]=10.103.186.138:6379"
redis.session.locking_enabled = 1
redis.session.lock_retries = -1
redis.session.lock_wait_time = 10000

If you are running docker on the command line, you can bind mount it into the appropriate place by adding -v ./custom-session.ini:/usr/local/etc/php/conf.d/custom-session.ini

If you are using Kubernetes, you can use a ConfigMap and a volumeMount with subPath to accomplish this. Here are some snippets from my config.

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: nextcloud
  name: nextcloud-config
data:
  custom-session.ini: |-
    session.save_handler = rediscluster
    session.save_path = "seed[]=10.103.186.138:6379"
    redis.session.locking_enabled = 1
    redis.session.lock_retries = -1
    redis.session.lock_wait_time = 10000

---

apiVersion: apps/v1
kind: Deployment
...
      volumes:
      - name: html
        persistentVolumeClaim:
          claimName: nextcloud-data-pvc
          readOnly: false
      - name: nextcloud-config
        configMap:
          name: nextcloud-config
...
        volumeMounts:
        - mountPath: /var/www/html
          name: html
        - mountPath: /usr/local/etc/php/conf.d/custom-session.ini
          name: nextcloud-config
          subPath: custom-session.ini

Once you php sessions using redis cluster and nextcloud itself using redis cluster, you can scale to multiple replicas of the nextcloud process and have a high availability setup.

Hope this helps someone in the future!

@n1mmy n1mmy closed this Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants