Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Horizontal scaling in k8s #966

Closed
swills opened this issue Feb 2, 2025 · 3 comments
Closed

Horizontal scaling in k8s #966

swills opened this issue Feb 2, 2025 · 3 comments
Labels
question Further information is requested

Comments

@swills
Copy link

swills commented Feb 2, 2025

Is it safe to increase the number of replicas for the web pod in k8s?

I've customized the number of replicas on the chrome pod and that seems fine, but of course that has no local data.

It looks to be unsafe to increase replicas on the meilisearch pod based on the reading I've done about that, since it uses LMDB.

But I don't know enough about the stuff running in the web pod to know if it's safe to increase the replicas. (This would of course mean using a storage class for the PVC which supports ReadWriteMany access.)

@MohamedBassem
Copy link
Collaborator

Honestly, I've not given it a try before but theoretically speaking I see nothing that would prevent it from working.

The only caveat that I can think of is that you'll be forced to have double the number of background job workers (because they are baked into the web pod). For example, if you have CRAWLER_NUM_WORKERS=1, each replica will have 1 worker (so you end up with X workers instead of 1). The work should get correctly distributed across them though, so it shouldn't be that big of a deal.

I'd recommend taking backups before going with that route. Let me know how it goes, I'm curious :)

@MohamedBassem MohamedBassem added the question Further information is requested label Feb 2, 2025
@swills
Copy link
Author

swills commented Feb 2, 2025

Got it, thanks! I'll definitely back up (and restore), as I have to do that any way in order to migrate from the rbd block (RWO) to NFS (RWM) storage. I'll let you know how it goes.

@swills
Copy link
Author

swills commented Feb 3, 2025

Well, it seems to be mostly OK, at least in my limited single user testing. I do occasionally get "Database is locked" errors when adding things.

@swills swills closed this as completed Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants