Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set up ClickHouse keeper cross AKS cluster #1617

Open
chaos827 opened this issue Jan 14, 2025 · 0 comments
Open

set up ClickHouse keeper cross AKS cluster #1617

chaos827 opened this issue Jan 14, 2025 · 0 comments

Comments

@chaos827
Copy link

I followed this article(https://www.farukcevik.com.tr/blog/2024/02/07/clickhousedisasterrecovery/) to set up ClickHouse in two data centers, for the ClickHouse instance, I used external Load Balancer ip to set up remote servers and it works well, so I followed the same way to set up ClickHouse Keeper, but seems the only 3 prod keeper instances are active, the bcp instances are not health.

this may config for the primary
<keeper_server>
<server_id>1</server_id>
<raft_configuration>
1hostname19444
2hostname29444
3hostname39444
4external load balancer ip for bcp19444
5external load balancer ip for bcp29444
6external load balancer ip for bcp39444
</raft_configuration>
</keeper_server>

for the bcp:
<keeper_server>
<server_id>1</server_id>
<raft_configuration>
1external load balancer ip for prod19444
2external load balancer ip for prod29444
3external load balancer ip for prod39444
4hostname19444
5hostname29444
6hostname39444
</raft_configuration>
</keeper_server>

I used clickhouse-keeper-client to show the config info, the prod keeper, shows there are 6 servers, and in the BCP, the clickhouse-keeper-client throws error:
Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: 10.200.0.10:2181
Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 24.5.1.1763 (official build)), 10.200.0.10:2181

I do not figure out why the clickhkouse-keeper-client throws this error, and if I set the bcp clickhouse to only connect to the bcp keeper, the bcp clickhouse data is uncoordinated, does someone know how to set up keeper in two AKS?
Many thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant