Cluster operations #189

yuqiu123 · 2023-03-31T09:15:20Z

No description provided.

Co-authored-by: nanana-w <[email protected]>

yixin-chen-23 · 2023-04-03T08:21:50Z

docs/cluster-operations/scale-cluster.md

+    chunk:
+    replicas: "" # Enter the number of chunk pods. 
+    ```
+2. Apply the modification.


0.11 的文档里这里有一句 replace iomesh with your release name @gxglls 这里还要加吗如果加的话，replace 的 iomesh 是哪个？

这句话去掉了

yixin-chen-23 · 2023-04-03T08:27:30Z

docs/cluster-operations/scale-cluster.md

+
+Increasing the number of meta pods is not supported for IOMesh Community Edition. For the Enterprise Edition, the minimum number of meta pods is 3 and the maximum number is 5.
+
+1. In `iomesh.yaml`, locate `meta` and then edit `replicas`. 


0.11 的文档里这里的字段是 replicaCount 麻烦确认下 @gxglls

确实是 replicaCount，麻烦 @yuqiu123 调整一下

只有这一处需要改吗？全文搜索了也有其他的地方是 replicas, 比如：

全局修改为 replicas，子银代码会改 @yixin-chen-23

yixin-chen-23 · 2023-04-03T08:33:47Z

docs/cluster-operations/upgrade-cluster.md

+3. Install the CRD of IOMesh 1.0.
+
+    ```shell
+    kubectl apply -f https://iomesh.run/config/crd/iomesh.com_blockdevicemonitors.yaml


会有离线升级的情况吗？ @gxglls

单集群里离线升级的内容需要补充在这 https://github.com/iomesh/docs_cn/pull/12/files

补充了 http://192.168.26.13:3000/next/cluster-operations/upgrade-cluster

docs/advanced-functions/multiple-cluster-management.md

gxglls · 2023-04-04T07:54:40Z

docs/advanced-functions/multiple-cluster-management.md

+sidebar_label: Multiple Cluster Management
+---
+
+In a large-scale Kubernetes cluster, you can deploy multiple IOMesh clusters for data isolation, and each IOMesh cluster is an independent storage pool. In this case, the IOMesh CSI driver enables different IOMesh clusters to communicate, reducing the number of CSI drivers that may consume unnecessary node resources.


In this case, the IOMesh CSI driver enables different IOMesh clusters to communicate
我们的功能是 csi 可以链接不同改动 IOMesh 集群，这句话读起来感觉是 csi 让不同的 iomesh 集群通信

”链接不同改动 IOMesh 集群“是啥意思？

嗯，我也觉得这里 to communicate 不太对，是不是可以改为不同的 IOMesh 集群共用 CSI Driver？

另外，reducing the number of CSI drivers 这个我建议修改为减少 CSI Driver 的 Controller Plugin Pod 的数量，因为下面的图其实也看不出具体是少了什么

【减少 CSI Driver 的 Controller Plugin Pod 的数量】需要这么细吗？当时问 @gxglls 这里的 benefit是从多个 CSI 减到只用我们的一个 IOMesh CSI driver 就可以了，减少资源消耗

多个 CSI 也是我们的 CSI，这里是不是有什么搞混了

这里是减少 Controller Plugin Pod 的数量。没有多个 csi 的概念，只有 IOMesh CSI driver

In a large-scale Kubernetes cluster, you can deploy multiple IOMesh clusters for data isolation. In this case, there is typically one management cluster that holds components such as the IOMesh Operator, IOMesh CSI Driver, and Node Disk Manager, while other clusters function as independent storage pools without any management responsibilities.

The IOMesh CSI driver is shared by all IOMesh clusters to facilitate connection, which reduces the number of Controller Plugin pods for the CSI driver.

docs/advanced-functions/multiple-cluster-management.md

gxglls · 2023-04-04T09:56:59Z

docs/advanced-functions/multiple-cluster-management.md

+
+        ```yaml
+        meta:
+              podPolicy:


gxglls · 2023-04-04T09:57:08Z

docs/advanced-functions/multiple-cluster-management.md

+        iomesh:
+        ...
+            chunk:
+            podPolicy:


docs/advanced-functions/multiple-cluster-management.md

nanana-w · 2023-04-06T03:48:34Z

docs/advanced-functions/multiple-cluster-management.md

+
+## Deployment
+
+To reduce the number of Pods required in a multi-cluster deployment of IOMesh, the management components shared by all IOMesh clusters, including the IOMesh Operator, IOMesh CSI driver, and Node Disk Manager, will be installed on the first IOMesh cluster, which is referred to as the management cluster.


建议调整：https://smartx1.slack.com/archives/C04AW6VDKF1/p1680752858292449?thread_ts=1680254292.921529&cid=C04AW6VDKF1

【operator 和 ndm 需要确认下】 @gxglls 子银看看？

operator 和 ndm 是共享的

nanana-w · 2023-04-06T03:52:04Z

docs/advanced-functions/multiple-cluster-management.md

+   operator-f5644b7f9-2vvw7                              1/1     Running   0          5h37m
+   ```
+
+### Deploy Second IOMesh Cluster


如果是一开始按照 install iomesh 里部署了集群，之后想部署第二个，该怎么做

不支持从一个扩到多个，支持从多个集群扩，比如从 2 个扩到 3 个 @gxglls

那在单集群的安装那里是不是加一个提示比较好？如果有部署多集群的计划，建议按照多集群部署里部署第一个集群的步骤部署第一个集群，这个建议可行吗？

目前 “从一个扩到多个” 这个功能我想了下已经是可以支持的，我提个 pr 去掉这个限制描述

这个 pr 有下文没？ @gxglls

nanana-w · 2023-04-06T03:55:51Z

docs/advanced-functions/multiple-cluster-management.md

+
+## Operations & Management 
+
+All procedures below are listed based on the example in [Multiple Cluster Deployment](../advanced-functions/multiple-cluster-management.md#deployment).


这里的 link 和文字要修改下？

done http://192.168.26.13:3000/next/advanced-functions/manage-multiple-cluster#configure-multiple-cluster-connection

nanana-w · 2023-04-06T03:57:02Z

docs/advanced-functions/multiple-cluster-management.md

+### Upgrade Multiple Clusters
+
+> _Note:_
+> When upgrading multiple IOMesh clusters, upgrade the management cluster first and then the other clusters. If not, the non-management cluster will be temporarily unavailable during the second upgrade, but all clusters will return to normal afterwards.


在 install 的时候建议就把 management cluster 和 non-management cluster 的概念介绍一下。以及用户在部署之后，可以通过什么样的方式来判断哪个是 management cluster，因为用户可能已经不记得哪个是第一个了

这里说了

用户在之后的运维里如何判断呢？

这里重写了一下，看看是不是好读一点？http://192.168.26.13:3000/next/advanced-functions/manage-multiple-cluster

docs/advanced-functions/multiple-cluster-management.md

nanana-w · 2023-04-06T04:00:24Z

docs/advanced-functions/multiple-cluster-management.md

+When uninstalling more than one IOMesh cluster, uninstall the other clusters first and then the management cluster last. If you do not follow the order, there may be resources resided in the namespace `iomesh-system`, which may affect next deployment in this namespace.
+
+**Procedure**
+1. Uninstall the second IOMesh cluster, which will also delete `iomesh` and `zookeeper` components. 


Suggested change

1. Uninstall the second IOMesh cluster, which will also delete `iomesh` and `zookeeper` components.

1. Uninstall the non-management IOMesh cluster, which will also delete `iomesh` and `zookeeper` components.

docs/advanced-functions/multiple-cluster-management.md

docs/cluster-operations/scale-cluster.md

nanana-w · 2023-04-06T04:43:24Z

docs/cluster-operations/scale-cluster.md

+
+## Scale Meta Server
+
+Add meta pods when the meta server is overloaded. 


这里怎么判断 overloaded？

chunk pod number 和 meta pod number 有没有关系？比如 chunk pod number 超过 xx，建议增加 meta pod number？

meta number 可以配置为 4 个吗？还是只奇数？这里应该是和 leader 选举有关系，那应该最好是奇数？

@gxglls

这里 zbs 是否有 meta 需要的扩容的指标呢 @nanana-w

没有

没有个数限制

http://docs.fev.smartx.com/cn/smtxos/5.0.4/whitepaper/whitepaper_zbs#zookeeper

集群规模不超过四个节点时，必须使用三个主节点的部署方式，此时可以容忍集群中任意一个主节点宕机；当集群规模大于或等于五个节点时，必须使用五个主节点的部署方式，此时可以容忍集群中任意两个主节点宕机。

ZK 和 Meta 都是运行在主节点上，要求是一样的。建议和 SMTX OS & SMTX ZBS 保持一致。

@nanana-w 【要求是一样的。建议和 SMTX OS & SMTX ZBS 保持一致。】具体是指啥呢？写在 iomesh 文档这里怎么说比较好呢？

建议：集群规模不超过四个节点时，无需 scale meta server，此时可以容忍集群中任意一个 meta server 故障；当集群规模大于或等于五个节点时，需将 meta server 从 3 个扩容到 5 个，此时可以容忍集群中任意两个 meta server 宕机。

@gxglls 再看下

容忍故障的数量不对，meta 3 个可以容忍两个故障 @nanana-w

nanana-w · 2023-04-06T04:45:20Z

docs/cluster-operations/upgrade-cluster.md

+    ```shell
+    kubectl delete sc iomesh-csi-driver
+    ```
+2. Temporarily disable IOMesh Webhook to avoid upgrade failure. Once the upgrade is successful, it will be reinstalled automatically.


前半句说 disable，后面是不是 reenabled？

下面的命令的确是 delete 删除，删除后之后会被重新安装。 @gxglls 帮忙确认下哈

改为 reenabled

nanana-w · 2023-04-06T04:46:10Z

docs/cluster-operations/upgrade-cluster.md

+    ```shell
+    kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration
+    ```
+3. Install the CRD of IOMesh 1.0.


这里的描述最好和版本无关，之后新版本可以复用

之前加上是想更明确点，如果写 new version，可能读者不知道哪个是 new version.

nanana-w · 2023-04-06T04:47:25Z

docs/cluster-operations/upgrade-cluster.md

+4. Upgrade the IOMesh cluster. Then wait for a few minutes till all pods are running.
+
+    ```bash
+    helm upgrade --namespace iomesh-system iomesh iomesh/iomesh --version v1.0.0


这里也是，建议描述和版本号脱离开来，比如这里是需要填要升级的目标版本号

是说让读者自己修改这个命令里的版本号吗？版本号是 1.0，还是 1.0.0？

现在是 1.0.0，所以这里是 1.0.0。之后比如 1.1.0，命令行也需要修改为 1.1.0。

docs/cluster-operations/manage-license.md

nanana-w · 2023-04-06T04:53:25Z

docs/cluster-operations/scale-cluster.md

+
+**Precaution**
+
+The minimum number of chunk pods is 3 and the maximum is 255.


扩容需要确保许可里的 Max Chunk Num 是足够的，扩容后的节点数量要小于等于 Max Chunk Num

扩容只针对企业版，企业版许可是 255，所以【确保 Max Chunk Num】和【扩容后的节点小于或等于 Max Chunk Num】是不是【The minimum number of chunk pods is 3 and the maximum is 255.】已经体现了？ @nanana-w

许可是客户买了几个节点的许可，如果买了 5 个，集群就最多 5 个，我们按节点数收费

@yuqiu123 企业版许可不是 255，许可里的 最大节点数 这个字段的取值是客户购买的节点数的数量，最大会是 255

明白了。
The minimum number of chunk pods is 3, and the maximum number you can add depends on the IOMesh Enterprise Edition node limit, which can be up to 255.

Co-authored-by: yixin-chen-23 <[email protected]>

Co-authored-by: nanana-w <[email protected]>

gxglls · 2023-04-10T02:45:18Z

docs/advanced-functions/multiple-cluster-management.md

+
+### Configure Multiple-Cluster Connection
+
+To enable the IOMesh CSI driver to connect to multiple IOMesh clusters, you need to configure a `ConfigMap` containing connection information for each IOMesh cluster. Before configuration, familarity with `ConfigMap` is suggested.


Suggested change

To enable the IOMesh CSI driver to connect to multiple IOMesh clusters, you need to configure a `ConfigMap` containing connection information for each IOMesh cluster. Before configuration, familarity with `ConfigMap` is suggested.

To enable the IOMesh CSI driver to connect to multiple IOMesh clusters, you need to create a `ConfigMap` containing connection information for each IOMesh cluster. Before configuration, familarity with `ConfigMap` is suggested.

yuqiu123 and others added 30 commits February 14, 2023 17:17

prerequisite & installing IOMesh 0214

be64d9c

Setting IOMesh 0215

4986cff

deploy section 0216

c40a3a9

volume operations 0217

b61c96b

iomesh docs 0220

0c644cc

stateful applications

eb500c7

iomesh revision 0224

ad9c4cd

IOMesh 0227

73d5075

iomesh 0302

f161a74

install & deploy 0303

b766f2b

iomesh 0303

6b79e2f

iomesh 0306

2324dba

iomesh 0307

7be9b91

iomesh monitor 0308

2112c32

monitor

156e8de

iomesh docs

80057f3

iomesh 0313

66029e8

iomesh revise 0313

0a1b4d3

iomesh revise 0313

fd8f375

iomesh revise 1 0313

693412c

iomesh revise 2

4a60738

iomesh revise 3 0313

21d9415

iomesh revise 3 0313

44433ca

iomesh revise 3 0313

dd59c2c

iomesh revise 0313

96d5ccc

iomesh revise 0313

7779b19

iomesh 0316

a2f858f

iomesh 0317

fccabbf

iomesh 0319

c2345d0

Update docs/deploy-iomesh-cluster/setup-iomesh.md

5399d41

Co-authored-by: nanana-w <[email protected]>

yuqiu123 added 3 commits April 2, 2023 17:56

iomesh 0402

20000b7

iomesh 0403

bb886fa

iomesh 0403

73bcf2e

yixin-chen-23 reviewed Apr 3, 2023

View reviewed changes

iomesh 0403

5abfbaf

gxglls reviewed Apr 4, 2023

View reviewed changes

yuqiu123 added 3 commits April 4, 2023 20:20

iomesh 0404

62c83af

iomesh 0404

8eb48af

iomesh 0404

bcb6aef

nanana-w reviewed Apr 6, 2023

View reviewed changes

docs/advanced-functions/multiple-cluster-management.md Outdated Show resolved Hide resolved

nanana-w reviewed Apr 6, 2023

View reviewed changes

docs/advanced-functions/multiple-cluster-management.md Outdated Show resolved Hide resolved

nanana-w reviewed Apr 6, 2023

View reviewed changes

docs/advanced-functions/multiple-cluster-management.md Outdated Show resolved Hide resolved

nanana-w reviewed Apr 6, 2023

View reviewed changes

docs/cluster-operations/scale-cluster.md Show resolved Hide resolved

nanana-w reviewed Apr 6, 2023

View reviewed changes

docs/cluster-operations/manage-license.md Show resolved Hide resolved

nanana-w reviewed Apr 6, 2023

View reviewed changes

docs/cluster-operations/manage-license.md Show resolved Hide resolved

nanana-w reviewed Apr 6, 2023

View reviewed changes

yuqiu123 and others added 4 commits April 9, 2023 13:54

Update docs/advanced-functions/multiple-cluster-management.md

dd04b11

Co-authored-by: yixin-chen-23 <[email protected]>

Update docs/advanced-functions/multiple-cluster-management.md

46a66a7

Co-authored-by: nanana-w <[email protected]>

Update docs/advanced-functions/multiple-cluster-management.md

c6c5cdf

Co-authored-by: nanana-w <[email protected]>

Update docs/advanced-functions/multiple-cluster-management.md

c8172f8

Co-authored-by: nanana-w <[email protected]>

gxglls reviewed Apr 10, 2023

View reviewed changes


		Increasing the number of meta pods is not supported for IOMesh Community Edition. For the Enterprise Edition, the minimum number of meta pods is 3 and the maximum number is 5.

		1. In `iomesh.yaml`, locate `meta` and then edit `replicas`.


		## Deployment

		To reduce the number of Pods required in a multi-cluster deployment of IOMesh, the management components shared by all IOMesh clusters, including the IOMesh Operator, IOMesh CSI driver, and Node Disk Manager, will be installed on the first IOMesh cluster, which is referred to as the management cluster.


		## Operations & Management

		All procedures below are listed based on the example in [Multiple Cluster Deployment](../advanced-functions/multiple-cluster-management.md#deployment).

	1. Uninstall the second IOMesh cluster, which will also delete `iomesh` and `zookeeper` components.
	1. Uninstall the non-management IOMesh cluster, which will also delete `iomesh` and `zookeeper` components.


		## Scale Meta Server

		Add meta pods when the meta server is overloaded.


		Precaution

		The minimum number of chunk pods is 3 and the maximum is 255.


		### Configure Multiple-Cluster Connection

		To enable the IOMesh CSI driver to connect to multiple IOMesh clusters, you need to configure a `ConfigMap` containing connection information for each IOMesh cluster. Before configuration, familarity with `ConfigMap` is suggested.

Cluster operations #189

Are you sure you want to change the base?

Cluster operations #189

Uh oh!

Conversation

yuqiu123 commented Mar 31, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gxglls Apr 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gxglls Apr 6, 2023 •

edited

Loading

nanana-w Apr 6, 2023 •

edited

Loading