From 7ff13bda2d9c88b9292df43550d00636548c0fb3 Mon Sep 17 00:00:00 2001 From: Yu Qiu Date: Wed, 9 Aug 2023 10:20:27 +0800 Subject: [PATCH 1/5] 1.0.1 upgrade cluster --- docs/advanced-functions/external-storage.md | 2 +- .../manage-multiple-cluster.md | 1 - .../setup-worker-node.md | 4 +- docs/cluster-operations/upgrade-cluster.md | 74 ++++++++++++++++--- docs/deploy-iomesh-cluster/install-iomesh.md | 20 ++++- 5 files changed, 81 insertions(+), 20 deletions(-) rename docs/{deploy-iomesh-cluster => appendices}/setup-worker-node.md (77%) diff --git a/docs/advanced-functions/external-storage.md b/docs/advanced-functions/external-storage.md index 703f7596..a1aa39f6 100644 --- a/docs/advanced-functions/external-storage.md +++ b/docs/advanced-functions/external-storage.md @@ -106,7 +106,7 @@ After the access point is configured, IOMesh can provide storage externally to a > **_NOTE_**: To use this function, the external Kubernetes cluster should be able to access IOMesh `DATA_CIDR` that was configured in [Prerequisites](../deploy-iomesh-cluster/prerequisites#network-requirements). -1. Set up [`open-iscsi`](../deploy-iomesh-cluster/setup-worker-node) on the worker nodes of the external Kubernetes cluster. +1. Set up [`open-iscsi`](../appendices/setup-worker-node) on the worker nodes of the external Kubernetes cluster. 2. Add the Helm chart repository. ```shell diff --git a/docs/advanced-functions/manage-multiple-cluster.md b/docs/advanced-functions/manage-multiple-cluster.md index d69d478b..bdfdf002 100644 --- a/docs/advanced-functions/manage-multiple-cluster.md +++ b/docs/advanced-functions/manage-multiple-cluster.md @@ -18,7 +18,6 @@ IOMesh CSI Driver is shared by all IOMesh clusters to facilitate connection, whi - Verify that all requirements in [Prerequisites](../deploy-iomesh-cluster/prerequisites.md) are met. - The IOMesh version should be 1.0.0 or above. - A Kubernetes cluster consisting of at least 6 worker nodes. -- Verify that [`open-iscsi`](../deploy-iomesh-cluster/setup-worker-node) has been set up for each worker node. **Procedure** diff --git a/docs/deploy-iomesh-cluster/setup-worker-node.md b/docs/appendices/setup-worker-node.md similarity index 77% rename from docs/deploy-iomesh-cluster/setup-worker-node.md rename to docs/appendices/setup-worker-node.md index 1942cd84..f78c056a 100644 --- a/docs/deploy-iomesh-cluster/setup-worker-node.md +++ b/docs/appendices/setup-worker-node.md @@ -4,9 +4,7 @@ title: Set Up Worker Node sidebar_label: Set Up Worker Node --- -Once you have confirmed that all requirements in [Prerequisites](../deploy-iomesh-cluster/prerequisites) are met, set up `open-iscsi` for each worker node on which IOMesh will be installed and running. - -It is important to note that if you intend for IOMesh to provide storage to other nodes within this Kubernetes cluster, you will also need to set up `open-iscsi` for those nodes. +Before setting up `open-iscsi` for the worker nodes on which IOMesh will be installed and running, ensure all requirements in [Prerequisites](../deploy-iomesh-cluster/prerequisites) are met. 1. On the node console, run the following command to install `open-iscsi`. diff --git a/docs/cluster-operations/upgrade-cluster.md b/docs/cluster-operations/upgrade-cluster.md index 9b504616..af5a9f6d 100644 --- a/docs/cluster-operations/upgrade-cluster.md +++ b/docs/cluster-operations/upgrade-cluster.md @@ -6,18 +6,19 @@ sidebar_label: Upgrade Cluster You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 either online or offline. Before proceeding, consider the following: - Upgrade is not supported if the Kubernetes cluster has only 1 meta pod or 1 chunk pod. -- Due to the limitations of the Kubernetes CRD upgrade mechanism, the IOMesh cluster upgraded to 1.0.0 from 0.11.1 cannot run on the Kubernetes cluster of version 1.25 or above. +- Due to the limitations of the Kubernetes CRD upgrade mechanism, the IOMesh cluster upgraded to this release from 0.11.1 cannot run on the Kubernetes cluster of version 1.25 or above. > _NOTE:_ > There might be temporary I/O latency fluctuations during the upgrade. -**Procedure** +## Upgrade IOMesh from 0.11.1 to 1.0.1 + 1. Delete the default StorageClass. - IOMesh 1.0.0 has a different default StorageClass with updated parameters compared to the previous version. You just need to delete the old one and any PVC using it will not be impacted. + This release of IOMesh has a different default StorageClass with updated parameters compared to the previous version. You just need to delete the old one and any PVC using it will not be impacted. ```shell kubectl delete sc iomesh-csi-driver @@ -27,14 +28,14 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e ```shell kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration ``` -3. Install the CRD of IOMesh 1.0.0. +3. Install the CRD of IOMesh 1.0.1. ```shell kubectl apply -f https://iomesh.run/config/crd/iomesh.com_blockdevicemonitors.yaml ``` -4. Get the new fields and values added in IOMesh 1.0.0. +4. Get the new fields and values added in IOMesh 1.0.1. ```shell - wget https://iomesh.run/config/merge-values/v1.0.0.yaml -O merge-values.yaml + wget https://iomesh.run/config/merge-values/v1.0.1.yaml -O merge-values.yaml ``` 5. Edit the YAML file `merge-values.yaml`. Set the value of the `iomesh.edition` field to match the edition specified for the previous version of IOMesh. ```yaml @@ -45,9 +46,8 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e 6. Upgrade the IOMesh cluster, which will merge new fields and values while keeping existing ones. Then wait for a few minutes till all pods are running. ```bash - helm upgrade --namespace iomesh-system iomesh iomesh/iomesh --version v1.0.0 --reuse-values -f merge-values.yaml + helm upgrade --namespace iomesh-system iomesh iomesh/iomesh --version v1.0.1 --reuse-values -f merge-values.yaml ``` - 7. Verify that all pods are in `Running` state. If so, then IOMesh has been successfully upgraded. ```bash watch kubectl get pod --namespace iomesh-system @@ -56,7 +56,7 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e 1. Delete the default StorageClass. - IOMesh 1.0.0 has a different default StorageClass with updated parameters compared to the previous version. You just need to delete the old one and any PVC using it will not be impacted. + This release of IOMesh has a different default StorageClass with updated parameters compared to the previous version. You just need to delete the old one and any PVC using it will not be impacted. ```shell kubectl delete sc iomesh-csi-driver @@ -67,7 +67,9 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e ```shell kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration ``` -3. Download [IOMesh Offline Installation Package](../appendices/downloads). Make sure to replace `` with `v1.0.0` and `` based on your CPU architecture. +3. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to extract it. + + Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. - Hygon x86_64: `hygon-amd64` - Intel x86_64: `amd64` - Kunpeng AArch64: `arm64` @@ -75,7 +77,7 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e ```shell tar -xf iomesh-offline--.tgz && cd iomesh-offline ``` -4. Install the CRD of IOMesh 1.0.0. +4. Install the CRD of IOMesh 1.0.1. ```shell kubectl apply -f ./configs/iomesh.com_blockdevicemonitors.yaml @@ -99,3 +101,53 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e watch kubectl get pod --namespace iomesh-system ``` + +### Upgrade IOMesh from 1.0.0 to 1.0.1 + + + +1. Temporarily disable IOMesh Webhook to avoid upgrade failure. Once the upgrade is successful, it will be automatically enabled again. + + ```shell + kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration + ``` + +2. Upgrade the IOMesh cluster, which will merge new fields and values while keeping existing ones. Then wait for a few minutes till all pods are running. + + ```bash + helm upgrade --namespace iomesh-system iomesh iomesh/iomesh --version v1.0.1 --reuse-values -f merge-values.yaml + ``` + +3. Verify that all pods are in `Running` state. If so, then IOMesh has been successfully upgraded. + ```bash + watch kubectl get pod --namespace iomesh-system + ``` + + +1. Temporarily disable IOMesh Webhook to avoid upgrade failure. Once the upgrade is successful, it will be automatically enabled again. + + ```shell + kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration + ``` +2. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to extract it. + + Make sure to replace `` with `v1.0.0` and `` based on your CPU architecture. + - Hygon x86_64: `hygon-amd64` + - Intel x86_64: `amd64` + - Kunpeng AArch64: `arm64` + + ```shell + tar -xf iomesh-offline--.tgz && cd iomesh-offline + ``` + +3. Upgrade the IOMesh cluster, which will merge new fields and values while keeping existing ones. Then wait for a few minutes till all pods are running. + + ```bash + ./helm upgrade --namespace iomesh-system iomesh ./charts/iomesh --reuse-values -f ./configs/merge-values.yaml + ``` + +4. Verify that all pods are in the `Running` state. If so, then IOMesh has been successfully upgraded. + ```bash + watch kubectl get pod --namespace iomesh-system + ``` + diff --git a/docs/deploy-iomesh-cluster/install-iomesh.md b/docs/deploy-iomesh-cluster/install-iomesh.md index e6536800..4de82d10 100644 --- a/docs/deploy-iomesh-cluster/install-iomesh.md +++ b/docs/deploy-iomesh-cluster/install-iomesh.md @@ -29,7 +29,7 @@ Before installing IOMesh, refer to the following to choose how you install IOMes ```shell # The IP address of each worker node running IOMesh must be within the same IOMESH_DATA_CIDR. - export IOMESH_DATA_CIDR=10.234.1.0/24; curl -sSL https://iomesh.run/install_iomesh.sh | sh - + export IOMESH_DATA_CIDR=10.234.1.0/24; curl -sSL https://iomesh.run/install_iomesh.sh | bash - ``` 3. Verify that all pods are in `Running` state. If so, then IOMesh has been successfully installed. @@ -129,7 +129,18 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon It is recommended that you only configure `values`. For more configurations, refer to [Pod Affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity). -6. On the master node, deploy the IOMesh cluster. + - An optional step. If a Pod is mounted with an RWO-type PVC created by IOMesh, you have the option to configure the `podDeletePolicy` field to automatically delete and rebuild the Pod on another healthy worker node if the Kubernetes node it resides in fails. + + If left unspecified, this field is set to `no-delete-pod` by default, indicating that the system won't automatically delete and rebuild the Pod in case of node failure. + ```yaml + csi-driver: + driver: + controller: + driver: + podDeletePolicy: "no-delete-pod" # support: no-delete-pod / delete-deployment-pod / delete-statefulset-pod / delete-both-statefulset-and-deployment-pod + ``` + +1. On the master node, deploy the IOMesh cluster. ```shell helm install iomesh iomesh/iomesh \ @@ -150,7 +161,7 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon TEST SUITE: None ``` -7. Verify that all pods are in `Running` state. If so, then IOMesh has been installed successfully. +2. Verify that all pods are in `Running` state. If so, then IOMesh has been installed successfully. ```bash kubectl --namespace iomesh-system get pods @@ -368,7 +379,8 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon operator-85877979-s94vz 1/1 Running 0 2m8s operator-85877979-xqtml 1/1 Running 0 2m8s ``` - + > _NOTE:_ + > After installing IOMesh, the `prepare-csi` Pod will be automatically launched on all schedulable nodes in the IOMesh cluster to install and set up `open-iscsi`. The Pod will be cleaned up automatically once `open-iscsi` is installed and configured. In case the `prepare-csi` Pod fails and exits on a specific node, you need to manually [set up open-iscsi](../appendices/setup-worker-node) to identify the reason for the installation failure. From 51a7ff4730160eb65a0949f211f909a63bc50716 Mon Sep 17 00:00:00 2001 From: Yu Qiu Date: Wed, 9 Aug 2023 10:20:27 +0800 Subject: [PATCH 2/5] add content about 1.0.1 upgrade cluster --- docs/advanced-functions/external-storage.md | 2 +- .../manage-multiple-cluster.md | 1 - .../setup-worker-node.md | 4 +- docs/cluster-operations/upgrade-cluster.md | 72 ++++++++++++++++--- docs/deploy-iomesh-cluster/install-iomesh.md | 42 ++++++++--- 5 files changed, 97 insertions(+), 24 deletions(-) rename docs/{deploy-iomesh-cluster => appendices}/setup-worker-node.md (77%) diff --git a/docs/advanced-functions/external-storage.md b/docs/advanced-functions/external-storage.md index 703f7596..a1aa39f6 100644 --- a/docs/advanced-functions/external-storage.md +++ b/docs/advanced-functions/external-storage.md @@ -106,7 +106,7 @@ After the access point is configured, IOMesh can provide storage externally to a > **_NOTE_**: To use this function, the external Kubernetes cluster should be able to access IOMesh `DATA_CIDR` that was configured in [Prerequisites](../deploy-iomesh-cluster/prerequisites#network-requirements). -1. Set up [`open-iscsi`](../deploy-iomesh-cluster/setup-worker-node) on the worker nodes of the external Kubernetes cluster. +1. Set up [`open-iscsi`](../appendices/setup-worker-node) on the worker nodes of the external Kubernetes cluster. 2. Add the Helm chart repository. ```shell diff --git a/docs/advanced-functions/manage-multiple-cluster.md b/docs/advanced-functions/manage-multiple-cluster.md index d69d478b..bdfdf002 100644 --- a/docs/advanced-functions/manage-multiple-cluster.md +++ b/docs/advanced-functions/manage-multiple-cluster.md @@ -18,7 +18,6 @@ IOMesh CSI Driver is shared by all IOMesh clusters to facilitate connection, whi - Verify that all requirements in [Prerequisites](../deploy-iomesh-cluster/prerequisites.md) are met. - The IOMesh version should be 1.0.0 or above. - A Kubernetes cluster consisting of at least 6 worker nodes. -- Verify that [`open-iscsi`](../deploy-iomesh-cluster/setup-worker-node) has been set up for each worker node. **Procedure** diff --git a/docs/deploy-iomesh-cluster/setup-worker-node.md b/docs/appendices/setup-worker-node.md similarity index 77% rename from docs/deploy-iomesh-cluster/setup-worker-node.md rename to docs/appendices/setup-worker-node.md index 1942cd84..f78c056a 100644 --- a/docs/deploy-iomesh-cluster/setup-worker-node.md +++ b/docs/appendices/setup-worker-node.md @@ -4,9 +4,7 @@ title: Set Up Worker Node sidebar_label: Set Up Worker Node --- -Once you have confirmed that all requirements in [Prerequisites](../deploy-iomesh-cluster/prerequisites) are met, set up `open-iscsi` for each worker node on which IOMesh will be installed and running. - -It is important to note that if you intend for IOMesh to provide storage to other nodes within this Kubernetes cluster, you will also need to set up `open-iscsi` for those nodes. +Before setting up `open-iscsi` for the worker nodes on which IOMesh will be installed and running, ensure all requirements in [Prerequisites](../deploy-iomesh-cluster/prerequisites) are met. 1. On the node console, run the following command to install `open-iscsi`. diff --git a/docs/cluster-operations/upgrade-cluster.md b/docs/cluster-operations/upgrade-cluster.md index 9b504616..c3317416 100644 --- a/docs/cluster-operations/upgrade-cluster.md +++ b/docs/cluster-operations/upgrade-cluster.md @@ -6,18 +6,19 @@ sidebar_label: Upgrade Cluster You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 either online or offline. Before proceeding, consider the following: - Upgrade is not supported if the Kubernetes cluster has only 1 meta pod or 1 chunk pod. -- Due to the limitations of the Kubernetes CRD upgrade mechanism, the IOMesh cluster upgraded to 1.0.0 from 0.11.1 cannot run on the Kubernetes cluster of version 1.25 or above. +- Due to the limitations of the Kubernetes CRD upgrade mechanism, the IOMesh cluster upgraded to this release from 0.11.1 cannot run on the Kubernetes cluster of version 1.25 or above. > _NOTE:_ > There might be temporary I/O latency fluctuations during the upgrade. -**Procedure** +## Upgrade IOMesh from 0.11.1 to 1.0.1 + 1. Delete the default StorageClass. - IOMesh 1.0.0 has a different default StorageClass with updated parameters compared to the previous version. You just need to delete the old one and any PVC using it will not be impacted. + This release of IOMesh has a different default StorageClass with updated parameters compared to the previous version. You just need to delete the old one and any PVC using it will not be impacted. ```shell kubectl delete sc iomesh-csi-driver @@ -27,14 +28,14 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e ```shell kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration ``` -3. Install the CRD of IOMesh 1.0.0. +3. Install the CRD of IOMesh 1.0.1. ```shell kubectl apply -f https://iomesh.run/config/crd/iomesh.com_blockdevicemonitors.yaml ``` -4. Get the new fields and values added in IOMesh 1.0.0. +4. Get the new fields and values added in IOMesh 1.0.1. ```shell - wget https://iomesh.run/config/merge-values/v1.0.0.yaml -O merge-values.yaml + wget https://iomesh.run/config/merge-values/v1.0.1.yaml -O merge-values.yaml ``` 5. Edit the YAML file `merge-values.yaml`. Set the value of the `iomesh.edition` field to match the edition specified for the previous version of IOMesh. ```yaml @@ -45,9 +46,8 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e 6. Upgrade the IOMesh cluster, which will merge new fields and values while keeping existing ones. Then wait for a few minutes till all pods are running. ```bash - helm upgrade --namespace iomesh-system iomesh iomesh/iomesh --version v1.0.0 --reuse-values -f merge-values.yaml + helm upgrade --namespace iomesh-system iomesh iomesh/iomesh --version v1.0.1 --reuse-values -f merge-values.yaml ``` - 7. Verify that all pods are in `Running` state. If so, then IOMesh has been successfully upgraded. ```bash watch kubectl get pod --namespace iomesh-system @@ -56,7 +56,7 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e 1. Delete the default StorageClass. - IOMesh 1.0.0 has a different default StorageClass with updated parameters compared to the previous version. You just need to delete the old one and any PVC using it will not be impacted. + This release of IOMesh has a different default StorageClass with updated parameters compared to the previous version. You just need to delete the old one and any PVC using it will not be impacted. ```shell kubectl delete sc iomesh-csi-driver @@ -67,7 +67,7 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e ```shell kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration ``` -3. Download [IOMesh Offline Installation Package](../appendices/downloads). Make sure to replace `` with `v1.0.0` and `` based on your CPU architecture. +3. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to unpack it. Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. - Hygon x86_64: `hygon-amd64` - Intel x86_64: `amd64` - Kunpeng AArch64: `arm64` @@ -75,7 +75,7 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e ```shell tar -xf iomesh-offline--.tgz && cd iomesh-offline ``` -4. Install the CRD of IOMesh 1.0.0. +4. Install the CRD of IOMesh 1.0.1. ```shell kubectl apply -f ./configs/iomesh.com_blockdevicemonitors.yaml @@ -99,3 +99,53 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e watch kubectl get pod --namespace iomesh-system ``` + +### Upgrade IOMesh from 1.0.0 to 1.0.1 + + + +1. Temporarily disable IOMesh Webhook to avoid upgrade failure. Once the upgrade is successful, it will be automatically enabled again. + + ```shell + kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration + ``` + +2. Upgrade the IOMesh cluster, which will merge new fields and values while keeping existing ones. Then wait for a few minutes till all pods are running. + + ```bash + helm upgrade --namespace iomesh-system iomesh iomesh/iomesh --version v1.0.1 --reuse-values -f merge-values.yaml + ``` + +3. Verify that all pods are in `Running` state. If so, then IOMesh has been successfully upgraded. + ```bash + watch kubectl get pod --namespace iomesh-system + ``` + + +1. Temporarily disable IOMesh Webhook to avoid upgrade failure. Once the upgrade is successful, it will be automatically enabled again. + + ```shell + kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration + ``` +2. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to extract it. + + Make sure to replace `` with `v1.0.0` and `` based on your CPU architecture. + - Hygon x86_64: `hygon-amd64` + - Intel x86_64: `amd64` + - Kunpeng AArch64: `arm64` + + ```shell + tar -xf iomesh-offline--.tgz && cd iomesh-offline + ``` + +3. Upgrade the IOMesh cluster, which will merge new fields and values while keeping existing ones. Then wait for a few minutes till all pods are running. + + ```bash + ./helm upgrade --namespace iomesh-system iomesh ./charts/iomesh --reuse-values -f ./configs/merge-values.yaml + ``` + +4. Verify that all pods are in the `Running` state. If so, then IOMesh has been successfully upgraded. + ```bash + watch kubectl get pod --namespace iomesh-system + ``` + diff --git a/docs/deploy-iomesh-cluster/install-iomesh.md b/docs/deploy-iomesh-cluster/install-iomesh.md index e6536800..3cb164e3 100644 --- a/docs/deploy-iomesh-cluster/install-iomesh.md +++ b/docs/deploy-iomesh-cluster/install-iomesh.md @@ -10,7 +10,7 @@ Before installing IOMesh, refer to the following to choose how you install IOMes - Custom Installation: Configure parameters during installation, but ensure that your Kubernetes cluster is connected to the public network. - Offline Installation: Recommended for Kubernetes clusters with no public network connectivity and supports custom parameter configuration during installation. -## Quick Installation +## One-Click Online Installation **Prerequisite** - The CPU architecture of the Kubernetes cluster must be Intel x86_64 or Kunpeng AArch64. @@ -29,7 +29,7 @@ Before installing IOMesh, refer to the following to choose how you install IOMes ```shell # The IP address of each worker node running IOMesh must be within the same IOMESH_DATA_CIDR. - export IOMESH_DATA_CIDR=10.234.1.0/24; curl -sSL https://iomesh.run/install_iomesh.sh | sh - + export IOMESH_DATA_CIDR=10.234.1.0/24; curl -sSL https://iomesh.run/install_iomesh.sh | bash - ``` 3. Verify that all pods are in `Running` state. If so, then IOMesh has been successfully installed. @@ -40,7 +40,9 @@ Before installing IOMesh, refer to the following to choose how you install IOMes > _NOTE:_ IOMesh resources left by running the above commands will be saved for troubleshooting if any error occurs during installation. You can run the command `curl -sSL https://iomesh.run/uninstall_iomesh.sh | sh -` to remove all IOMesh resources from the Kubernetes cluster. -## Custom Installation + > _NOTE:_ After installing IOMesh, the `prepare-csi` Pod will be automatically launched on all schedulable nodes in the Kubernetes cluster to install and set up `open-iscsi`. The Pod will be cleaned up automatically once `open-iscsi` is installed and configured. In case the `prepare-csi` Pod fails and exits on a specific node, you need to manually [set up open-iscsi](../appendices/setup-worker-node) to identify the reason for the installation failure. + +## Custom Online Installation **Prerequisite** @@ -129,7 +131,18 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon It is recommended that you only configure `values`. For more configurations, refer to [Pod Affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity). -6. On the master node, deploy the IOMesh cluster. + - An optional step. If a Pod is mounted with an RWO-type PVC created by IOMesh, you have the option to configure the `podDeletePolicy` field to automatically delete and rebuild the Pod on another healthy worker node if the Kubernetes node it resides in fails. + + If left unspecified, this field is set to `no-delete-pod` by default, indicating that the system won't automatically delete and rebuild the Pod in case of node failure. + ```yaml + csi-driver: + driver: + controller: + driver: + podDeletePolicy: "no-delete-pod" # Supports "no-delete-pod", "delete-deployment-pod", "delete-statefulset-pod", or "delete-both-statefulset-and-deployment-pod". + ``` + +1. On the master node, deploy the IOMesh cluster. ```shell helm install iomesh iomesh/iomesh \ @@ -150,7 +163,7 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon TEST SUITE: None ``` -7. Verify that all pods are in `Running` state. If so, then IOMesh has been installed successfully. +2. Verify that all pods are in `Running` state. If so, then IOMesh has been installed successfully. ```bash kubectl --namespace iomesh-system get pods @@ -201,8 +214,9 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon operator-87bb89877-kfs9d 1/1 Running 0 3m23s operator-87bb89877-z9tfr 1/1 Running 0 3m23s ``` - -## Offline Installation + > _NOTE:_ + > After installing IOMesh, the `prepare-csi` Pod will be automatically launched on all schedulable nodes in the Kubernetes cluster to install and set up `open-iscsi`. The Pod will be cleaned up automatically once `open-iscsi` is installed and configured. In case the `prepare-csi` Pod fails and exits on a specific node, you need to manually [set up open-iscsi](../appendices/setup-worker-node) to identify the reason for the installation failure. +## Custom Offline Installation **Prerequisite** @@ -303,6 +317,17 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon ``` It is recommended that you only configure `values`. For more configurations, refer to [Pod Affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity). + - An optional step. If a Pod is mounted with an RWO-type PVC created by IOMesh, you have the option to configure the `podDeletePolicy` field to automatically delete and rebuild the Pod on another healthy worker node if the Kubernetes node it resides in fails. + + If left unspecified, this field is set to `no-delete-pod` by default, indicating that the system won't automatically delete and rebuild the Pod in case of node failure. + ```yaml + csi-driver: + driver: + controller: + driver: + podDeletePolicy: "no-delete-pod" # Supports "no-delete-pod", "delete-deployment-pod", "delete-statefulset-pod", or "delete-both-statefulset-and-deployment-pod". + ``` + 6. On the master node, deploy the IOMesh cluster. ```shell @@ -368,7 +393,8 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon operator-85877979-s94vz 1/1 Running 0 2m8s operator-85877979-xqtml 1/1 Running 0 2m8s ``` - + > _NOTE:_ + > After installing IOMesh, the `prepare-csi` Pod will be automatically launched on all schedulable nodes in the Kubernetes cluster to install and set up `open-iscsi`. The Pod will be cleaned up automatically once `open-iscsi` is installed and configured. In case the `prepare-csi` Pod fails and exits on a specific node, you need to manually [set up open-iscsi](../appendices/setup-worker-node) to identify the reason for the installation failure. From 738d92af6be3eb18b979164ec59c91482b4d6a92 Mon Sep 17 00:00:00 2001 From: Yu Qiu Date: Wed, 9 Aug 2023 15:41:54 +0800 Subject: [PATCH 3/5] new format fix --- docs/cluster-operations/upgrade-cluster.md | 6 ++---- docs/deploy-iomesh-cluster/install-iomesh.md | 6 +++--- website/sidebars.json | 2 +- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/docs/cluster-operations/upgrade-cluster.md b/docs/cluster-operations/upgrade-cluster.md index c3317416..ae1ea878 100644 --- a/docs/cluster-operations/upgrade-cluster.md +++ b/docs/cluster-operations/upgrade-cluster.md @@ -67,7 +67,7 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e ```shell kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration ``` -3. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to unpack it. Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. +3. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to unpack it. Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Custom Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. - Hygon x86_64: `hygon-amd64` - Intel x86_64: `amd64` - Kunpeng AArch64: `arm64` @@ -127,9 +127,7 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e ```shell kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration ``` -2. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to extract it. - - Make sure to replace `` with `v1.0.0` and `` based on your CPU architecture. +2. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to unpack it. Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Custom Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. - Hygon x86_64: `hygon-amd64` - Intel x86_64: `amd64` - Kunpeng AArch64: `arm64` diff --git a/docs/deploy-iomesh-cluster/install-iomesh.md b/docs/deploy-iomesh-cluster/install-iomesh.md index 23ce1062..be7d4377 100644 --- a/docs/deploy-iomesh-cluster/install-iomesh.md +++ b/docs/deploy-iomesh-cluster/install-iomesh.md @@ -6,9 +6,9 @@ sidebar_label: Install IOMesh Before installing IOMesh, refer to the following to choose how you install IOMesh. -- Quick Installation: One-click online installation with default parameter values that cannot be modified. -- Custom Installation: Configure parameters during installation, but ensure that your Kubernetes cluster is connected to the public network. -- Offline Installation: Recommended for Kubernetes clusters with no public network connectivity and supports custom parameter configuration during installation. +- One-Click Online Installation: One-click online installation with default parameter values that cannot be modified. +- Custom Online Installation: Configure parameters during installation, but ensure that your Kubernetes cluster is connected to the public network. +- Offline Offline Installation: Recommended for Kubernetes clusters with no public network connectivity and supports custom parameter configuration during installation. ## One-Click Online Installation diff --git a/website/sidebars.json b/website/sidebars.json index b8b66e14..c801c597 100644 --- a/website/sidebars.json +++ b/website/sidebars.json @@ -6,7 +6,6 @@ ], "Deploy IOMesh": [ "deploy-iomesh-cluster/prerequisites", - "deploy-iomesh-cluster/setup-worker-node", "deploy-iomesh-cluster/install-iomesh", "deploy-iomesh-cluster/setup-iomesh", "deploy-iomesh-cluster/activate-license" @@ -45,6 +44,7 @@ ], "Appendices": [ "appendices/release-notes", + "appendices/setup-worker-node", "appendices/downloads", "appendices/iomesh-metrics", "appendices/faq" From 5d5dffa4f7d9418abbf9c93ddf07c4877b192deb Mon Sep 17 00:00:00 2001 From: Yu Qiu Date: Wed, 9 Aug 2023 15:41:54 +0800 Subject: [PATCH 4/5] added new content of open iscsi new format fix --- docs/cluster-operations/upgrade-cluster.md | 6 ++---- docs/deploy-iomesh-cluster/install-iomesh.md | 6 +++--- website/sidebars.json | 2 +- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/docs/cluster-operations/upgrade-cluster.md b/docs/cluster-operations/upgrade-cluster.md index c3317416..ae1ea878 100644 --- a/docs/cluster-operations/upgrade-cluster.md +++ b/docs/cluster-operations/upgrade-cluster.md @@ -67,7 +67,7 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e ```shell kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration ``` -3. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to unpack it. Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. +3. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to unpack it. Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Custom Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. - Hygon x86_64: `hygon-amd64` - Intel x86_64: `amd64` - Kunpeng AArch64: `arm64` @@ -127,9 +127,7 @@ You have the option to upgrade the IOMesh cluster from version 0.11.1 to 1.0.0 e ```shell kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration ``` -2. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to extract it. - - Make sure to replace `` with `v1.0.0` and `` based on your CPU architecture. +2. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to unpack it. Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Custom Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. - Hygon x86_64: `hygon-amd64` - Intel x86_64: `amd64` - Kunpeng AArch64: `arm64` diff --git a/docs/deploy-iomesh-cluster/install-iomesh.md b/docs/deploy-iomesh-cluster/install-iomesh.md index 23ce1062..be7d4377 100644 --- a/docs/deploy-iomesh-cluster/install-iomesh.md +++ b/docs/deploy-iomesh-cluster/install-iomesh.md @@ -6,9 +6,9 @@ sidebar_label: Install IOMesh Before installing IOMesh, refer to the following to choose how you install IOMesh. -- Quick Installation: One-click online installation with default parameter values that cannot be modified. -- Custom Installation: Configure parameters during installation, but ensure that your Kubernetes cluster is connected to the public network. -- Offline Installation: Recommended for Kubernetes clusters with no public network connectivity and supports custom parameter configuration during installation. +- One-Click Online Installation: One-click online installation with default parameter values that cannot be modified. +- Custom Online Installation: Configure parameters during installation, but ensure that your Kubernetes cluster is connected to the public network. +- Offline Offline Installation: Recommended for Kubernetes clusters with no public network connectivity and supports custom parameter configuration during installation. ## One-Click Online Installation diff --git a/website/sidebars.json b/website/sidebars.json index b8b66e14..c801c597 100644 --- a/website/sidebars.json +++ b/website/sidebars.json @@ -6,7 +6,6 @@ ], "Deploy IOMesh": [ "deploy-iomesh-cluster/prerequisites", - "deploy-iomesh-cluster/setup-worker-node", "deploy-iomesh-cluster/install-iomesh", "deploy-iomesh-cluster/setup-iomesh", "deploy-iomesh-cluster/activate-license" @@ -45,6 +44,7 @@ ], "Appendices": [ "appendices/release-notes", + "appendices/setup-worker-node", "appendices/downloads", "appendices/iomesh-metrics", "appendices/faq" From 52f8aa3d409a4e0c5ded82c0698ba6f67dbacdb0 Mon Sep 17 00:00:00 2001 From: Yu Qiu Date: Tue, 22 Aug 2023 10:34:01 +0800 Subject: [PATCH 5/5] added test comments --- docs/advanced-functions/localpv-manager.md | 2 +- docs/cluster-operations/upgrade-cluster.md | 9 +++++++-- docs/deploy-iomesh-cluster/install-iomesh.md | 2 +- docs/deploy-iomesh-cluster/setup-iomesh.md | 5 ++--- 4 files changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/advanced-functions/localpv-manager.md b/docs/advanced-functions/localpv-manager.md index ee024676..f37eed9c 100644 --- a/docs/advanced-functions/localpv-manager.md +++ b/docs/advanced-functions/localpv-manager.md @@ -275,7 +275,7 @@ The following example assumes that the local PV is created in the `/var/iomesh/l | `parameters.csi.storage.k8s.io/fstype ` | The filesystem type when the `volumeMode` is set to `Filesystem`, which defaults to `ext4`. | | `volumeBindingMode` | Controls when volume binding and dynamic provisioning should occur. IOMesh only supports `WaitForFirstConsumer`. | - When creating a StorageClass, you have the option to configure `deviceSelector` to filter disks as desired. For configuration details, refer to [`Device Selector`](../deploy-iomesh-cluster/setup-iomesh.md) and [Kubernetes Labels and Selectors](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). + When creating a StorageClass, you have the option to configure `deviceSelector` to filter disks as desired. For configuration details, refer to [`Device Selector`](../deploy-iomesh-cluster/install-iomesh) and [Kubernetes Labels and Selectors](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). For example, `iomesh.com/bd-driveType: SSD` means the StorageClass will only filter SSD for creating local PVs. diff --git a/docs/cluster-operations/upgrade-cluster.md b/docs/cluster-operations/upgrade-cluster.md index 4ff3bc75..b7ec5982 100644 --- a/docs/cluster-operations/upgrade-cluster.md +++ b/docs/cluster-operations/upgrade-cluster.md @@ -66,7 +66,9 @@ You have the option to upgrade the IOMesh cluster either online or offline. Befo ```shell kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration ``` -3. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to unpack it. Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Custom Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. +3. Download [IOMesh Offline Installation Package](../appendices/downloads) on each worker node and the master node. Then run the following command to unpack the installation package on each worker node and the master node. + + Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Custom Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. - Hygon x86_64: `hygon-amd64` - Intel x86_64: `amd64` - Kunpeng AArch64: `arm64` @@ -126,7 +128,10 @@ You have the option to upgrade the IOMesh cluster either online or offline. Befo ```shell kubectl delete Validatingwebhookconfigurations iomesh-validating-webhook-configuration ``` -2. Download [IOMesh Offline Installation Package](../appendices/downloads) and run the following command to unpack it. Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Custom Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. +3. Download [IOMesh Offline Installation Package](../appendices/downloads) on each worker node and the master node. Then run the following command to unpack the installation package on each worker node and the master node. + + Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. Then refer to [Custom Offline Installation](../deploy-iomesh-cluster/install-iomesh.md#offline-installation) to load the IOMesh image on each worker node. + - Hygon x86_64: `hygon-amd64` - Intel x86_64: `amd64` - Kunpeng AArch64: `arm64` diff --git a/docs/deploy-iomesh-cluster/install-iomesh.md b/docs/deploy-iomesh-cluster/install-iomesh.md index 3e1f3b54..a3777aee 100644 --- a/docs/deploy-iomesh-cluster/install-iomesh.md +++ b/docs/deploy-iomesh-cluster/install-iomesh.md @@ -229,7 +229,7 @@ Make sure the CPU architecture of your Kubernetes cluster is Intel x86_64, Hygon 1. Download the [IOMesh Offline Installation Package](../appendices/downloads) on each worker node and the master node, based on your CPU architecture. -2. Unpack the installation package on each worker node and the master node. Make sure to replace `` with `v1.0.0` and `` based on your CPU architecture. +2. Unpack the installation package on each worker node and the master node. Make sure to replace `` with `v1.0.1` and `` based on your CPU architecture. - Hygon x86_64: `hygon-amd64` - Intel x86_64: `amd64` - Kunpeng AArch64: `arm64` diff --git a/docs/deploy-iomesh-cluster/setup-iomesh.md b/docs/deploy-iomesh-cluster/setup-iomesh.md index e96d4f7c..b2b4e71a 100644 --- a/docs/deploy-iomesh-cluster/setup-iomesh.md +++ b/docs/deploy-iomesh-cluster/setup-iomesh.md @@ -86,7 +86,7 @@ Before configuring device map, familiarize yourself with the mount type and devi **Device Selector** |Parameter|Value|Description| |---|---|---| -|selector | [metav1.LabelSelector](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#labelselector-v1-meta) | The label selector to filter block devices. | +|selector | [metav1.LabelSelector](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#labelselector-v1-meta) | The label selector to filter block devices. | |exclude|[block-device-name]| The block device to be excluded. | For more information, refer to [Kubernetes Labels and Selectors](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). @@ -104,7 +104,7 @@ For more information, refer to [Kubernetes Labels and Selectors](https://kuberne spec: chunk: ``` -2. Configure `deviceMap`. Specifically, copy and paste the `deviceMap` content from the following sample code and fill in fields `mount-type`, `matchLabels` or `matchExpressions`, and `exclude` based on your deployment mode and block device information. Label information `` and `` can be obtained from Step 2 in [View Block Device Objects](#view-block-device-objects). +2. Configure `deviceMap`. Specifically, copy and paste the `deviceMap` content from the following sample code and fill in fields `mount-type`, `matchLabels`, `matchExpressions`, and `exclude` based on your deployment mode and block device information. Label information `` and `` can be obtained from Step 2 in [View Block Device Objects](#view-block-device-objects). > _NOTE:_ The field `FSTYPE` of each IOMesh block device should be blank. Make sure to exclude the block device that has a specified filesystem. > _NOTE:_ It is strictly prohibited to use disk names in the `deviceMap` in a production environment. @@ -188,7 +188,6 @@ spec: values: - HDD exclude: - - blockdevice-a6652946c90d5c3fca5ca452aac5b826 - blockdevice-f001933979aa613a9c32e552d05a704a # ... ```