|
| 1 | +# Upgrade Devtron to 1.5.0 |
| 2 | + |
| 3 | +This document outlines the step-by-step process to be followed before upgrading Devtron to version **1.5.0**. |
| 4 | + |
| 5 | +## Overview of the Upgrade Process |
| 6 | + |
| 7 | +The upgrade process consists of three sequential Kubernetes jobs: |
| 8 | + |
| 9 | +1. **devtron-pre-upgrade**: Prepares the environment for the upgrade. |
| 10 | +2. **devtron-upgrade-init**: Scales down Devtron and starts the migration process. |
| 11 | +3. **devtron-upgrade**: Performs the actual database migration and restores the system. |
| 12 | + |
| 13 | +After the completion of the above jobs, you may proceed to upgrade Devtron using the UI or command line. |
| 14 | + |
| 15 | +--- |
| 16 | + |
| 17 | +## Prerequisites |
| 18 | + |
| 19 | +* Ensure that you have [deployed the **devtron-backups** chart](../install/devtron-backup.md) and that at least one backup has been pushed successfully. [Click here](https://github.com/devtron-labs/charts/blob/main/charts/devtron-backups/README.md) to know more about the backups chart. |
| 20 | +* You must have administrative access to the cluster where Devtron is running, along with `kubectl` configured. |
| 21 | +* PVC creation must not be blocked by any policy. If it is, exclude the `devtroncd` namespace from it. |
| 22 | + |
| 23 | +--- |
| 24 | + |
| 25 | +## Steps |
| 26 | + |
| 27 | +### 1. Apply the 'pre-upgrade' job |
| 28 | + |
| 29 | +The `devtron-pre-upgrade` job creates the necessary resources and prepares for the database backup. |
| 30 | + |
| 31 | +```bash |
| 32 | +# Apply the devtron-pre-upgrade job |
| 33 | +kubectl apply -f https://raw.githubusercontent.com/devtron-labs/utilities/refs/heads/raw-links/scripts/postgres-upgrade/devtron-pre-upgrade.yaml |
| 34 | +``` |
| 35 | + |
| 36 | +This job will: |
| 37 | +1. Create a ConfigMap named `devtron-postgres-upgrade` in the `devtroncd` namespace. |
| 38 | +2. Determine the StorageClass and size of the existing PostgreSQL PVC. |
| 39 | +3. Create a new PVC named `devtron-db-upgrade-pvc` with additional storage (+5Gi). |
| 40 | +4. Automatically apply the upgrade-init job. |
| 41 | + |
| 42 | +To monitor the progress of this job: |
| 43 | + |
| 44 | +```bash |
| 45 | +kubectl logs -f job/devtron-pre-upgrade -n devtroncd |
| 46 | +``` |
| 47 | + |
| 48 | +Wait for this job to complete successfully before proceeding. |
| 49 | + |
| 50 | + |
| 51 | +### 2. Monitor the 'upgrade-init' job |
| 52 | + |
| 53 | +The `devtron-upgrade-init` job is automatically triggered by the `devtron-pre-upgrade` job: |
| 54 | +1. It scales down all Devtron components to ensure database consistency. |
| 55 | +2. Terminates active database connections. |
| 56 | +3. Starts the Postgres migration process. |
| 57 | + |
| 58 | +To monitor the progress of this job: |
| 59 | + |
| 60 | +```bash |
| 61 | +kubectl logs -f job/devtron-upgrade-init -n devtroncd |
| 62 | +``` |
| 63 | + |
| 64 | +Ensure this job completes successfully before proceeding to the next step. |
| 65 | + |
| 66 | + |
| 67 | +### 3. Apply the 'upgrade' job |
| 68 | + |
| 69 | +Once the backup is confirmed, apply the final upgrade job: |
| 70 | + |
| 71 | +```bash |
| 72 | +kubectl apply -f https://raw.githubusercontent.com/devtron-labs/utilities/refs/heads/raw-links/scripts/postgres-upgrade/devtron-upgrade.yaml |
| 73 | +``` |
| 74 | + |
| 75 | +This job will: |
| 76 | +1. Verify if the `devtron-upgrade-init` job was successful. |
| 77 | +2. Extract any nodeSelectors or tolerations from the existing PostgreSQL StatefulSet. |
| 78 | +3. Remove PostgreSQL 11 components. |
| 79 | +4. Install PostgreSQL 14 with the same configuration. |
| 80 | +5. Migrate the data. |
| 81 | +6. Scale up all Devtron components. |
| 82 | + |
| 83 | +To monitor the progress of this job: |
| 84 | + |
| 85 | +```bash |
| 86 | +kubectl logs -f job/devtron-upgrade -n devtroncd |
| 87 | +``` |
| 88 | + |
| 89 | +--- |
| 90 | + |
| 91 | +## Verify the Upgrade |
| 92 | + |
| 93 | +After the upgrade job completes, verify the PostgreSQL migration: |
| 94 | + |
| 95 | +```bash |
| 96 | +# Check if all pods are running |
| 97 | +kubectl get pods -n devtroncd |
| 98 | + |
| 99 | +# Verify PostgreSQL version (should now be 14) |
| 100 | +kubectl get configmap devtron-postgres-upgrade -n devtroncd -o jsonpath="{.data.POSTGRES_MIGRATED}" |
| 101 | +``` |
| 102 | + |
| 103 | +The value of `POSTGRES_MIGRATED` should be "14" if the migration was successful. |
| 104 | + |
| 105 | +--- |
| 106 | + |
| 107 | +## Potential Issues and Troubleshooting |
| 108 | + |
| 109 | +### Job Failure |
| 110 | + |
| 111 | +1. If the `devtron-upgrade-init` or the `devtron-upgrade` job fails, check the logs of job and the ConfigMap for error messages: |
| 112 | + |
| 113 | +```bash |
| 114 | +kubectl get configmap devtron-postgres-upgrade -n devtroncd -o yaml |
| 115 | +``` |
| 116 | + |
| 117 | +Look for any entries with "ERROR" in the keys. |
| 118 | + |
| 119 | +2. To reapply the devtron-upgrade-init job, delete the PVC named `devtron-db-upgrade-pvc`, recreate it with the same configurations and then reapply the `devtron-upgrade-init` job. |
| 120 | + |
| 121 | +3. If the `devtron-upgrade-init` job is in a pending state, check for the PVC named `devtron-db-upgrade-pvc`, ensure that the PVC is successfully created. |
| 122 | + |
| 123 | +--- |
| 124 | + |
| 125 | +## Next Steps |
| 126 | + |
| 127 | +Once the database migration is complete, you can proceed with upgrading the Devtron application through the UI as mentioned in the final message of the upgrade job. Alternatively, you may use the [upgrade commands](#upgrade-commands) mentioned below. |
| 128 | + |
| 129 | +### Upgrade Commands |
| 130 | + |
| 131 | +1. Update the Helm repository |
| 132 | + |
| 133 | +```bash |
| 134 | +helm repo update |
| 135 | +``` |
| 136 | + |
| 137 | +2. Run the upgrade command for Devtron |
| 138 | + |
| 139 | +```bash |
| 140 | +helm upgrade devtron devtron/devtron-operator -n devtroncd --reuse-values -f https://raw.githubusercontent.com/devtron-labs/devtron/main/charts/devtron/devtron-bom.yaml |
| 141 | +``` |
0 commit comments