update besu and deprecate besu nat (#248)

Consensys · Dec 17, 2024 · 2f5dbe3 · 2f5dbe3
1 parent f5c2ee2
commit 2f5dbe3
Show file tree

Hide file tree

Showing 53 changed files with 132 additions and 2,149 deletions.
diff --git a/README.md b/README.md
@@ -41,7 +41,7 @@ The current repo layout is:
   │   └── scripts                   # azure scripts to install CSI drivers on the AKS cluster and the like
   ├── playground                    # playground for users to get familiar with concepts and how to run and tweak things - START HERE 
   │   └── kubectl
-  │       ├── quorum-besu           # use Hyperledger Besu as the block chain client
+  │       ├── quorum-besu           # use Besu as the block chain client
   │       │   ├── clique
   │       │   │   ├── ...           # templates, config etc hidden here for brevity
   │       │   ├── ethash
@@ -95,7 +95,7 @@ When deploying a private network, eg: QBFT, if you use bootnodes, you need to en
 
 You need to ensure that the genesis file is accessible to **all** nodes joining the network.
 
-Hyperledger Besu supports [NAT mechanisms](https://besu.hyperledger.org/en/stable/Reference/CLI/CLI-Syntax/#nat-method) and the default is set to automatically handle NAT environments. If you experience issues with NAT and logs have messages that have the NATService throwing exceptions connecting to external IPs, please add this option in your Besu deployments `--nat-method = NONE`
+Besu has deprecated [NAT for K8S](https://github.com/hyperledger/besu/blob/main/CHANGELOG.md) with version 24.12.0 and we recommend setting `--nat-method = NONE`
 
 #### Data Volumes:
 We use separate data volumes to store the blockchain data, over the default of the host nodes. This is similar to using separate volumes to store data when using docker containers natively or via docker-compose. This is done for a couple of reasons; firstly, containers are mortal and we don't want to store data on them, secondly, host nodes can fail and we would like the chain data to persist.  

diff --git a/aws/README.md b/aws/README.md
@@ -2,7 +2,7 @@
 
 ## Background
 
-The following is meant to guide you through running Hyperledger Besu or GoQuorum clients in AWS EKS (Kubernetes) in both development and production scenarios. As always you are free to customize the charts to suit your requirements. It is highly recommended to familiarize yourself with EKS (or equivalent Kubernetes infrastructure) before running things in production on Kubernetes.
+The following is meant to guide you through running Besu or GoQuorum clients in AWS EKS (Kubernetes) in both development and production scenarios. As always you are free to customize the charts to suit your requirements. It is highly recommended to familiarize yourself with EKS (or equivalent Kubernetes infrastructure) before running things in production on Kubernetes.
 
 It essentially comprises base infrastructure that is used to build the cluster & other resources in AWS via a [template]('./templates/cluster.yml'). We also make use some AWS native services and features after the cluster is created. These include:
 
@@ -66,6 +66,6 @@ The `cluster.yml` file that is included in this folder uses the EBS drivers but
 
 6. Deploy the charts as per the `helm` folder readme files
 
-The following is meant to guide you through running Hyperledger Besu or GoQuorum clients in AWS EKS (Kubernetes). As always you are free to customize the charts to suit your requirements. It is highly recommended that you familiarize yourself with EKS (or equivalent Kubernetes infrastructure) before running things in production on Kubernetes.
+The following is meant to guide you through running Besu or GoQuorum clients in AWS EKS (Kubernetes). As always you are free to customize the charts to suit your requirements. It is highly recommended that you familiarize yourself with EKS (or equivalent Kubernetes infrastructure) before running things in production on Kubernetes.
 
 7. Optionally deploy the kubernetes [dashboard](./templates/k8s-dashboard/README.md)
diff --git a/aws/templates/cluster.yml b/aws/templates/cluster.yml
@@ -5,7 +5,7 @@ kind: ClusterConfig
 metadata:
   name: quorum-cluster
   region: ap-southeast-2
-  version: "1.23"
+  version: "1.31"
 
 # It is recommended that you let eksctl deploy a new VPC for your cluster, it will automatically tag the subnets with the appropriate tags. You may use an existing VPC if you wish but take note of the necessary subnet tags: https://github.com/weaveworks/eksctl/blob/main/examples/04-existing-vpc.yaml https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html
 vpc:

diff --git a/azure/README.md b/azure/README.md
@@ -2,7 +2,7 @@
 # Azure
 
 ## Background
-The following is meant to guide you through running Hyperledger Besu or GoQuorum clients in Azure AKS (Kubernetes) in both development and production scenarios. As always you are free to customize the charts to suit your requirements. It is highly recommended to familiarize yourself with AKS (or equivalent Kubernetes infrastructure) before running things in production on Kubernetes.
+The following is meant to guide you through running Besu or GoQuorum clients in Azure AKS (Kubernetes) in both development and production scenarios. As always you are free to customize the charts to suit your requirements. It is highly recommended to familiarize yourself with AKS (or equivalent Kubernetes infrastructure) before running things in production on Kubernetes.
 
 It essentially comprises base infrastructure that is used to build the cluster & other resources in Azure via an [ARM template](./arm/azuredeploy.json). We also make use some Azure native services and features (tha are provisioned via a [script](./scripts/bootstrap.sh)) after the cluster is created. These include:
 - [AAD pod identities](https://docs.microsoft.com/en-us/azure/aks/use-azure-ad-pod-identity).

diff --git a/azure/arm/azuredeploy.json b/azure/arm/azuredeploy.json
@@ -46,7 +46,7 @@
     "apiVersionRoleAssignments": "2018-09-01-preview",
     "apiVersionKeyVault": "2018-02-14",
     "apiVersionLogAnalytics": "2020-03-01-preview",
-    "kubernetesVersion": "1.26.6",
+    "kubernetesVersion": "1.30.6",
     "uniqueStr": "[toLower(substring(uniqueString(resourceGroup().id, deployment().name), 0, 8))]",
     "namePrefix": "[concat(parameters('bcClient'), '-', parameters('env'), '-', variables('uniqueStr') )]",
     "vnetName": "[concat(variables('namePrefix'), '-vnet')]",

diff --git a/helm/charts/besu-node/templates/node-statefulset.yaml b/helm/charts/besu-node/templates/node-statefulset.yaml
@@ -275,7 +275,7 @@ spec:
             exec
             /opt/besu/bin/besu \
               --config-file=/etc/besu/config.toml \
-              --Xdns-enabled=true --Xdns-update-enabled=true --Xnat-kube-service-name={{ include "besu-node.fullname" . }} \
+              --Xdns-enabled=true --Xdns-update-enabled=true \
               --min-gas-price=0 
 
         livenessProbe:

diff --git a/helm/charts/besu-node/values.yaml b/helm/charts/besu-node/values.yaml
@@ -146,11 +146,11 @@ node:
 image:
   besu:
     repository: hyperledger/besu
-    tag: 24.8.0
+    tag: 24.12.1
     pullPolicy: IfNotPresent
   tessera:
     repository: quorumengineering/tessera
-    tag: 22.1.7
+    tag: 24.4
     pullPolicy: IfNotPresent
   hooks:
     repository: consensys/quorum-k8s-hooks

diff --git a/playground/kubectl/quorum-besu/clique/configmap/besu-config-toml-configmap.yaml b/playground/kubectl/quorum-besu/clique/configmap/besu-config-toml-configmap.yaml
@@ -6,6 +6,17 @@ metadata:
     app: besu-config-toml-configmap
   namespace: besu
 data:
+  static-nodes.json: |-
+    [
+      "enode://c1979a8a48693db804316b5acebe35e11731e1fb1c9c21ff7268ab25db6f6e03390a429b83cf0ec0865a7205f2669ec1ace652a3def11e2e01571c74939cbe22@validator1-0.besu-validator1.besu.svc.cluster.local:30303",
+      "enode://e40129f02c9e29a02049668346d4777bb55809042746882b33b20a8b5a7310eb5f107a53f0aa3da766ee77f401557a79c0c328329ea48bf0996c6c9dff817f76@validator2-0.besu-validator2.besu.svc.cluster.local:30303",
+      "enode://a3e4af081a0ab853c959b9acd0596f818b91a9409b9d04c50af055072c929abfa340e14111dcfa76e049fdb16bb9198e722d5e7be3e8ef37562ea0d0ce1eda11@validator3-0.besu-validator3.besu.svc.cluster.local:30303",
+      "enode://8f4e444a73034236ab4244c7a572aa2c6198b9e0d483ef17bf4b751cac5c0370bc527a5b0c5d01aa3ef41704af838c74730aeecac0f0c22dc4c17b0a9f03ad76@validator4-0.besu-validator4.besu.svc.cluster.local:30303",
+      "enode://98496800174b3c73ae33cba59f8f5e686cd488f7897c2edb52e2cf46383d75cd03dbb58dde07185bc0953f98800ca9a89f4b5ef450c5e51292ea08ec6130ee0c@member1-0.besu-member1.besu.svc.cluster.local:30303",
+      "enode://ad2c79c6561bc8212c2e8382611c62e406e767d1f3106c68ca206900f575cb8ba9a8be111c645cd9803701d684454c782c40d2361b000a32ed03d26228b30ec1@member2-0.besu-member2.besu.svc.cluster.local:30303",
+      "enode://af19c92deb635bd7720634de9b2e7908208530d6f5e96eee003a8f1799e5be4037957d7e2fdbe3605e3a38dab05c961679c02133a0e624e23a72f7961e8af6ac@member3-0.besu-member3.besu.svc.cluster.local:30303"
+    ]
+  
   config.toml: |-
     # Every possible CLI should be in this file.
     #
@@ -17,6 +28,7 @@ data:
     # Node Information
     data-path="/data"
     genesis-file="/etc/genesis/genesis.json"
+    static-nodes-file="/etc/besu/static-nodes.json"
     # TODO : insert as command line & mount Volume
     # rpc-http-authentication-credentials-file="none"
     # TODO : insert as command line & mount Volume

diff --git a/playground/kubectl/quorum-besu/clique/statefulsets/node-statefulset.yaml b/playground/kubectl/quorum-besu/clique/statefulsets/node-statefulset.yaml
@@ -23,14 +23,14 @@ spec:
     spec:
       initContainers:
         - name: init-bootnode2
-          image: pegasyseng/k8s-helper:v1.18.4
+          image: curlimages/curl:8.11.1
           command:
             - "sh"
             - "-c"
             - "curl -X GET --connect-timeout 30 --max-time 10 --retry 6 --retry-delay 0 --retry-max-time 300 ${BESU_VALIDATOR1_SERVICE_HOST}:8545/liveness"
       containers:
         - name: node
-          image: hyperledger/besu:23.7.0
+          image: hyperledger/besu:24.12.1
           imagePullPolicy: IfNotPresent
           resources:
             requests:
@@ -44,6 +44,10 @@ spec:
               valueFrom:
                 fieldRef:
                   fieldPath: status.podIP
+            - name: BESU_P2P_HOST
+              valueFrom:
+                fieldRef:
+                  fieldPath: status.podIP                  
             - name: VALIDATOR1_PUBKEY
               valueFrom:
                 configMapKeyRef:
@@ -83,7 +87,6 @@ spec:
             - |
               exec /opt/besu/bin/besu \
                 --config-file=/etc/besu/config.toml \
-                --Xnat-kube-service-name=besu-node \
                 --bootnodes=enode://${VALIDATOR1_PUBKEY}@${BESU_VALIDATOR1_SERVICE_HOST}:30303,enode://${VALIDATOR2_PUBKEY}@${BESU_VALIDATOR2_SERVICE_HOST}:30303
           livenessProbe:
             httpGet:

diff --git a/playground/kubectl/quorum-besu/clique/statefulsets/validator1-statefulset.yaml b/playground/kubectl/quorum-besu/clique/statefulsets/validator1-statefulset.yaml
@@ -62,7 +62,7 @@ spec:
       serviceAccountName: validator1-sa
       containers:
         - name: validator1
-          image: hyperledger/besu:23.7.0
+          image: hyperledger/besu:24.12.1
           imagePullPolicy: IfNotPresent
           resources:
             requests:
@@ -80,6 +80,10 @@ spec:
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.name
+            - name: BESU_P2P_HOST
+              valueFrom:
+                fieldRef:
+                  fieldPath: status.podIP                  
             - name: VALIDATOR1_PUBKEY
               valueFrom:
                 configMapKeyRef:
@@ -123,7 +127,6 @@ spec:
               exec /opt/besu/bin/besu \
                 --node-private-key-file=/secrets/key \
                 --config-file=/etc/besu/config.toml \
-                --Xnat-kube-service-name=besu-validator1 \
                 --bootnodes=enode://${VALIDATOR1_PUBKEY}@${BESU_VALIDATOR1_SERVICE_HOST}:30303,enode://${VALIDATOR2_PUBKEY}@${BESU_VALIDATOR2_SERVICE_HOST}:30303
           livenessProbe:
             httpGet:

diff --git a/playground/kubectl/quorum-besu/clique/statefulsets/validator2-statefulset.yaml b/playground/kubectl/quorum-besu/clique/statefulsets/validator2-statefulset.yaml
@@ -62,14 +62,14 @@ spec:
       serviceAccountName: validator2-sa
       initContainers:
         - name: init-bootnode2
-          image: pegasyseng/k8s-helper:v1.18.4
+          image: curlimages/curl:8.11.1
           command:
             - "sh"
             - "-c"
             - "curl -X GET --connect-timeout 30 --max-time 10 --retry 6 --retry-delay 0 --retry-max-time 300 ${BESU_VALIDATOR1_SERVICE_HOST}:8545/liveness"
       containers:
         - name: validator2
-          image: hyperledger/besu:23.7.0
+          image: hyperledger/besu:24.12.1
           imagePullPolicy: IfNotPresent
           resources:
             requests:
@@ -87,6 +87,10 @@ spec:
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.name
+            - name: BESU_P2P_HOST
+              valueFrom:
+                fieldRef:
+                  fieldPath: status.podIP                  
             - name: VALIDATOR1_PUBKEY
               valueFrom:
                 configMapKeyRef:
@@ -130,7 +134,6 @@ spec:
               exec /opt/besu/bin/besu \
                 --node-private-key-file=/secrets/key \
                 --config-file=/etc/besu/config.toml \
-                --Xnat-kube-service-name=besu-validator2 \
                 --bootnodes=enode://${VALIDATOR1_PUBKEY}@${BESU_VALIDATOR1_SERVICE_HOST}:30303,enode://${VALIDATOR2_PUBKEY}@${BESU_VALIDATOR2_SERVICE_HOST}:30303
           livenessProbe:
             httpGet:

diff --git a/playground/kubectl/quorum-besu/clique/statefulsets/validator3-statefulset.yaml b/playground/kubectl/quorum-besu/clique/statefulsets/validator3-statefulset.yaml
@@ -62,14 +62,14 @@ spec:
       serviceAccountName: validator3-sa
       initContainers:
         - name: init-bootnode2
-          image: pegasyseng/k8s-helper:v1.18.4
+          image: curlimages/curl:8.11.1
           command:
             - "sh"
             - "-c"
             - "curl -X GET --connect-timeout 30 --max-time 10 --retry 6 --retry-delay 0 --retry-max-time 300 ${BESU_VALIDATOR1_SERVICE_HOST}:8545/liveness"
       containers:
         - name: validator3
-          image: hyperledger/besu:23.7.0
+          image: hyperledger/besu:24.12.1
           imagePullPolicy: IfNotPresent
           resources:
             requests:
@@ -87,6 +87,10 @@ spec:
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.name
+            - name: BESU_P2P_HOST
+              valueFrom:
+                fieldRef:
+                  fieldPath: status.podIP                  
             - name: VALIDATOR1_PUBKEY
               valueFrom:
                 configMapKeyRef:
@@ -130,7 +134,6 @@ spec:
               exec /opt/besu/bin/besu \
                 --node-private-key-file=/secrets/key \
                 --config-file=/etc/besu/config.toml \
-                --Xnat-kube-service-name=besu-validator3 \
                 --bootnodes=enode://${VALIDATOR1_PUBKEY}@${BESU_VALIDATOR1_SERVICE_HOST}:30303,enode://${VALIDATOR2_PUBKEY}@${BESU_VALIDATOR2_SERVICE_HOST}:30303
           livenessProbe:
             httpGet:

diff --git a/playground/kubectl/quorum-besu/clique/statefulsets/validator4-statefulset.yaml b/playground/kubectl/quorum-besu/clique/statefulsets/validator4-statefulset.yaml
@@ -62,14 +62,14 @@ spec:
       serviceAccountName: validator4-sa
       initContainers:
         - name: init-bootnode2
-          image: pegasyseng/k8s-helper:v1.18.4
+          image: curlimages/curl:8.11.1
           command:
             - "sh"
             - "-c"
             - "curl -X GET --connect-timeout 30 --max-time 10 --retry 6 --retry-delay 0 --retry-max-time 300 ${BESU_VALIDATOR1_SERVICE_HOST}:8545/liveness"
       containers:
         - name: validator4
-          image: hyperledger/besu:23.7.0
+          image: hyperledger/besu:24.12.1
           imagePullPolicy: IfNotPresent
           resources:
             requests:
@@ -87,6 +87,10 @@ spec:
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.name
+            - name: BESU_P2P_HOST
+              valueFrom:
+                fieldRef:
+                  fieldPath: status.podIP                  
             - name: VALIDATOR1_PUBKEY
               valueFrom:
                 configMapKeyRef:
@@ -130,7 +134,6 @@ spec:
               exec /opt/besu/bin/besu \
                 --node-private-key-file=/secrets/key \
                 --config-file=/etc/besu/config.toml \
-                --Xnat-kube-service-name=besu-validator4 \
                 --bootnodes=enode://${VALIDATOR1_PUBKEY}@${BESU_VALIDATOR1_SERVICE_HOST}:30303,enode://${VALIDATOR2_PUBKEY}@${BESU_VALIDATOR2_SERVICE_HOST}:30303
           livenessProbe:
             httpGet:

diff --git a/playground/kubectl/quorum-besu/ethash/README.md b/playground/kubectl/quorum-besu/ethash/README.md
diff --git a/playground/kubectl/quorum-besu/ethash/configmap/besu-bootnodes-configmap.yaml b/playground/kubectl/quorum-besu/ethash/configmap/besu-bootnodes-configmap.yaml