Skip to content

ibm-cloud-architecture/refarch-kc-gitops

BranchesTags

Repository files navigation

KC solution gitops

IBM Event Driven Architecture reference implementation GitOps repository, in support of https://ibm-cloud-architecture.github.io/refarch-eda/

This gitops repository supports three types of deployments of the solution, one using Strimzi Kafka to run on OpenShift, one with Event Streams with Cloud Pak for Integration, and one with Event Streams Managed service on IBM Cloud.

The microservices of the solutions are deployed to OpenShift and connect to the Kafka deployment you use.

Updated 09/14/2021.

Considerations

As presented in the product documentation, "Considerations before deployment", we have to assess user roles and platform requirements.

Here are the assumptions we define for the kc-solution:

  • Single admin team for OCP cluster and production projects within the cluster.
  • Developers manages staging and dev environment. This is a functional team developing the kc-solution
  • For the solution one gitops will define all environments and apps/services of the solution.
  • Developers will not have access to OpenShift cluster administration
  • Cloud Pak for integration operators are installed in all namespaces, and there is only one instance of each operator.
  • Only one Platform Navigator installed per cluster (in all namespaces) and it displays instances of capabilities from the whole cluster.
  • ibm-common-services is unique to the cluster.

For real production deployment, the production OpenShift cluster will be separate from dev and staging, running in different infrastructure.

Pre-requisites

Event Streams with Cloud Pak for Integration

The Cloud Pak for Integration installation is documented here and a cloud pak gitops repository helps to automate deployment.

"Manual deployment of event-streams"

In this section we just focusing on deploying Event Streams within an OpenShift with IBM Catalog defined.

  • Get IBM product catalog added to your OpenShift cluster
# To verify they are not already installed use:
oc get operators
# If you do not see any IBM operators then install IBM Catalog definition
oc apply -f ./ibm-catalog/subscription.yaml

If you just want event-streams and dependant operators deployed use the following:

# deploy dependent operators if not done before
oc apply -f https://raw.githubusercontent.com/ibm-cloud-architecture/eda-gitops-catalog/main/cp4i-operators/asset-repo.yaml
oc apply -f https://raw.githubusercontent.com/ibm-cloud-architecture/eda-gitops-catalog/main/cp4i-operators/automation-foundation.yaml
oc apply -f https://raw.githubusercontent.com/ibm-cloud-architecture/eda-gitops-catalog/main/cp4i-operators/common-services.yaml
oc apply -f https://raw.githubusercontent.com/ibm-cloud-architecture/eda-gitops-catalog/main/cp4i-operators/couchdb.yaml

# deploy event streams operator
oc apply -f https://raw.githubusercontent.com/ibm-cloud-architecture/eda-gitops-catalog/main/cp4i-operators/event-streams/subscription.yaml

Now create users and cluster using manifests declaration. For example the es-stage environment defines a shared cluster:

oc apply -k environments/kc-stage/apps/event-streams

Automating deployment with ArgoCD

The manual steps to bootstrap this ci/cd process are:

  1. Install the openShift GitOps Operator.

  2. Obtain IBM license entitlement key

  3. Update the OCP global pull secret of the openshift-config project

  4. Use our bootstrap folder to initiate the GitOps

    # Reference ibm catalog
oc apply -k https://raw.githubusercontent.com/ibm-cloud-architecture/eda-gitops-catalog/main/ibm-catalog/kustomization.yaml
# Install OpenShift pipeline and sealed-secrets
oc create -k bootstrap/cicd
# NOT TESTED YET need release 2021.3 Install CP4I operators
# oc create -k bootstrap/cp4i
# Bootstrap solution environment
oc apply -k bootstrap/kc-solution
# Create a secrets for entitlement key
oc create secret docker-registry ibm-entitlement-key \
    --docker-username=cp \
    --docker-server=cp.icr.io \
    --namespace=kc-dev \
    --docker-password=entitlement_key

  5. Bootstrap the argocd app of app within an argoCD project named kc-solutionusing the command

oc apply -k config/argocd
  1. Log to ArgoCD console: See the exposed route in the openshift-gitops project
oc get route openshift-gitops-server -n openshift-gitops -o jsonpath='{.saliaspec.host}'
# Get argoadmin password
oc extract secret/openshift-gitops-cluster -n openshift-gitops --to=-

Once logged to the console, you should see the kc-argo-app synced or performing synch operation.

Draft notes

  • Update structure using KAM
  • Add Strimzi operators and cluster instance manifest in environments/strimzi
  • Add quarkus pipelines

Issues encountered

  • Strimzi operator not installed via argocd app
  • openshift-gitops-argocd-application-controller service account need to be able to create or patch resource under target namespace
# verify service account
oc get sa
# Verify existing cluster role
oc get clusterrole
# select the cluster role like below and do under the openshift-gitops project
oc adm policy add-cluster-role-to-user  strimzi-cluster-operator.v0.25.0-strimzi-cluster-ope-65df8bbd78  --serviceaccount openshift-gitops-argocd-application-controller -n openshift-gitops
  • In cicd: "InvalidSpecError: Application referencing project kc-solution which does not exist". Resynch once the argocd project is created
  • In cicd: "Resource rbac.authorization.k8s.io:ClusterRole is not permitted in project kc-solution.". Change project definition with
clusterResourceWhitelist:
  - group: "*"
    kind: "*"

Example environments

These example environments will deploy the microservices and their associated configuration to the shipping-dev namespace.

Development environment (dev)

This environment is deployable to any Kubernetes or OCP cluster and provides its own dedicated backing services.

Prerequisites:

  • Strimzi operator must be installed, and configured to watch all namespaces.
  • Open Liberty operator must be installed, and configured to watch all namespaces.

Deploying microservices:

(note: the following oc adm command is required only if targeting an OpenShift cluster).

One-time setup to create namespace and Kafka cluster:

kubectl apply -k environments/dev/infrastructure
oc adm policy add-scc-to-user anyuid -z kcontainer-runtime -n shipping
kubectl wait --for=condition=ready kafka my-cluster --timeout 300s -n shipping

Deploy microservices and required configmaps and secrets:

kubectl apply -k environments/dev
kubectl wait --for=condition=available deploy -l app.kubernetes.io/part-of=refarch-kc --timeout 300s -n shipping

Environment with off-cluster backing services (example-credentials)

This example configures the microservices to connect to a Postgres database with SSL verification enabled, and to Event Streams using an API key.

The backing services should already exist (for example, hosted on an OpenShift cluster or IBM Cloud). In the case of Kafka, the topics should already exist. In the environments/example-credentials tree, the Kafka topics are prefixed with itg-integration-.

Prerequisites: supply credentials for backing services as files - see credentials README

kubectl apply -k environments/example-credentials

Environment with off-cluster Event Streams with certificates (example-es-truststore)

This example configures the microservices to connect to a Postgres database with SSL verification enabled, and to Event Streams using an API key, with SSL verification enabled. The ES certificate is provided as a Java truststore, or a PEM file to the non-Java apps.

Prerequisites: supply credentials for backing services as files - see credentials README

kubectl apply -k environments/example-es-truststore

TODO: rewrite environment creation instructions below

Deploying KContainer Reference Implementation using GitOps templates

  1. Create a new branch based on the starter-template branch, using the format of <namespace>/<clustername>

    • Example: git checkout starter-template && git checkout -b demo-sandbox/roks-demos.us-east.containers.appdomain.cloud

  2. Update the microservices' Route deployment YAMLs to contain an appropriate host value by uncommenting the lines and replacing the [project] and [cluster-domain] placeholder values.

  3. Commit and push your updated branch to a git repository that will be accessible from your cluster.

  4. Create all the necessary pre-requisites in the target cluster:

    1. Ensure ArgoCD is installed and functional. Link
    2. Configure necessary backing components, like Kafka, Event Streams, or Postgresql. Link
    3. Create necessary Kubernetes ConfigMaps and Secrets, which will connect the microservices to the backing components. Link
      • The template YAMLs use the reasonable default names of the necessary ConfigMaps and Secrets, so the YAMLs will not need to be altered if you follow the deployment instructions exactly.
      • You may use different names for the ConfigMaps and Secrets, but you will need to adjust the references in the YAMLs accordingly. This should only be necessary if deploying multiple times to the same namespace.
    4. Configure Service Account, as required by OpenShift or Kubernetes. Link

  5. Create an ArgoCD application deployment for each microservice you wish to deploy, using either the ArgoCD CLI, applying application manifest YAMLs through the ArgoCD UI, or apply application manifest YAMLs through the Kubernetes CRDs:

    • ArgoCD CLI:
    argocd app create kcontainer-order-command-ms \
--repo https://github.com/ibm-cloud-architecture/refarch-kc-gitops.git \
--revision demo-sandbox/roks-demos.us-east.containers.appdomain.cloud \
--path kc-ui --directory-recurse --dest-server https://kubernetes.default.svc \
--dest-namespace demo-sandbox --sync-policy automated --self-heal --auto-prune
    • ArgoCD UI Manifest YAML:
    project: default
source:
  repoURL: 'https://github.com/ibm-cloud-architecture/refarch-kc-gitops.git'
  path: kc-ui
  targetRevision: demo-sandbox/roks-demos.us-east.containers.appdomain.cloud
  directory:
    recurse: true
    jsonnet: {}
destination:
  server: 'https://kubernetes.default.svc'
  namespace: demo-sandbox
syncPolicy:
  automated:
    prune: true
    selfHeal: true
    • ArgoCD CRD YAML:
    apiVersion: argoproj.io/v1alpha1
metadata:
  name: kc-ui-ms
spec:
  project: default
  source:
   repoURL: 'https://github.com/ibm-cloud-architecture/refarch-kc-gitops.git'
   path: kc-ui
   targetRevision: demo-sandbox/roks-demos.us-east.containers.appdomain.cloud
   directory:
     recurse: true
     jsonnet: {}
  destination:
   server: 'https://kubernetes.default.svc'
   namespace: demo-sandbox
  syncPolicy:
   automated:
     prune: true
     selfHeal: true

  6. You should be able to see ArgoCD applying the YAMLs to the target cluster by watching kubectl get pods.

  7. Validate all application Status conditions are both Healthy and Synced before verifying your application deployment in the application UI.

Generating KContainer Reference Implementation GitOps templates

  1. Generate all your necessary Kubernetes YAMLs by following the deployment steps documented in https://ibm-cloud-architecture.github.io/refarch-kc/deployments/application-components/, passing in specific values for the desired configuration.
  2. Copy the generated YAMLs to the root of your new branch in the repository.
    • Example: <repo_root>/ordercommandms/templates
  3. Commit and push your generated YAMLs to this repository on your new branch. Note that the master branch is protected and will not accept pushes, so you will only be able to push to your branch.
  4. Continue with Templates Step #4 above.
  5. Validate all application Status conditions are both Healthy and Synced before verifying your application deployment in the UI.

About

Event-driven Architecture reference implementation GitOps repository

ibm-cloud-architecture.github.io/refarch-kc

Topics

eda

Resources

Readme
Activity
Custom properties

Stars

2 stars

Watchers

6 watching

Forks

10 forks
Report repository

Releases

No releases published

Packages

No packages published

Contributors 3

  •  
  •  
  •