SimpliVity-Prometheus-4.1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# HPE SimpliVity - Prometheus - Grafana Monitoring"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "This Jupyter notebook describes the deployment of the monitoring of a SimpliVity federation, using the HPE SimpliVity connector for Prometheus and the Grafana visualization tool \n",
    "\n",
    "The solution described is is deployed on a Kubernetes 1.17 cluster deployed with Rancher on a HPE SimpliVity lab environment, i.e. all parts of the monitoring stack are deployed as container."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect to the Kubernetes environment"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "Check that you are connected to the correct Kubernetes cluster. If not use the ClusterConnection Notebook in the Toolkit to get this fixed. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME           STATUS   ROLES                  AGE     VERSION\n",
      "suo04k8sr-c1   Ready    control-plane,worker   3h55m   v1.32.2\n",
      "suo04k8sr-c2   Ready    control-plane,worker   3h19m   v1.32.2\n",
      "suo04k8sr-c3   Ready    control-plane,worker   171m    v1.32.2\n",
      "suo04k8sr-w1   Ready    worker                 169m    v1.32.2\n",
      "suo04k8sr-w2   Ready    worker                 166m    v1.32.2\n",
      "suo04k8sr-w3   Ready    worker                 165m    v1.32.2\n",
      "No resources found in svtprometheus namespace.\n"
     ]
    }
   ],
   "source": [
    "kubectl get nodes\n",
    "kubectl -n svtprometheus get pods,svc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Create the namespace\n",
    "\n",
    "Create a namespace for your deployment. The example below creates the namespace svtprometheus. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/prometheus created\n"
     ]
    }
   ],
   "source": [
    "kubectl apply -f namespace.yml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/svtprometheus created\n"
     ]
    }
   ],
   "source": [
    "# Example content of the namespace.yml file - either run the command above or this one to create the namespace\n",
    "cat << 'EOF' | kubectl apply -f -\n",
    "apiVersion: v1\n",
    "kind: Namespace\n",
    "metadata:\n",
    "  name: svtprometheus\n",
    "EOF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Create the ConfigMap\n",
    "\n",
    "The connection and monitoring parameters are transferred to the HPE SimpliVity Prometheus connector as a Kubernetes ConfigMap. \n",
    "\n",
    "The ConfigMap can be created using th Python script: CreateConfigMap.py. \n",
    "\n",
    "Requirements:\n",
    "    - System with Python 3 and the following Python packages installed: \n",
    "      - Fernet\n",
    "      - getpass\n",
    "      - etree\n",
    "\n",
    "Run the script with the following command: python3 CreateConfigMap.py\n",
    "The script will ask for the following information:\n",
    "    - username               vCenter username \n",
    "    - password               vCenter password\n",
    "    - OVC/MVA IP address     IP address that the connector uses to connect to the federation\n",
    "    - name                   name of the yml-file (<name>.yml) and the configmap: <name>-xml\n",
    "    - port                   TCP Port that the connector uses to publish the counters. \n",
    "\n",
    "It will create the Kubernetes yml-file (<name>.yml) that can be used to  create the Configmap <name>-xml \n",
    "\n",
    "The CreateConfigMap.py scripts sets for some of the SimpliVity connector parameters default values that can be edited in the file if needed:\n",
    "    \n",
    "    Parameter               Default value   Comment\n",
    "    - namespace             svtprometheus   adjust this to your namespace)\n",
    "    - timerange             30\n",
    "    - resolution            SECOND\n",
    "    - monitoringinterval    30\n",
    "    - monitor               fcn\n",
    "    - cluster               \n",
    "    - limit                 500 \n",
    "    - offset                -1  \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/jupyter/deployments/Prometheus/SimpliVity-Prometheus-4.1\n",
      "CreateConfigMap.py               svtdemo.yml\n",
      "namespace.yml                    svtdemo.yml.old\n",
      "SimpliVityClass.py               svtinfrastructure.key\n",
      "\u001b[0m\u001b[01;32mSimpliVity-Prometheus-4.1.ipynb\u001b[0m  svtinfrastructure.xml\n",
      "svtconnect.k8s.Dockerfile        svtinfrastructure.yml\n",
      "svtconnector.yml                 svtinfra.yml\n",
      "svtdemo.key                      svtPromConnector.v4.1.py\n",
      "svtdemo.xml                      \u001b[01;32mSvtPromDeployment.ipynb\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "pwd\n",
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "configmap/svtdemo-xml created\n"
     ]
    }
   ],
   "source": [
    "# apply the configmap (in the following example the name svtdemo was used)\n",
    "kubectl apply -f svtdemo.yml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "configmap/svtinfra-xml created\n"
     ]
    }
   ],
   "source": [
    "kubectl apply -f svtinfra.yml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example for the SimpliVity Prometheus Connector Configmap = do not use this example code.\n",
    "# Create your own configmap using the CreateConfigMap.py script! \n",
    "apiVersion: v1\n",
    "kind: ConfigMap\n",
    "metadata:\n",
    "  name: svtdemo-xml\n",
    "  namespace: test\n",
    "data:\n",
    "  svtconnector.key: |- \n",
    "    SgR7_7aCa0YegMcLx0ivwDNA_qtZNil2aeZCpWb5vws=\n",
    "  svtconnector.xml: |- \n",
    "    <data> \n",
    "      <username>SvtCollector@vsphere.local</username>\n",
    "      <user>gAAAAABfWhQVvn8PxLT86MLghsqaeB56fpGK2GdyyZTnkuym18udWAl_i51Ddo51-ScwwVUSST2uIqHq7vtyyWqOZ5MVtvv2ESVMZ88tTiJijzoxAHNA_2Y=</user>\n",
    "      <password>gAAAAABfWhQV0DDfLv-prmLnU1ZMMEwXwJxdUugCHxcIjqOQDt7ePaN1d5ZAR96ox-qHJSaHF87B74hek5DvWleMRq__77Fogg==</password>\n",
    "      <ovc>10.1.40.15</ovc>\n",
    "      <timerange>30</timerange>\n",
    "      <resolution>SECOND</resolution>\n",
    "      <monitoringintervall>30</monitoringintervall>\n",
    "      <logfile>svtdemo-xml.log</logfile>\n",
    "      <port>9091</port>\n",
    "      <monitor>fcn</monitor>\n",
    "      <cluster></cluster>\n",
    "      <limit>500</limit>\n",
    "      <offset>-1</offset>\n",
    "    </data>\n",
    "EOF  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check that the configmap is created properly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME               DATA   AGE\n",
      "kube-root-ca.crt   1      143m\n",
      "svtdemo-xml        2      12m\n",
      "svtinfra-xml       2      2m13s\n"
     ]
    }
   ],
   "source": [
    "kubectl -n prometheus get configmap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "configmap \"svtdemo-xml\" deleted\n"
     ]
    }
   ],
   "source": [
    "kubectl -n prometheus delete configmap/svtdemo-xml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name:         svtdemo-xml\n",
      "Namespace:    prometheus\n",
      "Labels:       <none>\n",
      "Annotations:  <none>\n",
      "\n",
      "Data\n",
      "====\n",
      "svtconnector.xml:\n",
      "----\n",
      "<data> \n",
      "  <username>SvtCollector@vsphere.local</username>\n",
      "  <user>gAAAAABn6p9M4nlNVN27GKiijxeKDXmn_EWpyCvay2xH-AnrqFENshlpgEfh_u-V5OsPWL6p3-KdPN2XWIC7H2nsEk0-mg9nTFjsYGHjKrMD2RjX1Xtm4ps=</user>\n",
      "  <password>gAAAAABn6p9MU8shpKVTmEv_BjWQqyTmTadIrUotYJ1OWCTEWxYg5mG6dL_kWpaed2dfpkfCcutLtnc89kUZ94Wk4J-ijjqsAg==</password>\n",
      "  <ovc>10.1.40.15</ovc>\n",
      "  <logfile>svtdemo.log</logfile>\n",
      "  <timerange>30</timerange>\n",
      "  <resolution>SECOND</resolution>\n",
      "  <monitoringintervall>30</monitoringintervall>\n",
      "  <logfile>svtdemo.log</logfile>\n",
      "  <port>9091</port>\n",
      "  <monitor>fcn</monitor>\n",
      "  <cluster></cluster>\n",
      "  <limit>500</limit>\n",
      "  <offset>-1</offset>\n",
      "</data>\n",
      "\n",
      "svtconnector.key:\n",
      "----\n",
      "R3eEo3h55iSucYSRrpSIZzo4LV7ydOuxK0TlW638flc=\n",
      "\n",
      "\n",
      "BinaryData\n",
      "====\n",
      "\n",
      "Events:  <none>\n"
     ]
    }
   ],
   "source": [
    "kubectl -n prometheus describe configmap/svtdemo-xml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "configmap \"svtinfrastructure-xml\" deleted\n"
     ]
    }
   ],
   "source": [
    "kubectl -n prometheus delete configmap/svtinfrastructure-xml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name:         svtinfra-xml\n",
      "Namespace:    prometheus\n",
      "Labels:       <none>\n",
      "Annotations:  <none>\n",
      "\n",
      "Data\n",
      "====\n",
      "svtconnector.key:\n",
      "----\n",
      "R3eEo3h55iSucYSRrpSIZzo4LV7ydOuxK0TlW638flc=\n",
      "\n",
      "svtconnector.xml:\n",
      "----\n",
      "<data> \n",
      "  <username>SvtCollector@vsphere.local</username>\n",
      "  <user>gAAAAABn6p9M4nlNVN27GKiijxeKDXmn_EWpyCvay2xH-AnrqFENshlpgEfh_u-V5OsPWL6p3-KdPN2XWIC7H2nsEk0-mg9nTFjsYGHjKrMD2RjX1Xtm4ps=</user>\n",
      "  <password>gAAAAABn6p9MU8shpKVTmEv_BjWQqyTmTadIrUotYJ1OWCTEWxYg5mG6dL_kWpaed2dfpkfCcutLtnc89kUZ94Wk4J-ijjqsAg==</password>\n",
      "  <ovc>10.0.44.33</ovc>\n",
      "  <timerange>30</timerange>\n",
      "  <resolution>SECOND</resolution>\n",
      "  <monitoringintervall>30</monitoringintervall>\n",
      "  <logfile>svtdemo.log</logfile>\n",
      "  <port>9091</port>\n",
      "  <monitor>fcn</monitor>\n",
      "  <cluster></cluster>\n",
      "  <limit>500</limit>\n",
      "  <offset>-1</offset>\n",
      "</data>\n",
      "\n",
      "\n",
      "BinaryData\n",
      "====\n",
      "\n",
      "Events:  <none>\n"
     ]
    }
   ],
   "source": [
    "kubectl -n prometheus describe configmap/svtinfra-xml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Create the SimpliVity Connector POD and Service with the Config Maps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Create the SimpliVity Connector POD for the CTC Demo environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment.apps/svtdemo created\n"
     ]
    }
   ],
   "source": [
    "cat << 'EOF' | kubectl apply -f -\n",
    "apiVersion: apps/v1\n",
    "kind: Deployment\n",
    "metadata:\n",
    "  name: svtdemo\n",
    "  namespace: prometheus\n",
    "  labels:\n",
    "    app: svtdemo\n",
    "spec:\n",
    "  selector:\n",
    "    matchLabels:\n",
    "      app: svtdemo\n",
    "  template:\n",
    "    metadata:\n",
    "      labels:\n",
    "        app: svtdemo\n",
    "    spec:\n",
    "      containers:\n",
    "        - name: svtdemo\n",
    "          image: tb1378/svtconk8s\n",
    "          command: [\"/usr/bin/python3\"]\n",
    "          args: [\"/opt/svt/svtpromconnector.py\"]\n",
    "          volumeMounts:\n",
    "            - name: svtconnectorxml\n",
    "              mountPath: /opt/svt/data\n",
    "      volumes:\n",
    "        - name: svtconnectorxml\n",
    "          configMap:\n",
    "            name: svtdemo-xml   # the correct name of the configmap needs to be added here. \n",
    "EOF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "service/svtdemo unchanged\n"
     ]
    }
   ],
   "source": [
    "cat << 'EOF' | kubectl apply -f -\n",
    "apiVersion: v1\n",
    "kind: Service\n",
    "metadata:\n",
    "  name: svtdemo\n",
    "  namespace: prometheus\n",
    "#  labels: \n",
    "#     hpecp.hpe.com/hpecp-internal-gateway: \"true\"    \n",
    "spec:\n",
    "  selector:\n",
    "    app: svtdemo\n",
    "  ports:\n",
    "    - port: 9091               # The Port of that the SimpliVity connector uses\n",
    "      targetPort: 9091\n",
    "      protocol: TCP\n",
    "  type: NodePort               # expose the SimpliVity connector if you want/need to debug it. \n",
    "EOF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME      READY   UP-TO-DATE   AVAILABLE   AGE\n",
      "svtdemo   0/1     1            0           14s\n",
      "NAME      TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE\n",
      "svtdemo   NodePort   10.152.183.21   <none>        9091:30863/TCP   28m\n"
     ]
    }
   ],
   "source": [
    "kubectl -n prometheus get deployments\n",
    "kubectl -n prometheus get svc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME                       READY   STATUS             RESTARTS      AGE\n",
      "svtdemo-556f465646-mvgfv   0/1     CrashLoopBackOff   5 (15s ago)   3m33s\n"
     ]
    }
   ],
   "source": [
    "kubectl -n prometheus get pods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2025-03-31 14:18:25.050559: Path:    /opt/svt/ \n",
      "\n",
      "2025-03-31 14:18:25.050568: Keyfile: /opt/svt/data/svtconnector.key \n",
      "\n",
      "2025-03-31 14:18:25.050572: XMLfile: /opt/svt/data/svtconnector.xml \n",
      "\n",
      "2025-03-31 14:18:25.083836: Open a connection to the SimpliVity systems \n",
      "\n",
      "2025-03-31 14:18:25.084255: Open Connection to SimpliVity \n",
      "\n",
      "Traceback (most recent call last):\n",
      "  File \"/opt/svt/svtpromconnector.py\", line 227, in <module>\n",
      "    svt.Connect(svtuser,svtpassword)\n",
      "  File \"/opt/svt/SimpliVityClass.py\", line 104, in Connect\n",
      "    raise SvtError('SimpliVity.connect',response.status_code, response.json())\n",
      "SimpliVityClass.SvtError: ('SimpliVity.connect', 404, {'error_type': 'NOT_FOUND', 'messages': [{'args': [], 'default_message': 'Not found.', 'id': 'com.vmware.vapi.rest.httpNotFound'}]})\n"
     ]
    }
   ],
   "source": [
    "kubectl -n prometheus logs pod/svtdemo-556f465646-mvgfv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment.apps \"svtdemo\" deleted\n"
     ]
    }
   ],
   "source": [
    "kubectl -n prometheus delete deployment/svtdemo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Create the SimpliVity Connector POD for the CTC infrastructure cluster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment.apps/svtinfra created\n"
     ]
    }
   ],
   "source": [
    "cat << 'EOF' | kubectl apply -f -\n",
    "apiVersion: apps/v1\n",
    "kind: Deployment\n",
    "metadata:\n",
    "  name: svtinfra\n",
    "  namespace: svtprometheus\n",
    "  labels:\n",
    "    app: svtinfra\n",
    "spec:\n",
    "  selector:\n",
    "    matchLabels:\n",
    "      app: svtinfra\n",
    "  template:\n",
    "    metadata:\n",
    "      labels:\n",
    "        app: svtinfra\n",
    "    spec:\n",
    "      containers:\n",
    "        - name: svtinfra\n",
    "          image: tb1378/svtconk8s\n",
    "          command: [\"/usr/bin/python3\"]\n",
    "          args: [\"/opt/svt/svtpromconnector.py\"]\n",
    "          volumeMounts:\n",
    "            - name: svtconnectorxml\n",
    "              mountPath: /opt/svt/data\n",
    "      volumes:\n",
    "        - name: svtconnectorxml\n",
    "          configMap:\n",
    "            name: svtinfrastructure-xml   # the correct name of the configmap needs to be added here. \n",
    " \n",
    "EOF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "service/svtinfra created\n"
     ]
    }
   ],
   "source": [
    "cat << 'EOF' | kubectl apply -f -\n",
    "---\n",
    "apiVersion: v1\n",
    "kind: Service\n",
    "metadata:\n",
    "  name: svtinfra\n",
    "  namespace: svtprometheus\n",
    "#  labels: \n",
    "#     hpecp.hpe.com/hpecp-internal-gateway: \"true\"    \n",
    "spec:\n",
    "  selector:\n",
    "    app: svtinfra\n",
    "  ports:\n",
    "    - port: 9091               # The Port of that the SimpliVity connector uses\n",
    "      targetPort: 9091\n",
    "      protocol: TCP\n",
    "  type: NodePort               # expose the SimpliVity connector if you want/need to debug it.\n",
    "EOF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME                                                  READY   STATUS    RESTARTS       AGE    IP            NODE                      NOMINATED NODE   READINESS GATES\n",
      "pod/alletra6k-hpe-array-exporter-fc6b7cddc-kgfsx      1/1     Running   0              171d   10.244.3.19   suo04k8sr-w1.demo.local   <none>           <none>\n",
      "pod/alletra9060-hpe-array-exporter-6b749784c-2d6nm    1/1     Running   0              171d   10.244.4.17   suo04k8sr-w2.demo.local   <none>           <none>\n",
      "pod/alletramp16-hpe-array-exporter-86cd697857-sxl88   1/1     Running   0              171d   10.244.5.12   suo04k8sr-w3.demo.local   <none>           <none>\n",
      "pod/alletramp8-hpe-array-exporter-659d946545-zpwqh    1/1     Running   0              171d   10.244.5.13   suo04k8sr-w3.demo.local   <none>           <none>\n",
      "pod/dhci-hpe-array-exporter-6d7c968b4b-wg2ch          1/1     Running   0              171d   10.244.3.20   suo04k8sr-w1.demo.local   <none>           <none>\n",
      "pod/dnsutils                                          1/1     Running   0              206d   10.244.3.6    suo04k8sr-w1.demo.local   <none>           <none>\n",
      "pod/grafana-6c46b4b84d-rdkzr                          1/1     Running   0              206d   10.244.5.6    suo04k8sr-w3.demo.local   <none>           <none>\n",
      "pod/ilo1-6bd665f9b6-8bsv4                             1/1     Running   0              57d    10.244.5.22   suo04k8sr-w3.demo.local   <none>           <none>\n",
      "pod/ilo3-6bfbc4bf44-nxmhz                             1/1     Running   0              109d   10.244.3.22   suo04k8sr-w1.demo.local   <none>           <none>\n",
      "pod/ilo4-7b557b77dc-c244s                             1/1     Running   0              23h    10.244.4.26   suo04k8sr-w2.demo.local   <none>           <none>\n",
      "pod/ilo6-bd4b564d9-q9pvz                              1/1     Running   0              109d   10.244.4.23   suo04k8sr-w2.demo.local   <none>           <none>\n",
      "pod/prometheus-8659cf8954-hs7z9                       1/1     Running   0              171d   10.244.4.19   suo04k8sr-w2.demo.local   <none>           <none>\n",
      "pod/svtdemo-8445d59666-bffbj                          1/1     Running   12 (13d ago)   141d   10.244.4.20   suo04k8sr-w2.demo.local   <none>           <none>\n",
      "pod/svtinfra-8449c44d7d-kvpvf                         1/1     Running   0              6s     10.244.5.25   suo04k8sr-w3.demo.local   <none>           <none>\n",
      "\n",
      "NAME                                     TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE    SELECTOR\n",
      "service/alletra6k-hpe-array-exporter     NodePort   10.98.59.222     <none>        9090:30359/TCP   171d   app=hpe-array-exporter,release=alletra6k\n",
      "service/alletra9060-hpe-array-exporter   NodePort   10.103.11.236    <none>        9090:31653/TCP   171d   app=hpe-array-exporter,release=alletra9060\n",
      "service/alletramp16-hpe-array-exporter   NodePort   10.96.191.73     <none>        9090:30531/TCP   171d   app=hpe-array-exporter,release=alletramp16\n",
      "service/alletramp8-hpe-array-exporter    NodePort   10.104.129.20    <none>        9090:30382/TCP   171d   app=hpe-array-exporter,release=alletramp8\n",
      "service/dhci-hpe-array-exporter          NodePort   10.108.78.66     <none>        9090:30369/TCP   171d   app=hpe-array-exporter,release=dhci\n",
      "service/grafana-service                  NodePort   10.99.83.144     <none>        3000:30699/TCP   206d   app=grafana,tier=frontend\n",
      "service/ilo1                             NodePort   10.106.40.89     <none>        9091:31454/TCP   206d   app=ilo1\n",
      "service/ilo2                             NodePort   10.100.49.216    <none>        9091:32681/TCP   206d   app=ilo2\n",
      "service/ilo3                             NodePort   10.111.242.192   <none>        9091:30737/TCP   206d   app=ilo3\n",
      "service/ilo4                             NodePort   10.107.202.62    <none>        9091:31764/TCP   206d   app=ilo4\n",
      "service/ilo5                             NodePort   10.108.176.74    <none>        9091:32402/TCP   206d   app=ilo5\n",
      "service/ilo6                             NodePort   10.96.180.124    <none>        9091:30987/TCP   206d   app=ilo6\n",
      "service/prometheus                       NodePort   10.99.193.205    <none>        9090:32396/TCP   206d   app=grafana,tier=backend\n",
      "service/svtdemo                          NodePort   10.106.114.204   <none>        9091:30030/TCP   206d   app=svtdemo\n",
      "service/svtinfra                         NodePort   10.107.63.51     <none>        9091:31603/TCP   206d   app=svtinfra\n"
     ]
    }
   ],
   "source": [
    "kubectl -n svtprometheus get pods,svc -o wide"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment.apps \"svtinfra\" deleted\n"
     ]
    }
   ],
   "source": [
    "kubectl -n svtprometheus delete deployment/svtinfra"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Delete Deployments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME                                    TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE\n",
      "service/alletra6k-hpe-array-exporter    NodePort    10.108.226.130   <none>        9090:32714/TCP   334d\n",
      "service/dhci-hpe-array-exporter         ClusterIP   10.98.60.160     <none>        9090/TCP         557d\n",
      "service/grafana-service                 NodePort    10.99.178.61     <none>        3000:31200/TCP   590d\n",
      "service/ilo2                            NodePort    10.110.182.5     <none>        9091:31107/TCP   248d\n",
      "service/ilo3                            NodePort    10.109.45.7      <none>        9091:30291/TCP   244d\n",
      "service/ilo4                            NodePort    10.108.224.80    <none>        9091:32761/TCP   244d\n",
      "service/ilo5                            NodePort    10.100.192.31    <none>        9091:31598/TCP   243d\n",
      "service/iloredfish                      NodePort    10.101.190.222   <none>        9091:31949/TCP   334d\n",
      "service/ilotest                         NodePort    10.98.222.184    <none>        9091:31524/TCP   248d\n",
      "service/primera630-hpe-array-exporter   NodePort    10.103.139.106   <none>        9090:32110/TCP   557d\n",
      "service/primera650-hpe-array-exporter   NodePort    10.104.179.2     <none>        9090:30467/TCP   557d\n",
      "service/prometheus                      NodePort    10.96.144.183    <none>        9090:30585/TCP   590d\n",
      "service/svtdemo                         NodePort    10.108.124.136   <none>        9091:31390/TCP   590d\n",
      "service/svtinfra                        NodePort    10.107.29.214    <none>        9091:30441/TCP   590d\n",
      "\n",
      "NAME                                 CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                               STORAGECLASS   REASON   AGE\n",
      "persistentvolume/grafana-db-volume   1Gi        RWX            Retain           Bound    svtprometheus/grafana-pv-claim      nfs                     265d\n",
      "persistentvolume/horizon-db-volume   1Gi        RWX            Retain           Bound    horizon/horizon-pvc                 nfs                     93d\n",
      "persistentvolume/prom-db-volume      1Gi        RWX            Retain           Bound    svtprometheus/prometheus-pv-claim   nfs                     265d\n"
     ]
    }
   ],
   "source": [
    "kubectl -n svtprometheus get pods,svc,pv,deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment.apps \"svtinfra\" deleted\n"
     ]
    }
   ],
   "source": [
    "kubectl -n svtprometheus delete deployment/svtinfra"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment.apps \"svtdemo\" deleted\n"
     ]
    }
   ],
   "source": [
    "kubectl -n svtprometheus delete deployment/svtdemo"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}