A Virtual Kubelet provider that enables Kubernetes to schedule container workloads on Cisco Catalyst series switches and other IOS-XE devices with App-Hosting capabilities.
This provider allows Kubernetes pods to be deployed as containers directly on Cisco devices, enabling edge computing scenarios where compute workloads run on network infrastructure. The provider communicates with Cisco devices via RESTCONF APIs to manage the full container and device lifecycle.
- Native Kubernetes Integration — deploy containers to Cisco devices using standard
kubectlcommands - Driver-Based Architecture — extensible driver pattern currently supporting IOS-XE devices
- Full App-Hosting Lifecycle — create, monitor, and delete containers via RESTCONF
- Network as Code — declare IOS-XE device configuration in Kubernetes (
IOSXEConfigCRD) with continuous drift detection and transactional apply - Software Lifecycle (Beta) — drive IOS-XE software upgrades via the
IOSXESoftwareUpgradeCRD using gNOI OS install/activate/verify - Device Operations (Beta) — run auditable
showcommands and read-only gNOI probes from Kubernetes viaDeviceOperationCRD - IOS-XE Telemetry (Beta) — declare MDT-over-gNMI subscriptions and emit OpenTelemetry metrics, logs, and state-transition traces
- Topology Observability (Beta) — emit CDP/OSPF topology and hosted-app traces to any OTLP-compatible backend
- Health Monitoring — continuous node health checks, kubelet metrics (
/stats/summary,/metrics/resource), and device annotations - Resource Management — CPU, memory, and storage allocation per container
- Flexible Networking — DHCP via Virtual Port Groups or AppGigabitEthernet; automatic IP discovery from device operational data
- Cisco Catalyst 8000V virtual routers
- Cisco Catalyst 9000 switches
┌────────────────────────────────────────────────────────────────┐
│ Kubernetes Cluster │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Kubernetes API Server │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────┼───────────────┐ │
│ ▼ ▼ ▼ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ VK Provider │ │ VK Provider │ │ VK Provider │ │
│ │ (Device 1) │ │ (Device 2) │ │ (Device N) │ │
│ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ │
└───────────┼───────────────────┼───────────────────┼────────────┘
│ RESTCONF │ RESTCONF │ RESTCONF
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ Cisco IOS-XE │ │ Cisco IOS-XE │ │ Cisco IOS-XE │
│ ┌─────────┐ │ │ ┌─────────┐ │ │ ┌─────────┐ │
│ │Container│ │ │ │Container│ │ │ │Container│ │
│ └─────────┘ │ │ └─────────┘ │ │ └─────────┘ │
└───────────────┘ └───────────────┘ └───────────────┘
- A Kubernetes cluster
helmv3- Cisco IOS-XE device with:
- IOx enabled (
ioxconfiguration) - RESTCONF enabled
- App-hosting support
- Container image (tar file) on device flash
- IOx enabled (
The controller watches CiscoDevice CRs and automatically creates a VK pod per device. Deploy it via the included Helm chart.
# Build
docker build -t <your-registry>/cisco-vk:latest .
# Push
docker push <your-registry>/cisco-vk:latest# Install CRDs and the controller into the cvk-system namespace
helm install cvk ./charts/cisco-virtual-kubelet \
--namespace cvk-system --create-namespace \
--set image.repository=<your-registry>/cisco-vk \
--set image.tag=latestBoth the controller pod and the VK pods it spawns use the same image by default. To use different images:
helm install cvk ./charts/cisco-virtual-kubelet \
--namespace cvk-system --create-namespace \
--set controllerImage.repository=<your-registry>/cisco-vk-controller \
--set controllerImage.tag=latest \
--set vkImage.repository=<your-registry>/cisco-vk \
--set vkImage.tag=latestStore device credentials in a Kubernetes Secret before creating the CiscoDevice CR:
kubectl create secret generic cat9000-1-creds \
--from-literal=username=admin \
--from-literal=password=<device-password>Once the controller is running, create a CiscoDevice resource to provision a VK node:
apiVersion: cisco.vk/v1alpha1
kind: CiscoDevice
metadata:
name: cat9000-1
namespace: default
spec:
driver: XE
address: "192.168.1.100"
port: 443
credentialSecretRef:
name: cat9000-1-creds
tls:
enabled: true
insecureSkipVerify: true
xe:
networking:
interface:
type: VirtualPortGroup
virtualPortGroup:
dhcp: true
interface: "0"
guestInterface: 0The controller creates a VK Deployment and a matching Kubernetes virtual node. Pods scheduled to that node are deployed to the device via App-Hosting.
- Getting Started — Installation, first device, and first pod
- Architecture — Technical architecture and component deep-dive
- Configuration Reference —
CiscoDevicespec options and device setup - Network as Code — Declarative
IOSXEConfig, drift detection, and transactional apply - CLI & Plugin Reference —
cisco-vkbinary andkubectl-ciscovkplugin - Software Lifecycle (Beta) — IOS-XE upgrades via
IOSXESoftwareUpgrade - Operations Runbook —
DeviceOperationshow commands, CRD upgrade guide - Telemetry (Beta) — MDT-over-gNMI subscriptions and OpenTelemetry
- Observability — Metrics, traces, and Splunk integration
- CRD Reference — All custom resource definitions
- API Reference — RESTCONF and kubelet endpoint reference
- Environment Variables — Complete environment variable reference
- Security — TLS, RBAC, and credential management
- Troubleshooting — Common issues and debug techniques
cisco-virtual-kubelet/
├── api/
│ ├── v1alpha1/ # Core CRDs: CiscoDevice, XEConfig
│ ├── config/v1alpha1/ # Config CRDs: IOSXEConfig, IOSXEConfigBundle, IOSXETelemetry
│ └── ops/v1alpha1/ # Ops CRDs: DeviceOperation, IOSXESoftwareUpgrade, IOSXEOperationalAction
├── cmd/
│ └── cisco-vk/ # Unified binary entry point
│ ├── main.go # cobra root command
│ ├── run.go # 'run' subcommand — per-device VK provider
│ └── manager.go # 'manager' subcommand — CRD controller manager
├── charts/
│ └── cisco-virtual-kubelet/ # Helm chart
│ ├── crds/ # CRD manifests (synced by make generate)
│ └── templates/ # RBAC, Deployments, ServiceAccounts
├── config/
│ └── crd/ # Generated CRDs (source of truth)
├── internal/
│ ├── aggregator/ # In-process config aggregator (experimental)
│ ├── controller/ # CiscoDevice and IOSXEConfigBundle reconcilers
│ ├── drivers/ # Device driver implementations (IOS-XE, fake)
│ │ └── iosxe/
│ │ └── configdriver/ # Network as Code engine, writers, intent resolver
│ ├── provider/ # Virtual Kubelet provider, config reconciler, telemetry
│ └── telemetry/ # MDT gNMI mapper, OTel emit, classifier, correlation
├── examples/
├── dev/ # Development configs and test resources
├── docs/
├── Makefile
├── go.mod
└── README.md
For local development and testing, the VK provider can be run directly against a cluster without deploying it to Kubernetes.
- Go 1.23 or later
make build
cisco-vk run \
--config dev/deviceConfig.yaml \
--kubeconfig ~/.kube/config \
--nodename my-test-nodeThe device config file follows the same schema as the CiscoDevice CR spec. See examples for interface/networking options.
Runtime flags:
| Flag | Env Var | Default | Description |
|---|---|---|---|
--nodename |
VK_NODE_NAME |
from config | Kubernetes virtual node name |
--config / -c |
— | (required) | Path to device config YAML |
--kubeconfig |
KUBECONFIG |
in-cluster | Path to kubeconfig file |
--log-level |
LOG_LEVEL |
info |
debug, info, warn, or error |
See CLI Reference for the full flag and environment variable reference.
# Regenerates CRDs → config/crd, RBAC → chart templates, syncs CRDs into chart
make generateContributions are welcome! Please read our Contributing Guide for details on our code of conduct and the process for submitting pull requests.
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
- GitHub Issues: For bug reports and feature requests
- Cisco DevNet: developer.cisco.com
- Virtual Kubelet project
- Cisco IOS-XE and IOx teams