Skip to content

refactor: Modernize Bicep Secure Baseline with AVM, Workload Identity, AGC, KMS v2, CNI Overlay#189

Merged
matt-FFFFFF merged 34 commits into
mainfrom
update-bicep
Apr 2, 2026
Merged

refactor: Modernize Bicep Secure Baseline with AVM, Workload Identity, AGC, KMS v2, CNI Overlay#189
matt-FFFFFF merged 34 commits into
mainfrom
update-bicep

Conversation

@oZakari

@oZakari oZakari commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Comprehensive refactoring of the AKS Secure Baseline Bicep scenario to align with current AKS security baselines, adopt Azure Verified Modules (AVM), and modernize identity, networking, secrets management, encryption patterns, and availability zone support.

Issues Resolved

Closes #172 — BICEP: Migrate to Microsoft Entra Workload Identity
Closes #173 — BICEP: Replace AGIC with Application Gateway for Containers (AGC) + Gateway API
Closes #175 — BICEP: Enable etcd Encryption with KMS v2 and Private Key Vault
Closes #177 — BICEP: Adopt Azure CNI Overlay and Cilium Dataplane
Closes #179 — BICEP: Integrate Key Vault CSI Driver for Secrets Management
Closes #181 — BICEP: Istio Service Mesh Integration (new scenario)
Closes #185 — feat(bicep): AKS Automatic mode support
Closes #188 — feat(bicep): Namespace building block

Changes by Issue

#172 — Microsoft Entra Workload Identity

  • OIDC issuer + workload identity enabled on both Standard and Automatic cluster modes
  • workload-identity.bicep creates managed identity with federated credential via AVM module
  • K8s manifests: service-account.yaml, secret-provider-class.yaml, sample-workload.yaml all use azure.workload.identity annotations
  • Removed all EnablePodIdentityPreview references from docs
  • Updated README link from deprecated pod-managed identities to Workload Identity

#173 — Application Gateway for Containers + Gateway API

  • Removed: Application Gateway v2 resource, AppGW NSG, AppGW route table, AppGW public IP
  • Added: AGC subnet (10.1.2.0/24) delegated to Microsoft.ServiceNetworking/trafficControllers
  • Added: agc.bicep module deploying AGC traffic controller + subnet association
  • Added: Gateway API manifests (gateway-class.yaml, gateway.yaml, httproute.yaml)
  • Updated All-in-One-Bicep params and module call
  • Updated docs to reference Gateway API instead of AppGW backend pool CLI commands

#175 — etcd Encryption with KMS v2

  • Added KMS encryption key (aks-etcd-kms) to Key Vault in 05-AKS-supporting
  • Added securityProfile.azureKeyVaultKms to both Standard and Automatic cluster modules
  • Added Key Vault Crypto User role assignment for AKS identity
  • Key Vault set to publicNetworkAccess: 'Disabled' with enableRbacAuthorization: true
  • KMS is enabled by default with enableKmsEncryption parameter toggle

#177 — Azure CNI Overlay + Cilium

  • Set networkPluginMode: 'overlay' and networkDataplane: 'cilium' on Standard cluster
  • Removed networkPolicy: 'calico' (Cilium replaces Calico)
  • Pod CIDR set for overlay mode

#179 — Key Vault CSI Driver

  • enableKeyvaultSecretsProvider: true and enableSecretRotation: true on both cluster modes
  • Workload identity grants Key Vault Secrets User role on Key Vault
  • SecretProviderClass manifest uses workload identity (usePodIdentity: "false", clientID)
  • Sample workload mounts secrets as files from Key Vault (no base64 K8s Secrets)

#185 — AKS Automatic Mode

  • Both Standard (Base) and Automatic SKUs supported via aksSkuName parameter
  • Automatic mode uses NAP, KEDA, VPA, managed NAT Gateway, system-assigned identity
  • Comparison table in 06-aks-cluster.md documents trade-offs
  • Deployment command examples for both modes

#188 — Namespace Building Block

  • namespace-building-block.bicep uses AVM Flux extension + configuration modules
  • Kustomize templates: namespace, RBAC, network policy, Istio Gateway
  • GitOps-driven reconciliation from a Git repository

#181 — Istio Service Mesh (new scenario)

  • New AKS-Istio-Mesh/Bicep/main.bicep scenario with AVM managed-cluster
  • Istio service mesh add-on with configurable internal/external ingress gateways
  • Cilium dataplane, workload identity, CSI driver enabled
  • Comprehensive README with architecture diagram

Code Quality Improvements

  • @description() decorators added to all params/outputs across 03-Network-Hub, 04-Network-LZ, 05-AKS-supporting, 06-AKS-cluster
  • @secure() added to jumpbox admin password (extracted from hardcoded Password123)
  • All Bicep files pass diagnostics with zero errors/warnings
  • All-in-One ARM template (main.json) regenerated

Availability Zone Support

  • AKS Standard node pool: availabilityZones parameter defaults to [1, 2, 3], can be set to [] for regions without zone support
  • ACR: zoneRedundancy: 'Enabled' added (requires Premium SKU, already in use)
  • Jumpbox VM: jumpboxAvailabilityZone parameter defaults to 1, can be set to 0 for no zone pinning
  • Azure Firewall & PIPs (03-Hub): Already zone-redundant via availabilityZones parameter
  • AKS Automatic: Zone management handled by Node Auto Provisioning (NAP)

Breaking Changes

⚠️ The following changes are breaking for existing deployments:

  1. Application Gateway removed — The standalone Application Gateway v2, its NSG, route table, and public IP are no longer deployed by 04-Network-LZ. Replaced by AGC traffic controller. Existing deployments using AppGW backend pool wiring will need to migrate to Gateway API manifests.

  2. AppGW subnet renamedAppGWSubnet (10.1.2.0/27) replaced with AGCSubnet (10.1.2.0/24) with subnet delegation. The CIDR range changed from /27 to /24.

  3. Parameters renamed/removed in 04-Network-LZ/main.bicep:

    • appGatewayNameagcName
    • nsgAppGWName — removed
    • rtAppGWSubnetName — removed
    • appGwyAutoScale — removed
    • spokeSubnetAppGWPrefixspokeSubnetAGCPrefix
  4. Jumpbox password now required as parameter04-Network-LZ now has a @secure() jumpboxAdminPassword parameter instead of hardcoded Password123. The All-in-One deployment requires this to be provided at deploy time.

  5. Network policy engine changednetworkPolicy: 'calico' removed; Cilium dataplane handles network policy. Existing Calico NetworkPolicy CRDs should still work but Calico-specific features (e.g., GlobalNetworkPolicy) will not.

  6. KMS v2 enabled by default — etcd encryption is on by default. Requires Key Vault with a key named aks-etcd-kms. Disable with enableKmsEncryption=false if not needed.

  7. New required parameter in All-in-OnejumpboxAdminPassword (secure string) must be provided.

  8. AKS node pool zones changed — Standard cluster node pool now spans zones [1, 2, 3] (previously only zone 3). Set availabilityZones=[] for regions without zone support.

Testing

  • All .bicep files pass Bicep diagnostics (zero errors, zero warnings)
  • All-in-One ARM template regenerated successfully
  • End-to-end deployment with default parameters (requires Azure subscription)

@oZakari
oZakari marked this pull request as draft March 10, 2026 00:08
@oZakari oZakari changed the title refactor: Update Bicep scenarios for latest AVM versions and other refactoring refactor: Modernize Bicep Secure Baseline with AVM, Workload Identity, AGC, KMS v2, CNI Overlay Mar 18, 2026
@oZakari oZakari added the bicep label Mar 19, 2026
@oZakari
oZakari marked this pull request as ready for review March 25, 2026 02:57
@matt-FFFFFF
matt-FFFFFF merged commit e6c71c1 into main Apr 2, 2026
2 checks passed
@matt-FFFFFF
matt-FFFFFF deleted the update-bicep branch April 2, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment