Is there an existing issue for this?
Environment
Component: runtime-enforcer-agent (DaemonSet)
Cluster: RKE2
Container Runtime: containerd (NRI enabled)
Issue Description
Description
During runtime-enforcer-agent startup, daemonset rollout, or chart upgrade, the agent fails to reach Ready status if any pod in the cluster carries a security.rancher.io/policy: annotation referencing a WorkloadPolicy custom resource that does not exist in the cluster.
When the NRI pod/container synchronization handler encounters an annotated pod whose referenced policy cannot be found, it treats the missing resource as a blocking fatal error. This causes the ttrpc server to shut down, the /healthz check to fail, and the agent container to crash into an unrecoverable CrashLoopBackOff.
Expected Behavior
The agent startup should handle missing WorkloadPolicy references gracefully. If a pod references an un-applied or deleted policy, the agent should log a non-fatal warning/event, fall back to default monitoring mode for that specific pod, and allow the agent manager to complete startup and reach 1/1 Ready.
Actual Behavior
The missing policy causes a hard failure during NRI synchronization, shutting down the ttrpc server and causing health checks to fail.
Logs
"NRI pod/container sync not ready yet, will retry"
"error":"failed to apply policy to pod: pod 'cattle-system/rancher-64d988fb8b-rgcdv' has policy 'deploy-rancher' associated, but the policy does not exist"
"ttrpc server closed 00-runtime-enforcer-agent : ttrpc: server closed"
"healthz check failed"
Steps to Reproduce
Annotate a workload with a policy key referencing a non-existent policy
kubectl annotate pod <pod-name> -n <namespace> security.rancher.io/policy=deploy-rancher
Do not create the corresponding WorkloadPolicy custom resource (deploy-rancher).
Perform a rollout or restart of the runtime-enforcer-agent DaemonSet
kubectl rollout restart ds runtime-enforcer-agent -n runtime-enforcer
Observe the agent pod logs and status (kubectl get pods -n runtime-enforcer).
Is there an existing issue for this?
Environment
Issue Description
Description
During runtime-enforcer-agent startup, daemonset rollout, or chart upgrade, the agent fails to reach Ready status if any pod in the cluster carries a security.rancher.io/policy: annotation referencing a WorkloadPolicy custom resource that does not exist in the cluster.
When the NRI pod/container synchronization handler encounters an annotated pod whose referenced policy cannot be found, it treats the missing resource as a blocking fatal error. This causes the ttrpc server to shut down, the /healthz check to fail, and the agent container to crash into an unrecoverable CrashLoopBackOff.
Expected Behavior
The agent startup should handle missing WorkloadPolicy references gracefully. If a pod references an un-applied or deleted policy, the agent should log a non-fatal warning/event, fall back to default monitoring mode for that specific pod, and allow the agent manager to complete startup and reach 1/1 Ready.
Actual Behavior
The missing policy causes a hard failure during NRI synchronization, shutting down the ttrpc server and causing health checks to fail.
Logs
Steps to Reproduce
Annotate a workload with a policy key referencing a non-existent policy
kubectl annotate pod <pod-name> -n <namespace> security.rancher.io/policy=deploy-rancherDo not create the corresponding WorkloadPolicy custom resource (deploy-rancher).
Perform a rollout or restart of the runtime-enforcer-agent DaemonSet
kubectl rollout restart ds runtime-enforcer-agent -n runtime-enforcerObserve the agent pod logs and status (
kubectl get pods -n runtime-enforcer).