Importing bindings package in an executable used as OCI Hook always fails in rootless

[AbdelrahmanElawady]

Issue Description

Having github.com/containers/podman/v5/pkg/bindings/containers as one of the imports in a Go executable will make it fail if used as an OCI hook when running Podman rootless. It doesn't show any error other than execution failed: Error: OCI runtime error: error executing hook /hook (exit code: 1).
Example code:

package main

import (
        "os"

        _ "github.com/containers/podman/v5/pkg/bindings/containers"
)

func main() {
        if err := os.WriteFile("/tmp/test", []byte("123"), 0644); err != nil {
                panic(err)
        }
}

When running this outside of Podman (from terminal for example) it works fine and just exits with 0. Using strace to check where the error in the execution happens, I found that the executable tries to access /root/.config/containers/containers.conf.d and gets permission denied although its running rootless so not sure why it is trying to access /root. My guess is that some init() function in the imports use some env var set by Podman which cause it to look for config in this directory but I am not sure.

Steps to reproduce the issue

Steps to reproduce the issue

1. Import github.com/containers/podman/v5/pkg/bindings/containers in a Go file and build it. For example:

package main

import (
        "os"

        _ "github.com/containers/podman/v5/pkg/bindings/containers"
)

func main() {
        if err := os.WriteFile("/tmp/test", []byte("123"), 0644); err != nil {
                panic(err)
        }
}

2. Add the result binary as prestart hook
3. Run podman run --rm alpine for example and it should fail
4. Remove the import or run as root and it should work
5. To check where the access issue is you can try strace -f -s 1000 -o strace.out podman run --rm alpine (or strace -ff for better readability but be careful it will produce a lot of files). Check where execve happens with the hook executable and it will exit after trying to access /root/.config/containers/containers.conf.d and prints this (to /dev/null): time="2024-11-06T14:39:09Z" level=error msg="finding config on system: lstat /root/.config/containers/containers.conf.d: permission denied"

Describe the results you received

Got this error while trying podman run --rm alpine:
Error: OCI runtime error: error executing hook /hook (exit code: 1)

Describe the results you expected

To be able to use the bindings package in an OCI hook in Podman with no issues. The reason for that is we are trying to fetch some data from Podman based on some containers lifecycle so that's why we are using bindings packages.

podman info output

host: [55/1947]
arch: amd64
buildahVersion: 1.23.1
cgroupControllers:

• memory
• pids
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
  package: 'conmon: /usr/bin/conmon'
  path: /usr/bin/conmon
  version: 'conmon version 2.0.25, commit: unknown'
  cpus: 2
  distribution:
  codename: jammy
  distribution: ubuntu
  version: "22.04"
  eventLogger: journald
  hostname: tot
  idMappings:
  gidmap:
  • container_id: 0
    host_id: 1000
    size: 1
  • container_id: 1
    host_id: 100000
    size: 65536
    uidmap:
  • container_id: 0
    host_id: 1000
    size: 1
  • container_id: 1
    host_id: 100000
    size: 65536
    kernel: 5.15.0-52-generic
    linkmode: dynamic
    logDriver: journald
    memFree: 549412864
    memTotal: 4114452480
    ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
    crun version 0.17
    commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
    spec: 1.0.0
    +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
    os: linux
    remoteSocket:
    path: /tmp/podman-run-1000/podman/podman.sock
    security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
    serviceIsRemote: false
    slirp4netns:
    executable: /usr/bin/slirp4netns
    package: 'slirp4netns: /usr/bin/slirp4netns'
    version: |-
    slirp4netns version 1.0.1
    commit: 6a7b16babc95b6a3056b33fb45b74a6f62262dd4
    libslirp: 4.6.1
    swapFree: 0
    swapTotal: 0
    uptime: 20h 30m 46.68s (Approximately 0.83 days)
    plugins:
    log:
• k8s-file
• none
• journald
  network:
• bridge
• macvlan
  volume:
• local
  registries: {}
  store:
  configFile: /home/test/.config/containers/storage.conf
  containerStore:
  number: 0
  paused: 0
  running: 0
  stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/test/.local/share/containers/storage
  graphStatus:
  Backing Filesystem: extfs
  Native Overlay Diff: "true"
  Supports d_type: "true"
  Using metacopy: "false"
  imageStore:
  number: 1
  runRoot: /tmp/podman-run-1000/containers
  volumePath: /home/test/.local/share/containers/storage/volumes
  version:
  APIVersion: 3.4.4
  Built: 0
  BuiltTime: Thu Jan 1 00:00:00 1970
  GitCommit: ""
  GoVersion: go1.18.1
  OsArch: linux/amd64
  Version: 3.4.4

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

Fresh Ubuntu 22 VM with Podman and Go 1.23.2 installed

Additional information

Tested on Podman 5.2.4 too with the same issue.
Podman config:

$ cat /etc/containers/containers.conf 
[engine]
hooks_dir = ["/etc/containers/oci/hooks.d"]

Hook:

$ cat /etc/containers/oci/hooks.d/*
{
 "version": "1.0.0",
 "hook": {
"path": "/hook"
 },
 "when": {
     "always": true
 },
 "stages": ["prestart"]
}

[giuseppe]

• Version: 3.4.4

such an old version is not supported upstream. Please try with the latest Podman release

[AbdelrahmanElawady]

yeah tested that on 5.2.4 and same issue

[Luap99]

I would consider this is dup of #23818
The basic problem is the bindings should not even parse things like containers.conf

However what are you trying to do? You should never call podman from within oci hooks as this will cause deadlocks most likely. The issue here is that we run in the podman user namespace without the proper env setup in the hooks which is expected.

[AbdelrahmanElawady]

So the idea is to use the hook for sending containers data to runtime security enforcer KubeArmor.
We use the hook to detect if the enforcer container is created to list current containers on the system and send that to enforcer container (we fork in this case as to not block container startup) that's why we need Podman bindings. Any other non-enforcer containers we use the state input and send that to enforcer container too so we don't need Podman bindings here.

There is no deadlock issues since we don't use the bindings for the current container created but for all other containers in the system and this only happens when the enforcer container is created.

This works fine with CRI-O, Containerd and Rootfull Podman in these PRs:

• feat: Update KubeArmor to use OCI hooks instead of depending on container runtime socket kubearmor/KubeArmor#1714
• feat: Add support for containerd hooks kubearmor/KubeArmor#1763
• support for unorchestrated rootfull and rootless Podman using OCI hook kubearmor/KubeArmor#1874

The issue in Rootless is just the bindings package accessing directories it shouldn't.

But yeah #23818 seems to be the same issue. Thanks!