Unable to get docker output

[mo36924]

Bug Report

Describe the bug
I'm unable to get docker output when using the docker input.
No log is output when using cgroup v2 and cgroupfs cgroup driver.

To Reproduce
Run: sudo docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/containers/:/var/lib/docker/containers/ -v /sys/fs/cgroup:/sys/fs/cgroup -ti fluent/fluent-bit:3.2 /fluent-bit/bin/fluent-bit -i docker -o stdout

[2024/12/27 07:39:10] [ info] [fluent bit] version=3.2.3, commit=44a2d0d43f, pid=1
[2024/12/27 07:39:10] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/12/27 07:39:10] [ info] [simd    ] disabled
[2024/12/27 07:39:10] [ info] [cmetrics] version=0.9.9
[2024/12/27 07:39:10] [ info] [ctraces ] version=0.5.7
[2024/12/27 07:39:10] [ info] [input:docker:docker.0] initializing
[2024/12/27 07:39:10] [ info] [input:docker:docker.0] storage_strategy='memory' (memory only)
[2024/12/27 07:39:10] [ info] [input:docker:docker.0] Detected cgroups v2
[2024/12/27 07:39:10] [ info] [sp] stream processor started
[2024/12/27 07:39:10] [ info] [output:stdout:stdout.0] worker #0 started

Your Environment

• Version used: Fluent Bit v3.2.3
• Configuration: -i docker -o stdout
• Docker version: Docker version 27.4.1

sudo docker info | grep 'Cgroup Version'
# Cgroup Version: 2
sudo docker info | grep 'Cgroup Driver'
# Cgroup Driver: cgroupfs

If you change to systemd's Cgroup Driver, logs will be output normally.
/etc/docker/daemon.json

{
  "exec-opts": ["native.cgroupdriver=cgroupfs"]
}

↓

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

sudo systemctl restart docker.service

Run: sudo docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/containers/:/var/lib/docker/containers/ -v /sys/fs/cgroup:/sys/fs/cgroup -ti fluent/fluent-bit:3.2 /fluent-bit/bin/fluent-bit -i docker -o stdout

[2024/12/27 07:41:52] [ info] [fluent bit] version=3.2.3, commit=44a2d0d43f, pid=1
[2024/12/27 07:41:52] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/12/27 07:41:52] [ info] [simd    ] disabled
[2024/12/27 07:41:52] [ info] [cmetrics] version=0.9.9
[2024/12/27 07:41:52] [ info] [ctraces ] version=0.5.7
[2024/12/27 07:41:52] [ info] [input:docker:docker.0] initializing
[2024/12/27 07:41:52] [ info] [input:docker:docker.0] storage_strategy='memory' (memory only)
[2024/12/27 07:41:52] [ info] [input:docker:docker.0] Detected cgroups v2
[2024/12/27 07:41:52] [ info] [sp] stream processor started
[2024/12/27 07:41:52] [ info] [output:stdout:stdout.0] worker #0 started
[0] docker.0: [[1735285312.468842561, {}], {"id"=>"062d9b937d1d", "name"=>"zealous_jennings", "cpu_used"=>76003, "mem_used"=>23629824, "mem_limit"=>18446744073709551615}]
[0] docker.0: [[1735285313.468766782, {}], {"id"=>"062d9b937d1d", "name"=>"zealous_jennings", "cpu_used"=>76820, "mem_used"=>23896064, "mem_limit"=>18446744073709551615}]
...

Looking at the code, it seems that the combination of cgroup v1 and systemd driver, and the combination of cgroup v2 and cgroupfs driver do not output logs.

ref: https://docs.docker.com/engine/containers/runmetrics/#find-the-cgroup-for-a-given-container

fluent-bit/plugins/in_docker/docker.h

Lines 46 to 59 in d77c06d

	/* Files from sysfs containing metrics (cgroups v1) */
	#define DOCKER_CGROUP_V1_MEM_DIR "memory/docker"
	#define DOCKER_CGROUP_V1_CPU_DIR "cpu/docker"
	#define DOCKER_CGROUP_V1_MEM_LIMIT_FILE "memory.limit_in_bytes"
	#define DOCKER_CGROUP_V1_MEM_USAGE_FILE "memory.usage_in_bytes"
	#define DOCKER_CGROUP_V1_CPU_USAGE_FILE "cpuacct.usage"
	/* Files from sysfs containing metrics (cgroups v2) */
	#define DOCKER_CGROUP_V2_DOCKER_SERVICE_DIR "system.slice"
	#define DOCKER_CGROUP_V2_MEM_USAGE_FILE "memory.current"
	#define DOCKER_CGROUP_V2_MEM_MAX_FILE "memory.max"
	#define DOCKER_CGROUP_V2_CPU_USAGE_FILE "cpu.stat"
	#define DOCKER_CGROUP_V2_CPU_USAGE_KEY "usage_usec"
	#define DOCKER_CGROUP_V2_CPU_USAGE_TEMPLATE DOCKER_CGROUP_V2_CPU_USAGE_KEY" %lu"

ref: #3882

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.