Hotplugging devices in containers require a service restart

[alexgg]

When the supervisor bind mounts the /dev directory into the container is takes a snapshot of the device's /dev/, and future hotplugged devices are not updated until the container is restarted.

There is a workaround for container with at least CAP_SYS_ADMIN capabilities that is to remount the /dev/ directory:

#!/bin/sh
newdev='/tmp/dev'
mkdir -p "$newdev"
mount -t devtmpfs none "$newdev"
mount --move /dev/console "$newdev/console"
mount --move /dev/mqueue "$newdev/mqueue"
mount --move /dev/pts "$newdev/pts"
mount --move /dev/shm "$newdev/shm"
umount /dev
mount --move "$newdev" /dev
ln -sf /dev/pts/ptmx /dev/ptmx

Relates-to: https://balena.zendesk.com/agent/tickets/2728

[pipex]

@alexgg I don't think this applies, the supervisor never mounts /dev for user containers. Or do you mean for the supervisor itself?

[alexgg]

@pipex maybe it's wrongly phrased - when the --device compose option is passed and the SV mounts the /dev node in the container, these devices need to be already present in the device, so for hotplug to work containers need to be restarted.

[pipex]

@alexgg I believe if the device doesn't exist the engine won't let you start the container. Removing the device won't remove the node in the container /dev, so that will probably crash the program.

[alexgg]

@pipex I see. what this is trying to capture is that hotplugging works when it's the container that remounts the /dev filesystem as events are then handled in the container.

Ideally what we would want would be for applications to be able to specify a non-existent /dev/ device and flag it as hotpluggable in the compose file, and for the container to be setup in a way that /dev/ is remounted but the container does not need SYS_ADMIN capability.

I am guessing there is a mix of engine and supervisor changes to provide frictionless hotplugging support.

[pipex]

Ideally what we would want would be for applications to be able to specify a non-existent /dev/ device and flag it as hotpluggable in the compose file

I see, one way to do that would be for the supervisor to mount devtmpfs within its own container. If a device specified in the composition does not exist, the supervisor starts the container anyway, skipping that device and sets a filesystem watcher for the node. Once the device appears, the supervisor restarts (recreates) the service container, this time adding the device. For this, only supervisor changes would be necessary.

Is not truly hotplug, as the container would have to be restarted, and there is always the risk that the user may remove the device between the supervisor detecting the device and starting the new container