Add support for Docker daemon "-g" start up option to point to LVM-direct mount

[bklau]

Currently, docker-storage-setup assumes that the default location of the "root" filesystem for Docker images/containers to be at "/var/lib/docker". What if I want it to be at another place via "-g" option of Docker daemon to point to say, "/mnt/something" or "/var/something"?

[rhvgoyal]

docker-storage-setup is the one which is setting up the storage for docker. So it should be the one communicating location of images/containers to docker and not vice-a-versa.

So while we do hardcode the location, what's the use case for changing it. I am aware of "-g" option of docker, but who uses it and why and in what context.

[bklau]

@rhvgoyal : some folks mounted alots of host volumes to container, esp legacy softwares. So usually they want to specify the "-g" option for 1 logical volume(non-thin) and then another volume(thin) dedicated to devicemapper.

[rhvgoyal]

Sorry I don't understand. Can you explain it in little bit more detail.

[bklau]

@rhvgoyal : Assume that I have 3 devices : "/dev/sda", "/dev/sdb","/dev/sdc"

1. The primary OS is located on "/dev/sda" , maybe 8GB, say.

2. I want Docker's "/var/lib/docker" to be on "/dev/sdb" which has say, 100GB
   So i mount "/dev/sdb" to say, "/mnt/docker" and set docker daemon to start up with "-g /mnt/docker" option. I do that because "/dev/sda" do not have much diskspace(8GB) to mount a host-to-container volume: "docker run -v "/somedir":"/my_dir" nginx" where "/somedir" is located on "/dev/sda"
   So I used "-g" option to set the Docker root dir to "/dev/sdb" which has 100GB and leave "/dev/sda" alone for the OS.

3. I configured "/dev/sdc" as dedicated DeviceMapper storage driver's "thin pool" location to store container and images.

Hope that's clearer.

[rhvgoyal]

So you are taking care of creating a filesystem on /dev/sdb and mounting it on /mnt/docker? An easy workaround is to mount /dev/sdb at /var/lib/docker/ instead? How about that?

container-storage-setup does not know what's the root location docker wants to use. May be we can introduce an option say DOCKER_ROOT_DIR which will have /var/lib/docker as default but if user has provided a separate path, use that instead.

I did not feel strongly introducing that option as it is little corner case and it is easily fixable by mounting disk on /var/lib/docker instead.

[bklau]

I may not want to mount at "/var/lib/docker" for some reasons.

[rhatdan]

you may be pushing the tooling too far. Most people will be satisfied with mounting on /var/lib/docker trying to figure out just adds a ton of complexity for an incredibly small subset of the users, who most like could figure out how to configure the system themselves.

[rhvgoyal]

How about using new options which allow setting up rootfs for container run time.

Add your disk /dev/vdb and /dev/vdc to a single volume say docker-vg. And specify following.
DEVS=docker-vg
CONTAINER_ROOT_LV_MOUNT_PATH="/mnt/docker"
CONTAINER_ROOT_LV_NAME="docker-root-lv"
CONTAINER_ROOT_LV_SIZE="20G"

I think this will do what you are looking for. Create a separate volume for docker rootfs and mount it at the path you are looking for.

[bklau]

@rhvgoyal : So CONTAINER_ROOT_LV_NAME="docker-root-lv" will used to carve out a LV from the docker_vg for the container rootfs besides the 2 DM "thin " LVs pool data and metadata??.
Pls clarify. thx.

[rhvgoyal]

@bklau, yes. container-root-lv will be carved out and mounted at /mnt/container. And then code will go on to setup another logical volume (thin pool).

IOW, container-storage-setup has been enhanced to handle this use case where a user does not want docker's metadata storage from root file system but from a separate disk.

[bklau]

@rhvgoyal Thanks. Then that will work! :)

[bklau]

@rhvgoyal : How can i install this newer package?.
The old one be installed via:
"sudo yum install -y docker-storage-setup.noarch"

[rhvgoyal]

Package name now is container-storage-setup. what distribution and repository you are using? On fedora 25, I imagine this might be available.

[bklau]

@rhvgoyal : I'm using AWS Linux AMI.

[rhvgoyal]

@bklau I think you will have to talk to amazon for packaging new container-storage-setup in image for that. I can't do anything about that.