Add support for Docker daemon "-g" start up option to point to LVM-direct mount

[bklau]

Currently, docker-storage-setup assumes that the default location of the "root" filesystem for Docker images/containers to be at "/var/lib/docker". What if I want it to be at another place via "-g" option of Docker daemon to point to say, "/mnt/something" or "/var/something"?

[rhvgoyal]

docker-storage-setup is the one which is setting up the storage for docker. So it should be the one communicating location of images/containers to docker and not vice-a-versa.

So while we do hardcode the location, what's the use case for changing it. I am aware of "-g" option of docker, but who uses it and why and in what context.

[bklau]

@rhvgoyal : some folks mounted alots of host volumes to container, esp legacy softwares. So usually they want to specify the "-g" option for 1 logical volume(non-thin) and then another volume(thin) dedicated to devicemapper.

[rhvgoyal]

Sorry I don't understand. Can you explain it in little bit more detail.

[bklau]

@rhvgoyal : Assume that I have 3 devices : "/dev/sda", "/dev/sdb","/dev/sdc"

1. The primary OS is located on "/dev/sda" , maybe 8GB, say.

2. I want Docker's "/var/lib/docker" to be on "/dev/sdb" which has say, 100GB
   So i mount "/dev/sdb" to say, "/mnt/docker" and set docker daemon to start up with "-g /mnt/docker" option. I do that because "/dev/sda" do not have much diskspace(8GB) to mount a host-to-container volume: "docker run -v "/somedir":"/my_dir" nginx" where "/somedir" is located on "/dev/sda"
   So I used "-g" option to set the Docker root dir to "/dev/sdb" which has 100GB and leave "/dev/sda" alone for the OS.

3. I configured "/dev/sdc" as dedicated DeviceMapper storage driver's "thin pool" location to store container and images.

Hope that's clearer.

[rhvgoyal]

So you are taking care of creating a filesystem on /dev/sdb and mounting it on /mnt/docker? An easy workaround is to mount /dev/sdb at /var/lib/docker/ instead? How about that?

container-storage-setup does not know what's the root location docker wants to use. May be we can introduce an option say DOCKER_ROOT_DIR which will have /var/lib/docker as default but if user has provided a separate path, use that instead.

I did not feel strongly introducing that option as it is little corner case and it is easily fixable by mounting disk on /var/lib/docker instead.

[bklau]

I may not want to mount at "/var/lib/docker" for some reasons.

[rhatdan]

you may be pushing the tooling too far. Most people will be satisfied with mounting on /var/lib/docker trying to figure out just adds a ton of complexity for an incredibly small subset of the users, who most like could figure out how to configure the system themselves.

[rhvgoyal]

How about using new options which allow setting up rootfs for container run time.

Add your disk /dev/vdb and /dev/vdc to a single volume say docker-vg. And specify following.
DEVS=docker-vg
CONTAINER_ROOT_LV_MOUNT_PATH="/mnt/docker"
CONTAINER_ROOT_LV_NAME="docker-root-lv"
CONTAINER_ROOT_LV_SIZE="20G"

I think this will do what you are looking for. Create a separate volume for docker rootfs and mount it at the path you are looking for.

[bklau]

@rhvgoyal : So CONTAINER_ROOT_LV_NAME="docker-root-lv" will used to carve out a LV from the docker_vg for the container rootfs besides the 2 DM "thin " LVs pool data and metadata??.
Pls clarify. thx.

[rhvgoyal]

@bklau, yes. container-root-lv will be carved out and mounted at /mnt/container. And then code will go on to setup another logical volume (thin pool).

IOW, container-storage-setup has been enhanced to handle this use case where a user does not want docker's metadata storage from root file system but from a separate disk.

[bklau]

@rhvgoyal Thanks. Then that will work! :)

[bklau]

@rhvgoyal : How can i install this newer package?.
The old one be installed via:
"sudo yum install -y docker-storage-setup.noarch"

[rhvgoyal]

Package name now is container-storage-setup. what distribution and repository you are using? On fedora 25, I imagine this might be available.

[bklau]

@rhvgoyal : I'm using AWS Linux AMI.

[rhvgoyal]

@bklau I think you will have to talk to amazon for packaging new container-storage-setup in image for that. I can't do anything about that.

[LongLiveCHIEF]

@bklau i've packaged and hosted both rpm and deb formats for this service at https://packagecloud.io/LongLiveCHIEF/container-storage-setup. There are several different methods of installation, but the easiest (for RedHat family OS,s), is to run:

curl -s https://packagecloud.io/install/repositories/LongLiveCHIEF/container-storage-setup/script.rpm.sh | sudo bash

@rhatdan I mirror the git repo and run automated builds to this package hosting service whenever you run new releases. All the image metadata and stuff has your copyrights and licenses in there, I just have my own contact information in the maintainer metadata fields.

[cueedee]

For future visitors who like me are wondering what that -g option (or its --graph long form cousin) does; apparently, it is the former nomer for what is now --data-root;