Skip to content

Add apt clean to reduce layer size #1298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add apt clean to reduce layer size #1298

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions src/docker-in-docker/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ set -e

# Clean up
rm -rf /var/lib/apt/lists/*
apt clean
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this not make sense as it only removes the installation stuff in the previous layer?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I don't quite understand why this script cleans at the start and then again at the end 🤷

As a start I see an existing clean up, I add my change. If someone with a better overview of the project points me in a different direction to solve this at a higher level so that the apt cache is never stored in the image (e.g. using RUN --mount=type=cache,target=/var/cache/apt --mount=type=cache,target=/var/lib/apt/lists everywhere), I'd be happy to also do that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the removal of /var/lib/apt/lists/* is done because subsequent installations may fail if the cache remains in the lower layers.
The size of the image is irrelevant.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on "The size of the image is irrelevant"? My team is paying for image storage by the byte and devcontainer download time is a regular annoyance, so image size is very much of concern for us.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check the context #210.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I'm trying to say is that I'm wondering if it's worth adding this at the beginning of the script.
(Because it doesn't affect the layer size)

Copy link
Contributor

@Kaniska244 Kaniska244 Apr 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @DavidS-ovm ,

Thank you for the PR. I was trying to test docker-in-docker feature with this PR change. In fact I see that with apt clean right at the beginning of the install.sh, package installation failing. It doesn't remove the directory structure or the lock file, so later on the following condition in apt_get_update() function body doesn't satisfy & apt-get update -y doesn't execute before installing the additional packages for docker-in-docker feature.

apt_get_update()
{
    if [ "$(find /var/lib/apt/lists/* | wc -l)" = "0" ]; then
        echo "Running apt-get update..."
        apt-get update -y
    fi
}

It would be best to have the clean-up done once & done right at the end of the install.sh execution & that should serve the purpose of reducing the layer size in the resultant image. Kindly let me know in case of further concern.

With Regards,
Kaniska

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the analysis! This is indeed more complex than I initially expected. Meanwhile I've done some local optimizations for other containers in my work and I'm now using

RUN --mount=type=cache,target=/var/cache/apt \
    --mount=type=cache,target=/var/lib/apt/lists \
    [...]

any time apt is getting called. This ensures that

  • apt caches and metadata are never baked into any layer
  • individual steps can re-use previous steps' already downloaded data, and
  • if buildx is appropriately configured, the cache can be used across multiple builds, further improving efficiency if requested
  • individual steps don't need to care about refreshing and cleaning apt metadata

Poking around in the devcontainer CLI, this seems to be the appropriate location to add that:

https://github.com/devcontainers/cli/blob/5c9623c78d17ee4f2d75899c1d9c59a9b266611e/src/spec-configuration/containerFeaturesConfiguration.ts#L292-L357

additionally there's getContainerFeaturesBaseDockerFile in the same file that looks like a good place to apt update once for the entire build.

Does that sound sensible?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

devcontainers/cli#989 as an experiment

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @DavidS-ovm ,

Thank you for the contribution & the detailed analysis on this issue. I will check this further & update you.


# Setup STDERR.
err() {
Expand Down Expand Up @@ -427,6 +428,7 @@ if [ -f "/usr/local/share/docker-init.sh" ]; then
echo "/usr/local/share/docker-init.sh already exists, so exiting."
# Clean up
rm -rf /var/lib/apt/lists/*
apt clean
exit 0
fi
echo "docker-init doesn't exist, adding..."
Expand Down Expand Up @@ -640,5 +642,6 @@ chown ${USERNAME}:root /usr/local/share/docker-init.sh

# Clean up
rm -rf /var/lib/apt/lists/*
apt clean

echo 'docker-in-docker-debian script has completed!'