From 97d895bf4df7cf3e26689c9839474918b6226bc6 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 13:42:36 -0500 Subject: [PATCH 01/31] Remove references to dedicated ROS distros as final ROS 1 version, Noetic, is now EOL --- ros/content.md | 48 ++---------------------------------------------- 1 file changed, 2 insertions(+), 46 deletions(-) diff --git a/ros/content.md b/ros/content.md index 86033fbbfda8..d1b6ecd12405 100644 --- a/ros/content.md +++ b/ros/content.md @@ -119,7 +119,7 @@ Developing such complex systems with cutting edge implementations of newly publi With the advancements and standardization of software containers, roboticists are primed to acquire a host of improved developer tooling for building and shipping software. To help alleviate the growing pains and technical challenges of adopting new practices, we have focused on providing an official resource for using ROS with these new technologies. -For a complete listing of supported architectures and base images for each ROS Distribution Release, please read the official REP on target platforms for either [ROS 1](https://www.ros.org/reps/rep-0003.html) or for [ROS 2](https://www.ros.org/reps/rep-2000.html). +For a complete listing of supported architectures and base images for each ROS Distribution Release, please read the official REP on target platforms [here](https://www.ros.org/reps/rep-2001.html). ## Deployment suggestions @@ -127,11 +127,10 @@ The available tags include supported distros along with a hierarchy tags based o - `ros-core`: minimal ROS install - `ros-base`: basic tools and libraries (also tagged with distro name with LTS version as `latest`) -- `ros1-bridge`: tools and libraries to run hybrid ROS 1 - ROS 2 systems and bridge messages between them In the interest of keeping `ros-core` tag minimal in image size, developer tools such as `rosdep`, `colcon` and `vcstools` are not shipped in `ros_core`, but in `ros-base` instead. -The rest of the common meta-packages such as `desktop` are hosted on repos under OSRF's Docker Hub profile [here](https://hub.docker.com/r/osrf/ros/). These meta-packages include graphical dependencies and hook a host of other large packages such as X11, X server, etc. So in the interest of keeping the official images lean and secure, the desktop packages are just being hosted with OSRF's profile. For an extensive list of available variants, please read the official REP on target platforms for either [ROS 1](https://ros.org/reps/rep-0150.html) or for [ROS 2](https://www.ros.org/reps/rep-2001.html). +The rest of the common meta-packages such as `desktop` are hosted on repos under OSRF's Docker Hub profile [here](https://hub.docker.com/r/osrf/ros/). These meta-packages include graphical dependencies and hook a host of other large packages such as X11, X server, etc. So in the interest of keeping the official images lean and secure, the desktop packages are just being hosted with OSRF's profile. ### Volumes @@ -201,45 +200,6 @@ $ docker compose rm > Note: the auto-generated network, `ros_demos_default`, will persist until you explicitly remove it using `docker compose down`. -### ROS 1 Bridge - -To ease ROS 2 migration, [`ros1_bridge`](https://index.ros.org/p/ros1_bridge) is a ROS 2 package that provides bidirectional communication between ROS 1 and ROS 2. As a minimal example, given the ROS 2 Dockerfile above, we'll create the ROS 1 equivalent below, and name the Dockerfile appropriately. - -```dockerfile -FROM %%IMAGE%%:noetic - -# install ros package -RUN apt-get update && apt-get install -y \ - ros-${ROS_DISTRO}-ros-tutorials \ - ros-${ROS_DISTRO}-common-tutorials && \ - rm -rf /var/lib/apt/lists/* - -# launch ros package -CMD ["roslaunch", "roscpp_tutorials", "talker_listener_launch"] -``` - -The compose file bellow spawns services for both talker listener demos while connecting the two via a dynamic bridge. You may then view the log output from both pairs of talker and listener nodes cross talking over the `/chatter` topic. - -```yaml -services: - ros1: - build: - context: ./ - dockerfile: ros1.Dockerfile - - ros2: - build: - context: ./ - dockerfile: ros2.Dockerfile - - bridge: - image: ros:foxy-ros1-bridge - environment: - - "ROS_HOSTNAME=bridge" - - "ROS_MASTER_URI=http://ros1:11311" - command: ros2 run ros1_bridge dynamic_bridge -``` - # More Resources [ROS.org](http://www.ros.org/): Main ROS website @@ -253,7 +213,3 @@ services: [Index](https://docs.ros.org): ROS 2 Documentation [Design](https://design.ros2.org/): ROS 2 Design Articles - -## ROS 1 - -[Wiki](http://wiki.ros.org/Documentation): ROS 1 Documentation From 78f15a45177dc91f8e5e0aee4001942fd1448ab7 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 14:03:35 -0500 Subject: [PATCH 02/31] Redirect primary links to documentation landing page as docs page is more up-to-date and pertinent for users --- ros/content.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ros/content.md b/ros/content.md index d1b6ecd12405..45d2622f8b1a 100644 --- a/ros/content.md +++ b/ros/content.md @@ -1,10 +1,10 @@ -# What is [ROS](https://www.ros.org/)? +# What is [ROS](https://docs.ros.org/)? The Robot Operating System (ROS) is a set of software libraries and tools that help you build robot applications. From drivers to state-of-the-art algorithms, and with powerful developer tools, ROS has what you need for your next robotics project. And it's all open source. > [wikipedia.org/wiki/Robot_Operating_System](https://en.wikipedia.org/wiki/Robot_Operating_System) -[%%LOGO%%](https://www.ros.org/) +[%%LOGO%%](https://docs.ros.org/) # How to use this image @@ -202,7 +202,7 @@ $ docker compose rm # More Resources -[ROS.org](http://www.ros.org/): Main ROS website +[Docs](https://docs.ros.org/): ROS Developer Documentation [Q&A](https://answers.ros.org/questions/): Ask questions. Get answers [Forums](https://discourse.ros.org/): Hear the latest discussions [Blog](http://www.ros.org/news/): Stay up-to-date From 45ae7874073c4fa5d4d82751af249687aef9e06d Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 14:05:35 -0500 Subject: [PATCH 03/31] Remove retired ROS blog - https://www.ros.org/news/2020/01/ros-blog-retired.html --- ros/content.md | 1 - 1 file changed, 1 deletion(-) diff --git a/ros/content.md b/ros/content.md index 45d2622f8b1a..22fbb878b6b5 100644 --- a/ros/content.md +++ b/ros/content.md @@ -205,7 +205,6 @@ $ docker compose rm [Docs](https://docs.ros.org/): ROS Developer Documentation [Q&A](https://answers.ros.org/questions/): Ask questions. Get answers [Forums](https://discourse.ros.org/): Hear the latest discussions -[Blog](http://www.ros.org/news/): Stay up-to-date [Packages](https://index.ros.org/?search_packages=true): Discover indexed packages [OSRF](https://www.osrfoundation.org/): Open Source Robotics Foundation From a608cd5870f835e7e5fd904c4e845101e81380da Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 14:07:41 -0500 Subject: [PATCH 04/31] Update OSRF URL redirect --- ros/content.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ros/content.md b/ros/content.md index 22fbb878b6b5..50cc833e636b 100644 --- a/ros/content.md +++ b/ros/content.md @@ -206,7 +206,7 @@ $ docker compose rm [Q&A](https://answers.ros.org/questions/): Ask questions. Get answers [Forums](https://discourse.ros.org/): Hear the latest discussions [Packages](https://index.ros.org/?search_packages=true): Discover indexed packages -[OSRF](https://www.osrfoundation.org/): Open Source Robotics Foundation +[OSRF](https://www.openrobotics.org/): Open Source Robotics Foundation ## ROS 2 From b02e4224ab6a8b4291d7e049e9fac108e7ccc09f Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 14:10:28 -0500 Subject: [PATCH 05/31] Update example image tag to rolling to avoid docs from becoming stale --- ros/content.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ros/content.md b/ros/content.md index 50cc833e636b..7b8a3b0fc70f 100644 --- a/ros/content.md +++ b/ros/content.md @@ -13,7 +13,7 @@ The Robot Operating System (ROS) is a set of software libraries and tools that h To create your own ROS docker images and install custom packages, here's a simple example of installing the C++, Python client library demos using the official released Debian packages via apt-get. ```dockerfile -FROM %%IMAGE%%:foxy +FROM %%IMAGE%%:rolling # install ros package RUN apt-get update && apt-get install -y \ @@ -44,7 +44,7 @@ $ docker run -it --rm my/ros:app To create your own ROS docker images and build custom packages, here's a simple example of installing a package's build dependencies, compiling it from source, and installing the resulting build artifacts into a final multi-stage image layer. ```dockerfile -ARG FROM_IMAGE=%%IMAGE%%:foxy +ARG FROM_IMAGE=%%IMAGE%%:rolling ARG OVERLAY_WS=/opt/ros/overlay_ws # multi-stage for caching From beb06637623ed6849d854de3130e1a9b2f87867e Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 14:13:31 -0500 Subject: [PATCH 06/31] Consolidate branding and dated references --- ros/content.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/ros/content.md b/ros/content.md index 7b8a3b0fc70f..63fbca424c12 100644 --- a/ros/content.md +++ b/ros/content.md @@ -109,7 +109,7 @@ The example above starts by using [`vcstool`](https://github.com/dirk-thomas/vcs Note: `--from-paths` and `--packages-select` are set here as so to only install the dependencies and build for the demo C++ and Python packages, among many in the demo git repo that was cloned. To install the dependencies and build all the packages in the source workspace, merely change the scope by setting `--from-paths src/` and dropping the `--packages-select` arguments. -For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the ROS 2 [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. +For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. ## Deployment use cases @@ -207,8 +207,3 @@ $ docker compose rm [Forums](https://discourse.ros.org/): Hear the latest discussions [Packages](https://index.ros.org/?search_packages=true): Discover indexed packages [OSRF](https://www.openrobotics.org/): Open Source Robotics Foundation - -## ROS 2 - -[Index](https://docs.ros.org): ROS 2 Documentation -[Design](https://design.ros2.org/): ROS 2 Design Articles From 62341edf40c6b26593d5f0bef60c43c64d608e33 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 14:24:02 -0500 Subject: [PATCH 07/31] Update Q&A links to stackexchange given migration - https://discourse.ros.org/t/planned-migration-of-ros-answers-and-gazebo-answers-to-robotics-stack-exchange/28068 - https://discourse.ros.org/t/new-static-archives-for-ros-and-gazebo-answers/41346 --- ros/content.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ros/content.md b/ros/content.md index 63fbca424c12..62a067e33e75 100644 --- a/ros/content.md +++ b/ros/content.md @@ -203,7 +203,7 @@ $ docker compose rm # More Resources [Docs](https://docs.ros.org/): ROS Developer Documentation -[Q&A](https://answers.ros.org/questions/): Ask questions. Get answers +[Q&A](https://robotics.stackexchange.com/): Ask questions. Get answers [Forums](https://discourse.ros.org/): Hear the latest discussions [Packages](https://index.ros.org/?search_packages=true): Discover indexed packages [OSRF](https://www.openrobotics.org/): Open Source Robotics Foundation From d34dc62c7fcf6217a60e03d476875bae0b7fa53d Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 17:01:07 -0500 Subject: [PATCH 08/31] Simplify cloning overlay source --- ros/content.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/ros/content.md b/ros/content.md index 62a067e33e75..f8cfed227bc8 100644 --- a/ros/content.md +++ b/ros/content.md @@ -53,14 +53,13 @@ FROM $FROM_IMAGE AS cacher # clone overlay source ARG OVERLAY_WS WORKDIR $OVERLAY_WS/src -RUN echo "\ -repositories: \n\ - ros2/demos: \n\ - type: git \n\ - url: https://github.com/ros2/demos.git \n\ - version: ${ROS_DISTRO} \n\ -" > ../overlay.repos -RUN vcs import ./ < ../overlay.repos +RUN cat < Date: Wed, 28 May 2025 17:03:24 -0500 Subject: [PATCH 09/31] Simplify use of mixin setting --- ros/content.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/ros/content.md b/ros/content.md index f8cfed227bc8..67272cd8f91e 100644 --- a/ros/content.md +++ b/ros/content.md @@ -86,13 +86,12 @@ RUN . /opt/ros/$ROS_DISTRO/setup.sh && \ # build overlay source COPY --from=cacher $OVERLAY_WS/src ./src -ARG OVERLAY_MIXINS="release" RUN . /opt/ros/$ROS_DISTRO/setup.sh && \ colcon build \ --packages-select \ demo_nodes_cpp \ demo_nodes_py \ - --mixin $OVERLAY_MIXINS + --mixin release # source entrypoint setup ENV OVERLAY_WS $OVERLAY_WS From 993af9c71fb687c63fc1ae7685490b3091986ce5 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 17:06:11 -0500 Subject: [PATCH 10/31] Split runner and builder stages as example to save on image size as well as on build time using caching --- ros/content.md | 62 ++++++++++++++++++++++++++++++++++---------------- 1 file changed, 42 insertions(+), 20 deletions(-) diff --git a/ros/content.md b/ros/content.md index 67272cd8f91e..92f5e6eb2367 100644 --- a/ros/content.md +++ b/ros/content.md @@ -61,30 +61,41 @@ repositories: version: ${ROS_DISTRO} EOF -# copy manifests for caching -WORKDIR /opt -RUN mkdir -p /tmp/opt && \ - find ./ -name "package.xml" | \ - xargs cp --parents -t /tmp/opt && \ - find ./ -name "COLCON_IGNORE" | \ - xargs cp --parents -t /tmp/opt || true +# update apt cache +RUN rm /etc/apt/apt.conf.d/docker-clean && apt-get update + +# derive build/exec dependencies +SHELL ["/bin/bash", "-o", "pipefail", "-c"] +RUN dep_types=(\ + "exec:--dependency-types=exec" \ + "build:") && \ + for dep_type in "${dep_types[@]}"; do \ + IFS=":"; set -- $dep_type; \ + rosdep install -y \ + --from-paths \ + ros2/demos/demo_nodes_cpp \ + ros2/demos/demo_nodes_py \ + --ignore-src \ + --reinstall \ + --simulate \ + ${2} \ + | grep 'apt-get install' \ + | awk -F' ' '{print $4}' | sed "s/'//g" \ + | sort > /tmp/${1}_debs.txt; \ + done # multi-stage for building FROM $FROM_IMAGE AS builder -# install overlay dependencies -ARG OVERLAY_WS -WORKDIR $OVERLAY_WS -COPY --from=cacher /tmp/$OVERLAY_WS/src ./src -RUN . /opt/ros/$ROS_DISTRO/setup.sh && \ - apt-get update && rosdep install -y \ - --from-paths \ - src/ros2/demos/demo_nodes_cpp \ - src/ros2/demos/demo_nodes_py \ - --ignore-src \ - && rm -rf /var/lib/apt/lists/* +# install build dependencies +COPY --from=cacher /tmp/build_debs.txt /tmp/build_debs.txt +RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ + --mount=type=cache,from=cacher,target=/var/lib/apt/lists,source=/var/lib/apt/lists \ + < /tmp/build_debs.txt xargs apt-get install -y --no-install-recommends # build overlay source +ARG OVERLAY_WS +WORKDIR $OVERLAY_WS COPY --from=cacher $OVERLAY_WS/src ./src RUN . /opt/ros/$ROS_DISTRO/setup.sh && \ colcon build \ @@ -93,8 +104,19 @@ RUN . /opt/ros/$ROS_DISTRO/setup.sh && \ demo_nodes_py \ --mixin release -# source entrypoint setup -ENV OVERLAY_WS $OVERLAY_WS +# multi-stage for running +FROM $FROM_IMAGE-ros-core AS runner + +# install exec dependencies +COPY --from=cacher /tmp/exec_debs.txt /tmp/exec_debs.txt +RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ + --mount=type=cache,from=cacher,target=/var/lib/apt/lists,source=/var/lib/apt/lists \ + < /tmp/exec_debs.txt xargs apt-get install -y --no-install-recommends + +# setup workspace install +ARG OVERLAY_WS +ENV OVERLAY_WS=$OVERLAY_WS +COPY --from=builder $OVERLAY_WS/install $OVERLAY_WS/install RUN sed --in-place --expression \ '$isource "$OVERLAY_WS/install/setup.bash"' \ /ros_entrypoint.sh From b0cc3c765c1a1c9b9887192c8f556cb7155aaf54 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 17:36:56 -0500 Subject: [PATCH 11/31] Simplify scripting for dependencies --- ros/content.md | 35 +++++++++++++++++------------------ 1 file changed, 17 insertions(+), 18 deletions(-) diff --git a/ros/content.md b/ros/content.md index 92f5e6eb2367..f4553cc806bc 100644 --- a/ros/content.md +++ b/ros/content.md @@ -65,24 +65,23 @@ EOF RUN rm /etc/apt/apt.conf.d/docker-clean && apt-get update # derive build/exec dependencies -SHELL ["/bin/bash", "-o", "pipefail", "-c"] -RUN dep_types=(\ - "exec:--dependency-types=exec" \ - "build:") && \ - for dep_type in "${dep_types[@]}"; do \ - IFS=":"; set -- $dep_type; \ - rosdep install -y \ - --from-paths \ - ros2/demos/demo_nodes_cpp \ - ros2/demos/demo_nodes_py \ - --ignore-src \ - --reinstall \ - --simulate \ - ${2} \ - | grep 'apt-get install' \ - | awk -F' ' '{print $4}' | sed "s/'//g" \ - | sort > /tmp/${1}_debs.txt; \ - done +RUN bash -e <<'EOF' +declare -A dep_types=( + [exec]="--dependency-types=exec" + [build]="") +for type in "${!dep_types[@]}"; do + rosdep install -y \ + --from-paths \ + ros2/demos/demo_nodes_cpp \ + ros2/demos/demo_nodes_py \ + --ignore-src \ + --reinstall \ + --simulate \ + ${dep_types[$type]} \ + | grep 'apt-get install' | awk -F' ' '{print $4}' \ + | sed "s/'//g" | sort > /tmp/${type}_debs.txt +done +EOF # multi-stage for building FROM $FROM_IMAGE AS builder From ccdfef41c4389049bd7af4e687b92c958caa5282 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 17:38:26 -0500 Subject: [PATCH 12/31] Lint use of ARGs for readability of each stage --- ros/content.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/ros/content.md b/ros/content.md index f4553cc806bc..f8ae1f2c26de 100644 --- a/ros/content.md +++ b/ros/content.md @@ -49,9 +49,9 @@ ARG OVERLAY_WS=/opt/ros/overlay_ws # multi-stage for caching FROM $FROM_IMAGE AS cacher +ARG OVERLAY_WS # clone overlay source -ARG OVERLAY_WS WORKDIR $OVERLAY_WS/src RUN cat < Date: Wed, 28 May 2025 17:40:30 -0500 Subject: [PATCH 13/31] Simplify example cache mount for apt and update rosdep along with apt list to ensure package index is fresh --- ros/content.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/ros/content.md b/ros/content.md index f8ae1f2c26de..753ab2ab0250 100644 --- a/ros/content.md +++ b/ros/content.md @@ -61,8 +61,10 @@ repositories: version: ${ROS_DISTRO} EOF -# update apt cache -RUN rm /etc/apt/apt.conf.d/docker-clean && apt-get update +# update package cache +RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ + --mount=type=cache,sharing=locked,target=/var/lib/apt/lists \ + apt-get update && rosdep update --rosdistro $ROS_DISTRO # derive build/exec dependencies RUN bash -e <<'EOF' @@ -90,7 +92,7 @@ ARG OVERLAY_WS # install build dependencies COPY --from=cacher /tmp/build_debs.txt /tmp/build_debs.txt RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ - --mount=type=cache,from=cacher,target=/var/lib/apt/lists,source=/var/lib/apt/lists \ + --mount=type=cache,sharing=locked,target=/var/lib/apt/lists \ < /tmp/build_debs.txt xargs apt-get install -y --no-install-recommends # build overlay source @@ -110,7 +112,7 @@ ARG OVERLAY_WS # install exec dependencies COPY --from=cacher /tmp/exec_debs.txt /tmp/exec_debs.txt RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ - --mount=type=cache,from=cacher,target=/var/lib/apt/lists,source=/var/lib/apt/lists \ + --mount=type=cache,sharing=locked,target=/var/lib/apt/lists \ < /tmp/exec_debs.txt xargs apt-get install -y --no-install-recommends # setup overlay install From f2fdf2586628a0c9a61d59f57daebf7c0f971829 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 18:56:30 -0500 Subject: [PATCH 14/31] Update categories for ROS to include additional relevant entries --- ros/metadata.json | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ros/metadata.json b/ros/metadata.json index df07586b5b35..de234f00b76c 100644 --- a/ros/metadata.json +++ b/ros/metadata.json @@ -1,6 +1,8 @@ { "hub": { "categories": [ + "languages-and-frameworks", + "machine-learning-and-ai", "operating-systems" ] } From d08a6c764f35c162c956d638dbd0d286434e2240 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 19:15:28 -0500 Subject: [PATCH 15/31] Shorten derivation script --- ros/content.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/ros/content.md b/ros/content.md index 753ab2ab0250..dd00f585ed26 100644 --- a/ros/content.md +++ b/ros/content.md @@ -80,8 +80,9 @@ for type in "${!dep_types[@]}"; do --reinstall \ --simulate \ ${dep_types[$type]} \ - | grep 'apt-get install' | awk -F' ' '{print $4}' \ - | sed "s/'//g" | sort > /tmp/${type}_debs.txt + | grep 'apt-get install' \ + | awk '{gsub(/'\''/,"",$4); print $4}' \ + | sort -u > /tmp/${type}_debs.txt done EOF From 3eaef3cc10fd2ffa97dc24369a9881e865e52023 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 19:56:54 -0500 Subject: [PATCH 16/31] Use ros-core tag for minimal installer --- ros/content.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ros/content.md b/ros/content.md index dd00f585ed26..ea0618d411d6 100644 --- a/ros/content.md +++ b/ros/content.md @@ -13,7 +13,7 @@ The Robot Operating System (ROS) is a set of software libraries and tools that h To create your own ROS docker images and install custom packages, here's a simple example of installing the C++, Python client library demos using the official released Debian packages via apt-get. ```dockerfile -FROM %%IMAGE%%:rolling +FROM %%IMAGE%%:rolling-ros-core as installer # install ros package RUN apt-get update && apt-get install -y \ From e0df89b798adeeebb9f646b62bb4bac1de7dcd66 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 19:59:38 -0500 Subject: [PATCH 17/31] Stage example image sizes for demos --- ros/content.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/ros/content.md b/ros/content.md index ea0618d411d6..37fbae0e85d9 100644 --- a/ros/content.md +++ b/ros/content.md @@ -127,6 +127,17 @@ RUN sed --in-place --expression \ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] ``` +```console +$ docker image ls ros --format "table {{.Tag}}\t{{.Size}}" +TAG SIZE +rolling-ros-core 489MB +installer 504MB +runner 510MB +rolling 876MB +cacher 885MB +builder 941MB +``` + The example above starts by using [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone source repos of interest into the cacher stage. One could similarly `COPY` code from the local build context into the source directory as well. Package manifest files are then cached in a temporary directory where the following builder stage may copy from to install necessary dependencies with [`rosdep`](https://github.com/ros-infrastructure/rosdep). This is done prior to copying the rest of the source files to preserve the multi-stage build cache, given unaltered manifests do not alter declared dependencies, saving time and bandwidth. The overlay is then built using [`colcon`](https://colcon.readthedocs.io/en/released/), the entrypoint updated to source the workspace, and the default command set to launch the demo. Note: `--from-paths` and `--packages-select` are set here as so to only install the dependencies and build for the demo C++ and Python packages, among many in the demo git repo that was cloned. To install the dependencies and build all the packages in the source workspace, merely change the scope by setting `--from-paths src/` and dropping the `--packages-select` arguments. From 8e2851354cfc4def043fc24bcb009bc7289f8f88 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 20:15:28 -0500 Subject: [PATCH 18/31] Simplify naming --- ros/content.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ros/content.md b/ros/content.md index 37fbae0e85d9..07192c3196bf 100644 --- a/ros/content.md +++ b/ros/content.md @@ -68,10 +68,10 @@ RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ # derive build/exec dependencies RUN bash -e <<'EOF' -declare -A dep_types=( +declare -A types=( [exec]="--dependency-types=exec" [build]="") -for type in "${!dep_types[@]}"; do +for type in "${!types[@]}"; do rosdep install -y \ --from-paths \ ros2/demos/demo_nodes_cpp \ @@ -79,7 +79,7 @@ for type in "${!dep_types[@]}"; do --ignore-src \ --reinstall \ --simulate \ - ${dep_types[$type]} \ + ${types[$type]} \ | grep 'apt-get install' \ | awk '{gsub(/'\''/,"",$4); print $4}' \ | sort -u > /tmp/${type}_debs.txt From 2a69ff60a9211781561a5d63f4141aa4348dc438 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Wed, 28 May 2025 20:16:36 -0500 Subject: [PATCH 19/31] Omit cacher from size table to focus on primary targets --- ros/content.md | 1 - 1 file changed, 1 deletion(-) diff --git a/ros/content.md b/ros/content.md index 07192c3196bf..7734688f3a68 100644 --- a/ros/content.md +++ b/ros/content.md @@ -134,7 +134,6 @@ rolling-ros-core 489MB installer 504MB runner 510MB rolling 876MB -cacher 885MB builder 941MB ``` From ada7866e5ec8071b782de6cc6d31fd91eec8e938 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Thu, 29 May 2025 14:07:52 -0500 Subject: [PATCH 20/31] Modify and mount custom apt config accounting for included docker-clean default --- ros/content.md | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/ros/content.md b/ros/content.md index 7734688f3a68..70f327b94852 100644 --- a/ros/content.md +++ b/ros/content.md @@ -62,9 +62,12 @@ repositories: EOF # update package cache -RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ - --mount=type=cache,sharing=locked,target=/var/lib/apt/lists \ - apt-get update && rosdep update --rosdistro $ROS_DISTRO +RUN rm /etc/apt/apt.conf.d/docker-clean && apt-get update && \ + rosdep update --rosdistro $ROS_DISTRO && \ + cat < /etc/apt/apt.conf.d/docker-no-recommends +APT::Install-Recommends "false"; +APT::Install-Suggests "false"; +EOF # derive build/exec dependencies RUN bash -e <<'EOF' @@ -92,9 +95,10 @@ ARG OVERLAY_WS # install build dependencies COPY --from=cacher /tmp/build_debs.txt /tmp/build_debs.txt -RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ - --mount=type=cache,sharing=locked,target=/var/lib/apt/lists \ - < /tmp/build_debs.txt xargs apt-get install -y --no-install-recommends +RUN --mount=type=cache,target=/etc/apt/apt.conf.d,from=cacher,source=/etc/apt/apt.conf.d \ + --mount=type=cache,target=/var/lib/apt/lists,from=cacher,source=/var/lib/apt/lists \ + --mount=type=cache,target=/var/cache/apt,sharing=locked \ + < /tmp/build_debs.txt xargs apt-get install -y # build overlay source WORKDIR $OVERLAY_WS @@ -112,9 +116,10 @@ ARG OVERLAY_WS # install exec dependencies COPY --from=cacher /tmp/exec_debs.txt /tmp/exec_debs.txt -RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ - --mount=type=cache,sharing=locked,target=/var/lib/apt/lists \ - < /tmp/exec_debs.txt xargs apt-get install -y --no-install-recommends +RUN --mount=type=cache,target=/etc/apt/apt.conf.d,from=cacher,source=/etc/apt/apt.conf.d \ + --mount=type=cache,target=/var/lib/apt/lists,from=cacher,source=/var/lib/apt/lists \ + --mount=type=cache,target=/var/cache/apt,sharing=locked \ + < /tmp/exec_debs.txt xargs apt-get install -y # setup overlay install ENV OVERLAY_WS=$OVERLAY_WS From 385098dff997b5c78fd3b1b19bdfcefa27cf8721 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Thu, 29 May 2025 14:11:24 -0500 Subject: [PATCH 21/31] Simplify RUN directive for apt config --- ros/content.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/ros/content.md b/ros/content.md index 70f327b94852..f401c53187cf 100644 --- a/ros/content.md +++ b/ros/content.md @@ -62,12 +62,11 @@ repositories: EOF # update package cache -RUN rm /etc/apt/apt.conf.d/docker-clean && apt-get update && \ - rosdep update --rosdistro $ROS_DISTRO && \ - cat < /etc/apt/apt.conf.d/docker-no-recommends +RUN cat < /etc/apt/apt.conf.d/docker-clean APT::Install-Recommends "false"; APT::Install-Suggests "false"; -EOF +EOF && apt-get update && \ + rosdep update --rosdistro $ROS_DISTRO # derive build/exec dependencies RUN bash -e <<'EOF' From d16f462fbc664ec5858d078cdaa7b592ee94ff30 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Thu, 29 May 2025 20:43:25 -0500 Subject: [PATCH 22/31] Stage WIP edits --- ros/content.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/ros/content.md b/ros/content.md index f401c53187cf..e6a7ad7db949 100644 --- a/ros/content.md +++ b/ros/content.md @@ -131,6 +131,21 @@ RUN sed --in-place --expression \ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] ``` +The example above uses consists of three sequential stages. The `cacher` stage first uses [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone a demo repo into the workspace source directory, updates the apt lists and ROS index, and derives build and runtime dependencies sets using [`rosdep`](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Rosdep.html). The `builder` stage apt installs the derived build dependencies, sources the ROS install underlay, and compiles the source via release mode using [`colcon`](https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Colcon-Tutorial.html). Finally, `runner` stage apt installs only runtime dependencies, copies the compiled workspace artifacts, and sets up the environment to launch the demo. Note the example consists of several subtle optimizations: + +- colcon + - build cache if deps are unchanged + - build and install select packages +- Caching apt and rosdep + - IO download + - synchronizing apt lists +- multi-state parallelization +- Final Image Size + +example uses the [`--mount`](https://docs.docker.com/engine/reference/builder/#run---mount) option to cache apt lists + +For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. + ```console $ docker image ls ros --format "table {{.Tag}}\t{{.Size}}" TAG SIZE @@ -141,12 +156,6 @@ rolling 876MB builder 941MB ``` -The example above starts by using [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone source repos of interest into the cacher stage. One could similarly `COPY` code from the local build context into the source directory as well. Package manifest files are then cached in a temporary directory where the following builder stage may copy from to install necessary dependencies with [`rosdep`](https://github.com/ros-infrastructure/rosdep). This is done prior to copying the rest of the source files to preserve the multi-stage build cache, given unaltered manifests do not alter declared dependencies, saving time and bandwidth. The overlay is then built using [`colcon`](https://colcon.readthedocs.io/en/released/), the entrypoint updated to source the workspace, and the default command set to launch the demo. - -Note: `--from-paths` and `--packages-select` are set here as so to only install the dependencies and build for the demo C++ and Python packages, among many in the demo git repo that was cloned. To install the dependencies and build all the packages in the source workspace, merely change the scope by setting `--from-paths src/` and dropping the `--packages-select` arguments. - -For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. - ## Deployment use cases This dockerized image of ROS is intended to provide a simplified and consistent platform to build and deploy distributed robotic applications. Built from the [official Ubuntu image](https://hub.docker.com/_/ubuntu/) and ROS's official Debian packages, it includes recent supported releases for quick access and download. This provides roboticists in research and industry with an easy way to develop, reuse and ship software for autonomous actions and task planning, control dynamics, localization and mapping, swarm behavior, as well as general system integration. From 63ff4c78446ff2986de349d0c400efb2d3126c2e Mon Sep 17 00:00:00 2001 From: ruffsl Date: Thu, 29 May 2025 20:49:05 -0500 Subject: [PATCH 23/31] Update package cache before cloning source to avoid running rosdep update for each source change as docker build --no-cache can still be used to force update --- ros/content.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/ros/content.md b/ros/content.md index e6a7ad7db949..6119586a83a9 100644 --- a/ros/content.md +++ b/ros/content.md @@ -51,9 +51,16 @@ ARG OVERLAY_WS=/opt/ros/overlay_ws FROM $FROM_IMAGE AS cacher ARG OVERLAY_WS +# update package cache +RUN cat < /etc/apt/apt.conf.d/docker-clean +APT::Install-Recommends "false"; +APT::Install-Suggests "false"; +EOF && apt-get update && \ + rosdep update --rosdistro $ROS_DISTRO + # clone overlay source WORKDIR $OVERLAY_WS/src -RUN cat < /etc/apt/apt.conf.d/docker-clean -APT::Install-Recommends "false"; -APT::Install-Suggests "false"; -EOF && apt-get update && \ - rosdep update --rosdistro $ROS_DISTRO - # derive build/exec dependencies RUN bash -e <<'EOF' declare -A types=( @@ -131,7 +131,7 @@ RUN sed --in-place --expression \ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] ``` -The example above uses consists of three sequential stages. The `cacher` stage first uses [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone a demo repo into the workspace source directory, updates the apt lists and ROS index, and derives build and runtime dependencies sets using [`rosdep`](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Rosdep.html). The `builder` stage apt installs the derived build dependencies, sources the ROS install underlay, and compiles the source via release mode using [`colcon`](https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Colcon-Tutorial.html). Finally, `runner` stage apt installs only runtime dependencies, copies the compiled workspace artifacts, and sets up the environment to launch the demo. Note the example consists of several subtle optimizations: +The example above uses consists of three sequential stages. The `cacher` stage first updates the apt lists and ROS index, uses [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone a demo repo into the workspace source directory, and derives build and runtime dependencies sets using [`rosdep`](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Rosdep.html). The `builder` stage apt installs the derived build dependencies, sources the ROS install underlay, and compiles the source via release mode using [`colcon`](https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Colcon-Tutorial.html). Finally, `runner` stage apt installs only runtime dependencies, copies the compiled workspace artifacts, and sets up the environment to launch the demo. Note the example consists of several subtle optimizations: - colcon - build cache if deps are unchanged From 12b2835ccbfabaf70f0cb57992d705fb0ea57ee2 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Thu, 29 May 2025 21:00:44 -0500 Subject: [PATCH 24/31] Fix use of EOF in Dockerfile --- ros/content.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/ros/content.md b/ros/content.md index 6119586a83a9..b5a0554d92e1 100644 --- a/ros/content.md +++ b/ros/content.md @@ -55,8 +55,9 @@ ARG OVERLAY_WS RUN cat < /etc/apt/apt.conf.d/docker-clean APT::Install-Recommends "false"; APT::Install-Suggests "false"; -EOF && apt-get update && \ - rosdep update --rosdistro $ROS_DISTRO +EOF +RUN apt-get update && \ + rosdep update --rosdistro $ROS_DISTRO # clone overlay source WORKDIR $OVERLAY_WS/src From dba8a8285ff2043bcffe4363db37fe4ddbe34535 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Thu, 29 May 2025 21:05:10 -0500 Subject: [PATCH 25/31] Simply RUN directives --- ros/content.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/ros/content.md b/ros/content.md index b5a0554d92e1..37d28822710e 100644 --- a/ros/content.md +++ b/ros/content.md @@ -52,12 +52,11 @@ FROM $FROM_IMAGE AS cacher ARG OVERLAY_WS # update package cache -RUN cat < /etc/apt/apt.conf.d/docker-clean +RUN rosdep update --rosdistro $ROS_DISTRO && \ + cat < /etc/apt/apt.conf.d/docker-clean && apt-get update APT::Install-Recommends "false"; APT::Install-Suggests "false"; EOF -RUN apt-get update && \ - rosdep update --rosdistro $ROS_DISTRO # clone overlay source WORKDIR $OVERLAY_WS/src From eeee2a75fcf62dceed982c1d4ad58520be0158aa Mon Sep 17 00:00:00 2001 From: ruffsl Date: Fri, 30 May 2025 14:03:45 -0500 Subject: [PATCH 26/31] Expand bullet points about optimizations --- ros/content.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/ros/content.md b/ros/content.md index 37d28822710e..6294522c3dfb 100644 --- a/ros/content.md +++ b/ros/content.md @@ -133,16 +133,18 @@ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] The example above uses consists of three sequential stages. The `cacher` stage first updates the apt lists and ROS index, uses [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone a demo repo into the workspace source directory, and derives build and runtime dependencies sets using [`rosdep`](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Rosdep.html). The `builder` stage apt installs the derived build dependencies, sources the ROS install underlay, and compiles the source via release mode using [`colcon`](https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Colcon-Tutorial.html). Finally, `runner` stage apt installs only runtime dependencies, copies the compiled workspace artifacts, and sets up the environment to launch the demo. Note the example consists of several subtle optimizations: -- colcon - - build cache if deps are unchanged - - build and install select packages -- Caching apt and rosdep - - IO download - - synchronizing apt lists -- multi-state parallelization -- Final Image Size - -example uses the [`--mount`](https://docs.docker.com/engine/reference/builder/#run---mount) option to cache apt lists +- Multi Stage Build + - Dependency derivation, compilation, and runtime setup are partitioned + - Maximums cache retention despite package source or build/runtime changes + - Greater concurrency, e.g: colcon build while runtime apt instals +- Persistent Cache Propagation + - Use of [`--mount`](https://docs.docker.com/engine/reference/builder/#run---mount) to cache temp data without bloating layers + - Maintain temporally consistent apt lists between parallel stages + - Avoid needless network IO between stages or across docker rebuilds +- Minimal Image Size + - Final stage builds from `ros-core` for smallest runtime image + - Builds and installs for just selected packages amongst workspace + - Only workspace install artifacts are copied into final layers For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. From 130999221366f3f8f9cee029eb2a8ff9da1b4227 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Fri, 30 May 2025 14:28:04 -0500 Subject: [PATCH 27/31] Expand upon runner vs installer comparison --- ros/content.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/ros/content.md b/ros/content.md index 6294522c3dfb..00d77dba53a8 100644 --- a/ros/content.md +++ b/ros/content.md @@ -28,8 +28,8 @@ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] Note: all ROS images include a default entrypoint that sources the ROS environment setup before executing the configured command, in this case the demo packages launch file. You can then build and run the Docker image like so: ```console -$ docker build -t my/ros:app . -$ docker run -it --rm my/ros:app +$ docker build -t my/ros:installer . +$ docker run -it --rm my/ros:installer [INFO] [launch]: process[talker-1]: started with pid [813] [INFO] [launch]: process[listener-2]: started with pid [814] [INFO] [talker]: Publishing: 'Hello World: 1' @@ -146,18 +146,22 @@ The example above uses consists of three sequential stages. The `cacher` stage f - Builds and installs for just selected packages amongst workspace - Only workspace install artifacts are copied into final layers -For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. +For comparison, the resulting `runner` image is similar in size to the earlier `installer` example. This allows you to develop and distribute custom ROS packages without significantly increasing image size compared to pre-built Debian installations: ```console -$ docker image ls ros --format "table {{.Tag}}\t{{.Size}}" +$ docker image ls my/ros --format "table {{.Tag}}\t{{.Size}}" TAG SIZE -rolling-ros-core 489MB installer 504MB runner 510MB -rolling 876MB builder 941MB +$ docker image ls ros --format "table {{.Tag}}\t{{.Size}}" +TAG SIZE +rolling-ros-core 489MB +rolling 876MB ``` +For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. + ## Deployment use cases This dockerized image of ROS is intended to provide a simplified and consistent platform to build and deploy distributed robotic applications. Built from the [official Ubuntu image](https://hub.docker.com/_/ubuntu/) and ROS's official Debian packages, it includes recent supported releases for quick access and download. This provides roboticists in research and industry with an easy way to develop, reuse and ship software for autonomous actions and task planning, control dynamics, localization and mapping, swarm behavior, as well as general system integration. From 879872f8f723c4201c13d8b854d2cd4f9ac49423 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Fri, 30 May 2025 14:38:08 -0500 Subject: [PATCH 28/31] Linting --- ros/content.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/ros/content.md b/ros/content.md index 00d77dba53a8..f4501f8fdff1 100644 --- a/ros/content.md +++ b/ros/content.md @@ -133,18 +133,18 @@ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] The example above uses consists of three sequential stages. The `cacher` stage first updates the apt lists and ROS index, uses [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone a demo repo into the workspace source directory, and derives build and runtime dependencies sets using [`rosdep`](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Rosdep.html). The `builder` stage apt installs the derived build dependencies, sources the ROS install underlay, and compiles the source via release mode using [`colcon`](https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Colcon-Tutorial.html). Finally, `runner` stage apt installs only runtime dependencies, copies the compiled workspace artifacts, and sets up the environment to launch the demo. Note the example consists of several subtle optimizations: -- Multi Stage Build - - Dependency derivation, compilation, and runtime setup are partitioned - - Maximums cache retention despite package source or build/runtime changes - - Greater concurrency, e.g: colcon build while runtime apt instals -- Persistent Cache Propagation - - Use of [`--mount`](https://docs.docker.com/engine/reference/builder/#run---mount) to cache temp data without bloating layers - - Maintain temporally consistent apt lists between parallel stages - - Avoid needless network IO between stages or across docker rebuilds -- Minimal Image Size - - Final stage builds from `ros-core` for smallest runtime image - - Builds and installs for just selected packages amongst workspace - - Only workspace install artifacts are copied into final layers +- Multi Stage Build + - Dependency derivation, compilation, and runtime setup are partitioned + - Maximums cache retention despite package source or build/runtime changes + - Greater concurrency, e.g: colcon build while runtime apt instals +- Persistent Cache Propagation + - Use of [`--mount`](https://docs.docker.com/engine/reference/builder/#run---mount) to cache temp data without bloating layers + - Maintain temporally consistent apt lists between parallel stages + - Avoid needless network IO between stages or across docker rebuilds +- Minimal Image Size + - Final stage builds from `ros-core` for smallest runtime image + - Builds and installs for just selected packages amongst workspace + - Only workspace install artifacts are copied into final layers For comparison, the resulting `runner` image is similar in size to the earlier `installer` example. This allows you to develop and distribute custom ROS packages without significantly increasing image size compared to pre-built Debian installations: From 505cd29c48d300d4d507ccf6b9326d616870d821 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Fri, 30 May 2025 14:56:13 -0500 Subject: [PATCH 29/31] Fix grammar and spelling --- ros/content.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/ros/content.md b/ros/content.md index f4501f8fdff1..88934621733f 100644 --- a/ros/content.md +++ b/ros/content.md @@ -131,20 +131,20 @@ RUN sed --in-place --expression \ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] ``` -The example above uses consists of three sequential stages. The `cacher` stage first updates the apt lists and ROS index, uses [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone a demo repo into the workspace source directory, and derives build and runtime dependencies sets using [`rosdep`](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Rosdep.html). The `builder` stage apt installs the derived build dependencies, sources the ROS install underlay, and compiles the source via release mode using [`colcon`](https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Colcon-Tutorial.html). Finally, `runner` stage apt installs only runtime dependencies, copies the compiled workspace artifacts, and sets up the environment to launch the demo. Note the example consists of several subtle optimizations: +The example above consists of three sequential stages. The `cacher` stage first updates the apt lists and ROS index, uses [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone a demo repo into the workspace source directory, and derives build and runtime dependency sets using [`rosdep`](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Rosdep.html). The `builder` stage installs the derived build dependencies, sources the ROS install underlay, and compiles the source in release mode using [`colcon`](https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Colcon-Tutorial.html). Finally, the `runner` stage installs only runtime dependencies, copies the compiled workspace artifacts, and sets up the environment to launch the demo. Note the example consists of several subtle optimizations: -- Multi Stage Build - - Dependency derivation, compilation, and runtime setup are partitioned - - Maximums cache retention despite package source or build/runtime changes - - Greater concurrency, e.g: colcon build while runtime apt instals +- Multi-Stage Build + - Dependency derivation, compilation, and runtime setup are partitioned + - Maximizes cache retention despite package source or build/runtime changes + - Greater concurrency, e.g., colcon build while runtime apt installs - Persistent Cache Propagation - - Use of [`--mount`](https://docs.docker.com/engine/reference/builder/#run---mount) to cache temp data without bloating layers - - Maintain temporally consistent apt lists between parallel stages - - Avoid needless network IO between stages or across docker rebuilds + - Use of [`--mount`](https://docs.docker.com/engine/reference/builder/#run---mount) to cache temp data without bloating layers + - Maintain temporally consistent apt lists between parallel stages + - Avoid needless network I/O between stages or across Docker rebuilds - Minimal Image Size - - Final stage builds from `ros-core` for smallest runtime image - - Builds and installs for just selected packages amongst workspace - - Only workspace install artifacts are copied into final layers + - Final stage builds from `ros-core` for smallest runtime image + - Builds and installs only a select few packages in the workspace + - Only workspace install artifacts are copied into final layers For comparison, the resulting `runner` image is similar in size to the earlier `installer` example. This allows you to develop and distribute custom ROS packages without significantly increasing image size compared to pre-built Debian installations: @@ -160,7 +160,7 @@ rolling-ros-core 489MB rolling 876MB ``` -For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. +For more advanced examples, such as daisy-chaining multiple overlay workspaces to improve caching of Docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using BuildKit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. ## Deployment use cases From ebf1820ba4d958d79042f9b0dcf6a2198f04773a Mon Sep 17 00:00:00 2001 From: ruffsl Date: Fri, 30 May 2025 14:58:19 -0500 Subject: [PATCH 30/31] Linting --- ros/content.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/ros/content.md b/ros/content.md index 88934621733f..8ca9236a763a 100644 --- a/ros/content.md +++ b/ros/content.md @@ -134,17 +134,17 @@ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] The example above consists of three sequential stages. The `cacher` stage first updates the apt lists and ROS index, uses [`vcstool`](https://github.com/dirk-thomas/vcstool) to clone a demo repo into the workspace source directory, and derives build and runtime dependency sets using [`rosdep`](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Rosdep.html). The `builder` stage installs the derived build dependencies, sources the ROS install underlay, and compiles the source in release mode using [`colcon`](https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Colcon-Tutorial.html). Finally, the `runner` stage installs only runtime dependencies, copies the compiled workspace artifacts, and sets up the environment to launch the demo. Note the example consists of several subtle optimizations: - Multi-Stage Build - - Dependency derivation, compilation, and runtime setup are partitioned - - Maximizes cache retention despite package source or build/runtime changes - - Greater concurrency, e.g., colcon build while runtime apt installs + - Dependency derivation, compilation, and runtime setup are partitioned + - Maximizes cache retention despite package source or build/runtime changes + - Greater concurrency, e.g., colcon build while runtime apt installs - Persistent Cache Propagation - - Use of [`--mount`](https://docs.docker.com/engine/reference/builder/#run---mount) to cache temp data without bloating layers - - Maintain temporally consistent apt lists between parallel stages - - Avoid needless network I/O between stages or across Docker rebuilds + - Use of [`--mount`](https://docs.docker.com/engine/reference/builder/#run---mount) to cache temp data without bloating layers + - Maintain temporally consistent apt lists between parallel stages + - Avoid needless network I/O between stages or across Docker rebuilds - Minimal Image Size - - Final stage builds from `ros-core` for smallest runtime image - - Builds and installs only a select few packages in the workspace - - Only workspace install artifacts are copied into final layers + - Final stage builds from `ros-core` for smallest runtime image + - Builds and installs only a select few packages in the workspace + - Only workspace install artifacts are copied into final layers For comparison, the resulting `runner` image is similar in size to the earlier `installer` example. This allows you to develop and distribute custom ROS packages without significantly increasing image size compared to pre-built Debian installations: @@ -160,7 +160,7 @@ rolling-ros-core 489MB rolling 876MB ``` -For more advanced examples, such as daisy-chaining multiple overlay workspaces to improve caching of Docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using BuildKit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. +For more advance examples such as daisy chaining multiple overlay workspaces to improve caching of docker image build layers, using tools such as ccache to accelerate compilation with colcon, or using buildkit to save build time and bandwidth even when dependencies change, the project `Dockerfile`s in the [Navigation2](https://github.com/ros-planning/navigation2) repo are excellent resources. ## Deployment use cases From 5f3ac57b8b65471898164a68b80ab233bd1d1ba7 Mon Sep 17 00:00:00 2001 From: ruffsl Date: Mon, 2 Jun 2025 17:20:32 -0500 Subject: [PATCH 31/31] Rename stage in apt get example while keeping with uniquely spelled noun that is self descriptive yet unmistakable --- ros/content.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ros/content.md b/ros/content.md index 8ca9236a763a..297d297e6ec9 100644 --- a/ros/content.md +++ b/ros/content.md @@ -13,7 +13,7 @@ The Robot Operating System (ROS) is a set of software libraries and tools that h To create your own ROS docker images and install custom packages, here's a simple example of installing the C++, Python client library demos using the official released Debian packages via apt-get. ```dockerfile -FROM %%IMAGE%%:rolling-ros-core as installer +FROM %%IMAGE%%:rolling-ros-core as aptgetter # install ros package RUN apt-get update && apt-get install -y \ @@ -28,8 +28,8 @@ CMD ["ros2", "launch", "demo_nodes_cpp", "talker_listener_launch.py"] Note: all ROS images include a default entrypoint that sources the ROS environment setup before executing the configured command, in this case the demo packages launch file. You can then build and run the Docker image like so: ```console -$ docker build -t my/ros:installer . -$ docker run -it --rm my/ros:installer +$ docker build -t my/ros:aptgetter . +$ docker run -it --rm my/ros:aptgetter [INFO] [launch]: process[talker-1]: started with pid [813] [INFO] [launch]: process[listener-2]: started with pid [814] [INFO] [talker]: Publishing: 'Hello World: 1' @@ -146,12 +146,12 @@ The example above consists of three sequential stages. The `cacher` stage first - Builds and installs only a select few packages in the workspace - Only workspace install artifacts are copied into final layers -For comparison, the resulting `runner` image is similar in size to the earlier `installer` example. This allows you to develop and distribute custom ROS packages without significantly increasing image size compared to pre-built Debian installations: +For comparison, the resulting `runner` image is similar in size to the earlier `aptgetter` example. This allows you to develop and distribute custom ROS packages without significantly increasing image size compared to pre-built Debian installations: ```console $ docker image ls my/ros --format "table {{.Tag}}\t{{.Size}}" TAG SIZE -installer 504MB +aptgetter 504MB runner 510MB builder 941MB $ docker image ls ros --format "table {{.Tag}}\t{{.Size}}"