feat: pull-through-cache-registry

.github/workflows/test.yaml

@@ -26,6 +26,7 @@ jobs:
           - local-pkgsite
           - nerdctl
           - pin-github-action
+          - pull-through-cache-registry
           - wal-wahl
         baseImage:
           - mcr.microsoft.com/devcontainers/base:ubuntu-24.04

README.md

@@ -12,21 +12,22 @@ Alpine](https://github.com/devcontainers/features/blob/091886b3568dad70f835cc428
 Same unfortunate situation Microsoft's node Dev Container feature that also
 doesn't support Alpine.
 
-| Feature           | Alpine | Alma | Debian | Fedora | Ubuntu |
-| ----------------- | ------ | ---- | ------ | ------ | ------ |
-| bpftool           |  ✅    |  ✅  |  ✅    |  ✅    | ✅     |
-| cni-plugins       |        |  ✅  |  ✅    |  ✅    | ✅     |
-| docsify           |        |  ?   |  ✅    |  ?     | ✅     |
-| go-ebpf           |        |  ✅  |  ✅    |  ✅    | ✅     |
-| go-mod-upgrade    |        |  ✅  |  ✅    |  ✅    | ✅     |
-| gocover           |        |  ✅  |  ✅    |  ✅    | ✅     |
-| goreportcard      |        |  ✅  |  ✅    |  ✅    | ✅     |
-| grafanactl        |  ✅    |  ✅  |  ✅    |  ✅    | ✅     |
-| lazygit           |  ✅    |  ✅  |  ✅    |  ✅    | ✅     |
-| local-pkgsite     |        |  ✅  |  ✅    |  ✅    | ✅     |
-| nerdctl           |        |  ✅  |  ✅    |  ✅    | ✅     |
-| pin-github-action |        |  ✅  |  ✅    |  ✅    | ✅     |
-| wal-wahl          |        |  ✅  |  ✅    |  ✅    | ✅     |
+| Feature                     | Alpine | Alma | Debian | Fedora | Ubuntu |
+| --------------------------- | ------ | ---- | ------ | ------ | ------ |
+| bpftool                     |  ✅    |  ✅  |  ✅    |  ✅    | ✅     |
+| cni-plugins                 |        |  ✅  |  ✅    |  ✅    | ✅     |
+| docsify                     |        |  ?   |  ✅    |  ?     | ✅     |
+| go-ebpf                     |        |  ✅  |  ✅    |  ✅    | ✅     |
+| go-mod-upgrade              |        |  ✅  |  ✅    |  ✅    | ✅     |
+| gocover                     |        |  ✅  |  ✅    |  ✅    | ✅     |
+| goreportcard                |        |  ✅  |  ✅    |  ✅    | ✅     |
+| grafanactl                  |  ✅    |  ✅  |  ✅    |  ✅    | ✅     |
+| lazygit                     |  ✅    |  ✅  |  ✅    |  ✅    | ✅     |
+| local-pkgsite               |        |  ✅  |  ✅    |  ✅    | ✅     |
+| nerdctl                     |        |  ✅  |  ✅    |  ✅    | ✅     |
+| pin-github-action           |        |  ✅  |  ✅    |  ✅    | ✅     |
+| pull-through-cache-registry |        |      |  ✅    |        | ✅     |
+| wal-wahl                    |        |  ✅  |  ✅    |  ✅    | ✅     |
 
 - [bpftool](src/bpftool/README.md) – installs `bpftool` directly from upstream
   https://github.com/libbpf/bpftool binary releases, especially avoiding the
@@ -70,6 +71,10 @@ doesn't support Alpine.
 - [pin-github-action](src/pin-github-action/README.md) – provides mheaps's
   `pin-github-action` for pinning GitHub actions to specific hashes.
 
+- [pull-through-cache-registry](src/pull-through-cache-registry/README.md) –
+  provides a CNCF Distribution Registry acting as pull-through cache for your
+  Docker-in-Docker.
+
 - [wal-wahl](src/wal-wahl/README.md) – install multiple Docker CE versions
   inside your devcontainer, quickly switching and activating one of them at a
   time.

src/pull-through-cache-registry/NOTES.md

@@ -0,0 +1,18 @@
+## OS Support
+
+As this feature relies on the [Docker-in-Docker
+feature](https://github.com/devcontainers/features/tree/main/src/docker-in-docker)
+we only support the same Debian/Ubuntu platforms.
+
+## Registry Configuration
+
+- pull-through caching is enabled by passing `REGISTRY_PROXY_REMOTEURL` (a.k.a.
+  `proxy:{remoteulr:}`).
+- logging is set to info level by passing `REGISTRY_LOG_LEVEL` (a.k.a.
+  `log:{level:}`).
+
+## Acknowledgement
+
+[Registry as a pull through
+cache](https://distribution.github.io/distribution/recipes/mirror/), CNCF
+Distribution.

src/pull-through-cache-registry/README.md

@@ -0,0 +1,44 @@
+
+# OCI registry pull-through cache to mirror rate-limited upstream registries, such as Docker Hub (pull-through-cache-registry)
+
+provides a devcontainer-local CNCF Distribution Registry configured as a pull-through cache for the local docker-in-docker
+
+## Example Usage
+
+```json
+"features": {
+    "ghcr.io/thediveo/devcontainer-features/pull-through-cache-registry:0": {}
+}
+```
+
+## Options
+
+| Options Id | Description | Type | Default Value |
+|-----|-----|-----|-----|
+| port | port to bind the CNCF Distribution Registry service to | string | 5000 |
+| registry-name | the Docker container name to give the CNCF Distribution Registry | string | registry-cache |
+| wait | maximum wait time in seconds for Docker to become available when starting the CNCF Distribution Registry service | string | 30 |
+
+## OS Support
+
+As this feature relies on the [Docker-in-Docker
+feature](https://github.com/devcontainers/features/tree/main/src/docker-in-docker)
+we only support the same Debian/Ubuntu platforms.
+
+## Registry Configuration
+
+- pull-through caching is enabled by passing `REGISTRY_PROXY_REMOTEURL` (a.k.a.
+  `proxy:{remoteulr:}`).
+- logging is set to info level by passing `REGISTRY_LOG_LEVEL` (a.k.a.
+  `log:{level:}`).
+
+## Acknowledgement
+
+[Registry as a pull through
+cache](https://distribution.github.io/distribution/recipes/mirror/), CNCF
+Distribution.
+
+
+---
+
+_Note: This file was auto-generated from the [devcontainer-feature.json](https://github.com/thediveo/devcontainer-features/blob/main/src/pull-through-cache-registry/devcontainer-feature.json).  Add additional notes to a `NOTES.md`._

src/pull-through-cache-registry/devcontainer-feature.json

@@ -0,0 +1,34 @@
+{
+    "name": "OCI registry pull-through cache to mirror rate-limited upstream registries, such as Docker Hub",
+    "id": "pull-through-cache-registry",
+    "version": "0.0.1",
+    "description": "Deploys a devcontainer-local CNCF Distribution Registry configured as a pull-through cache for the local docker-in-docker",
+    "documentationURL": "https://github.com/thediveo/devcontainer-features/blob/master/src/registry-pull-through-cache/README.md",
+    "options": {
+        "proxy-remote-url": {
+            "type": "string",
+            "default": "https://registry-1.docker.io",
+            "description": "URL of the upstream OCI registry"
+        },
+        "port": {
+            "type": "string",
+            "default": "5000",
+            "description": "port to bind the CNCF Distribution Registry service to",
+            "proposals": ["5000", "9999"]
+        },
+        "registry-name": {
+            "type": "string",
+            "default": "registry-cache",
+            "description": "the Docker container name to give the CNCF Distribution Registry"
+        },
+        "wait": {
+            "type": "string",
+            "default": "30",
+            "description": "maximum wait time in seconds for Docker to become available when starting the CNCF Distribution Registry service"
+        }
+    },
+    "postCreateCommand": "/usr/local/bin/registry-pull-through-cache",
+    "dependsOn": {
+        "ghcr.io/devcontainers/features/docker-in-docker:2": {}
+    }
+}

src/pull-through-cache-registry/install.sh

@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+
+set -e
+
+REGISTRYDEPLOYSCRIPT_PATH="/usr/local/bin/registry-pull-through-cache"
+
+PROXY_REMOTE_URL=${PROXY_REMOTE_URL:-"https://registry-1.docker.io"}
+PORT=${PORT:-5000}
+REGISTRY_NAME=${REGISTRY_NAME:-"registry-cache"}
+WAIT=${WAIT:-30}
+
+echo "installing feature registry-pull-through-cache"
+
+cat <<EOF >"${REGISTRYDEPLOYSCRIPT_PATH}"
+PROXY_REMOTE_URL=${PROXY_REMOTE_URL}
+PORT="${PORT}"
+REGISTRY_NAME="${REGISTRY_NAME}"
+
+timeout=${WAIT}
+echo "waiting up to ${WAIT}s for Docker daemon to become responsive..."
+while ! docker ps >/dev/null 2>&1; do
+    sleep 1
+    timeout=$((timeout-1))
+    if [ "$timeout" -le 0 ]; then
+        echo "Docker did not become responsive, aborting"
+        exit 1
+    fi
+done
+
+if docker ps -a --format '{{.Names}}' | grep -q "^\${REGISTRY_NAME}$"; then
+    echo "(re)starting pull-through cache registry container"
+    docker start "\${REGISTRY_NAME}"
+else
+    echo "running pull-through cache registry container"
+    docker run -d \
+        --restart always \
+        --name "\${REGISTRY_NAME}" \
+        -p \${PORT}:5000 \
+        -e REGISTRY_PROXY_REMOTEURL="\${PROXY_REMOTE_URL}" \
+        -e REGISTRY_LOG_LEVEL=info \
+        -e OTEL_TRACES_EXPORTER=none \
+        registry:3
+fi
+echo "pull-through cache registry started"
+EOF
+
+echo generating /etc/docker/daemon.json configuring registry-mirrors
+cat <<EOF >/etc/docker/daemon.json
+{
+  "registry-mirrors": [ "http://localhost:${PORT}" ]
+}
+EOF
+
+chmod 0755 "${REGISTRYDEPLOYSCRIPT_PATH}"

test/pull-through-cache-registry/9999.sh

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+set -e
+
+source dev-container-features-test-lib
+
+check "registry service is up" bash -c "source ./wait.sh && whalewaiting registry-cache"
+check "registry service responds" bash -c "source ./wait.sh && registrywaiting http://localhost:9999"
+
+reportResults

test/pull-through-cache-registry/quay.io.sh

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+set -e
+
+source dev-container-features-test-lib
+
+check "registry service is up" bash -c "source ./wait.sh && whalewaiting registry-cache"
+check "registry service responds" bash -c "source ./wait.sh && registrywaiting http://localhost:5000"
+check "pulling an image" bash -c "docker pull quay.io/libpod/busybox | tee >(grep 'quay.io/libpod/busybox:latest')"
+
+reportResults

test/pull-through-cache-registry/scenarios.json

@@ -0,0 +1,18 @@
+{
+    "9999": {
+        "image": "mcr.microsoft.com/devcontainers/base:ubuntu-24.04",
+        "features": {
+            "pull-through-cache-registry": {
+                "port": "9999"
+            }
+        }
+    },
+    "quay.io": {
+        "image": "mcr.microsoft.com/devcontainers/base:ubuntu-24.04",
+        "features": {
+            "pull-through-cache-registry": {
+                "proxy-remote-url": "https://quay.io"
+            }
+        }
+    }
+}

test/pull-through-cache-registry/test.sh

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+set -e
+
+source dev-container-features-test-lib
+
+check "registry service is up" bash -c "source ./wait.sh && whalewaiting registry-cache"
+check "registry service responds" bash -c "source ./wait.sh && registrywaiting http://localhost:5000"
+
+reportResults

test/pull-through-cache-registry/wait.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+whalewaiting() {
+    local container="$1"
+    local timeout="${2:-30}"
+
+    while true; do
+        if docker ps --filter "name=^${container}$" --filter "status=running" --format '{{.Names}}' |
+            grep -qx "${container}"; then
+                return 0
+        fi
+        sleep 1
+        timeout=$((timeout-1))
+        if [ "$timeout" -le 0 ]; then
+            exit 1
+        fi
+    done
+}
+
+registrywaiting() {
+    local url="$1"
+    local timeout="${2:-10}"
+
+    while true; do
+        if curl -s -o /dev/null -w "%{http_code}" "${url}/v2/" | grep -q '200'; then
+            return 0
+        fi
+        sleep 1
+        timeout=$((timeout-1))
+        if [ "$timeout" -le 0 ]; then
+            exit 1
+        fi
+    done
+}