GitLab CI demo: Add workaround for IPv6 VMs

[MakisH]

Description

In the demo, we use bwCloud. However, it looks like for new users (like me), bwCloud only offers IPv6 networks, while the network bridge of the Docker daemon only supports IPv4 by default and needs explicit configuration to work with IPv6.

This PR extends the demo instructions with some workarounds.

Main issues and solution elements

• bwCloud only allows IPv6 VMs
• Docker does not enable IPv6 by default
  • Pass --network host to docker run
• Go (in which GitLab Runner is built on) prioritizes IPv4
  • Pass -e GODEBUG="netdns=go+ipv6" to docker run
• registry.gitlab.com only accepts IPv4 and tries to get a helper image from there
  • Override the helper image origin to use the equivalent one from Docker Hub, by setting helper_image = "gitlab/gitlab-runner-helper:x86_64-v17.10.1" in /srv/gitlab-runner/config/config.toml (section [runners.docker])
• GitLab Runner spawns more containers, which also need to use the host network stack
  • Configure network_mode = "host" in /srv/gitlab-runner/config/config.toml (section [runners.docker])

Known issues

• The helper image has a version tag hard-coded. Can we provide a more future-proof tag?

Debugging steps

Current state: (not really interesting anymore)

Current state:

I can register the runner, GitLab sees it, but when I try to run a CI job, I get (example):

Running with gitlab-runner 17.10.1 (ef334dcc)
  on bwcloud-makis t3_nqDp5B, system ID: r_E4K3qIHONJAE
Preparing the "docker" executor 00:03
Using Docker executor with image tumi5/latex ...
Using helper image:  registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-v17.10.1
Pulling docker image registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-v17.10.1 ...
WARNING: Failed to pull image with policy "always": Error response from daemon: Get "https://registry.gitlab.com/v2/": dial tcp 35.227.35.254:443: connect: no route to host (manager.go:254:3s)
ERROR: Job failed: failed to pull image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-v17.10.1" with specified policies [always]: Error response from daemon: Get "https://registry.gitlab.com/v2/": dial tcp 35.227.35.254:443: connect: no route to host (manager.go:254:3s)

It looks like the GitLab Container Registry does not support IPv6. At the same time, the issue is currently not prioritized due to high effort / low demand.

I asked the bwCloud helpdesk, and it is possible to enable IPv4 on demand. I don't see this to be a sustainable solution.

So: bwCloud only supports IPv6 (by default), but registry.gitlab.com only supports IPv4. Maybe a next question would be if we can replace the connections to registry.gitlab.com with something else.

I tried overriding the helper image to the one from Docker Hub, and this went a bit further. I added to the [runners.docker] section of /srv/gitlab-runner/config/config.toml:

helper_image = "gitlab/gitlab-runner-helper:x86_64-v17.10.1"

and now it goes through the previous step, but has an issue resolving the GitLab instance domain:

Preparing environment 00:02
Using helper image:  gitlab/gitlab-runner-helper:x86_64-v17.10.1  (overridden, default would be  registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-v17.10.1 )
Using docker image sha256:400c4003975a58723d3fb433ae7cbd8e5d532897b126c5b16739452f1ecff2f6 for gitlab/gitlab-runner-helper:x86_64-v17.10.1 with digest gitlab/gitlab-runner-helper@sha256:8c8ee8b72a7bd3a4951b255b55b70efdda714b375c0ada6ea3aaee0ef23528a0 ...
Running on runner-t3nqdp5b-project-914-concurrent-0 via gitlab-ci...
Getting source from Git repository 00:06
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/sgswimi/teaching/seminar-simulation/.git/
Created fresh repository.
fatal: unable to access 'https://gitlab-sim.informatik.uni-stuttgart.de/sgswimi/teaching/seminar-simulation.git/': Could not resolve host: gitlab-sim.informatik.uni-stuttgart.de

I tried (a) setting the Google DNS servers (dns = ["2001:4860:4860::8888", "2001:4860:4860::8844"]), (b) setting the host dns (volumes = ["/etc/resolv.conf:/etc/resolv.conf:ro"]), and (c) hard-coding the address in the /srv/gitlab-runner/custom-hosts. Only the last option worked:

# Get the address
dig aaaa gitlab-sim.informatik.uni-stuttgart.de

# mount the respective files
volumes = ["/etc/resolv.conf:/etc/resolv.conf:ro", "/srv/gitlab-runner/custom-hosts:/etc/hosts:ro"]

# create the /srv/gitlab-runner/custom-hosts with content
2001:7c0:2015:216::19 gitlab-sim.informatik.uni-stuttgart.de

The domain can now be resolved, but connection cannot be established:

Getting source from Git repository 02:13
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/sgswimi/teaching/seminar-simulation/.git/
Created fresh repository.
fatal: unable to access 'https://gitlab-sim.informatik.uni-stuttgart.de/sgswimi/teaching/seminar-simulation.git/': Failed to connect to gitlab-sim.informatik.uni-stuttgart.de port 443 after 132766 ms: Could not connect to server

I could overcome this by setting in the config:

network_mode = "host"

and the job succeeds.

Instead of hard-coding the address, I could also just pass -e GODEBUG="netdns=go+ipv6", and this somehow worked.

Checklist

• I made sure that the markdown files are formatted properly.
• I made sure that all hyperlinks are working.