Skip to content

Commit

Permalink
Use Ubuntu 18.10 in Linux build container
Browse files Browse the repository at this point in the history
Background
==========

Shared Library Dependencies
---------------------------

The Linux build of Arduino IDE has dynamic linkage against the libstdc++ and glibc shared libraries. This results in
it having a dependency on the version of the libraries that happens to be present in the environment it is built in.

Although newer versions of the shared libraries are compatible with executables linked against an older version, the
reverse is not true. This means that building Arduino IDE on a Linux machine with a recent distro version installed
causes the IDE to error on startup for users who have a distro with older versions of the dependencies.

For example, if Arduino IDE were built on a machine with version 3.4.33 of libstdc++, then attempting to run it on a
machine with an older version of libstdc++ would fail with an error like:

```
Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.33' not found (required by /home/foo/arduino-ide/resources/app/lib/backend/native/nsfw.node)
```

Likewise,  if Arduino IDE were built on a machine with version 2.39 of glibc, then attempting to run it on a machine
with an older version of glibc would fail with an error like:

```
Error: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.39' not found (required by /home/foo/arduino-ide/resources/app/node_modules/nsfw/build/Release/nsfw.node)
```

Build Machine Requirements
--------------------------

The IDE builds distributed by Arduino should be compatible with a reasonable range of Linux distribution versions. In
order to achieve this, the builds must be performed in a machine with an older version of the shared libraries. The
shared libraries are part of the Linux distro, and installing a different version is not feasible. So this imposes a
maximum limit on the build machine's distro version.

The distributed builds are generated via a GitHub Actions workflow. The most simple approach is to run the build in the
machine of the GitHub-hosted runners provided for each operating system. However, GitHub provides a limited range of
operating system versions in their runners, and removes the older versions as newer versions are added. This means that
building in the GitHub-hosted runner machine would not allow for the desired range of Linux distro version
compatibility. For this reason, the Linux build is performed in a Docker container that provides an older version of
Ubuntu.

The same situation of incompatibility with Linux distro versions that have a version of the shared library dependencies
older than the version present on the build machine occurs for several of the tools and frameworks used by the build
process (e.g., Node.js, Python). In this case, the tables are turned as we are now the user rather than the distributor
and so are at the mercy of the Linux distro version compatibility range provided by the distributor. So this imposes a
minimum limit on the build machine's distro version.

Although several of the dependencies used by the standard build system have dependencies on versions of glibc higher
than the version 2.27 present in Ubuntu 18.04, it was possible to use this distro version in the Linux build container
by using alternative distributions and/or versions of these dependencies.

Workflow Artifacts
------------------

The build workflow uses GitHub actions workflow artifacts to transfer the files generated by the build job to subsequent
jobs in the workflow. The "actions/upload-artifact" action is used for this purpose.

Problem
=======

GitHub is dropping support for the workflow artifacts produced by the version 3.x of the "actions/upload-artifact"
action that was previously used by the build job. So the action version used in the build workflow was updated to the
current version 4.x. This version of the action uses a newer version of the Node.js runtime (20). Unfortunately the the
Node.js 20 runtime used by the action has a dependency on glibc version 2.28, which causes the Linux build job to fail
after the update of the "actions/upload-artifact" action:

```
Run actions/upload-artifact@v4
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
```

Unlike the other dependencies of the build process, it is no longer possible to work around this incompatibility by
continuing to use the older compatible version of the "actions/upload-artifact" action. It is also impossible to replace
the incompatible Node.js 20.x distribution used by the action, since it comes from the read-only file system of the
runner image. Likewise, it is not possible to configure or force the action to use a Node.js installation at a different
path on the runner machine.

Resolution
==========

Compatibility with the new version of the "actions/upload-artifact" action is attained by updating the version of Linux
in the build container to 18.10, which is the oldest version that has glibc 2.28. The presence of a newer glibc version
in the container also makes it compatible with several other dependencies of the build process, meaning the code in the
Dockerfile and workflow for working around the incompatibilities of Ubuntu 18.04 can be removed.

Consequences
============

Unfortunately this means the loss of compatibility of the Linux Arduino IDE builds with distros that use glibc 2.27
(e.g., Ubuntu 18.04). User of those distros will now find that Arduino IDE fails to start with an error like:

```
Error: node-loader:
Error: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/foo/arduino-ide/resources/app/lib/backend/native/pty.node)
    at 85467 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:2766)
    at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
    at 23571 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:3374073)
    at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
    at 55444 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:3369761)
    at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
    at 24290 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:1780542)
    at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
    at 43416 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:1770138)
    at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
```
  • Loading branch information
per1234 committed Nov 27, 2024
1 parent dba57b3 commit c09b5f7
Show file tree
Hide file tree
Showing 2 changed files with 37 additions and 95 deletions.
93 changes: 22 additions & 71 deletions .github/workflows/assets/linux.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,43 +1,28 @@
# The Arduino IDE Linux build workflow job runs in this container.
# syntax=docker/dockerfile:1

FROM ubuntu:18.04

# See: https://unofficial-builds.nodejs.org/download/release/
ARG node_version="18.17.1"
# See: https://hub.docker.com/_/ubuntu/tags
FROM ubuntu:18.10

# This is required in order to use the Ubuntu package repositories for EOL Ubuntu versions:
# https://help.ubuntu.com/community/EOLUpgrades#Update_sources.list
RUN \
apt-get \
--yes \
update
sed \
--in-place \
--regexp-extended \
--expression='s/([a-z]{2}\.)?archive.ubuntu.com|security.ubuntu.com/old-releases.ubuntu.com/g' \
"/etc/apt/sources.list"

# This is required to get add-apt-repository
RUN \
apt-get \
--yes \
install \
"software-properties-common=0.96.24.32.22"
update

# Install Git
# The PPA is required to get a modern version of Git. The version in the Ubuntu 18.04 package repository is 2.17.1,
# while action/checkout@v3 requires 2.18 or higher.
RUN \
add-apt-repository \
--yes \
"ppa:git-core/ppa" && \
apt-get \
--yes \
update && \
\
apt-get \
--yes \
install \
"git" && \
\
apt-get \
--yes \
purge \
"software-properties-common"
"git"

# The repository path must be added to safe.directory, otherwise any Git operations on it would fail with a
# "dubious ownership" error. actions/checkout configures this, but it is not applied to containers.
Expand All @@ -51,62 +36,28 @@ ENV \

# Install Python
# The Python installed by actions/setup-python has dependency on a higher version of glibc than available in the
# ubuntu:18.04 container.
# container.
RUN \
apt-get \
--yes \
install \
"python3.8-minimal=3.8.0-3ubuntu1~18.04.2" && \
\
ln \
--symbolic \
--force \
"$(which python3.8)" \
"/usr/bin/python3"
"python3.7-minimal=3.7.3-2~18.10"

# Install Theia's package dependencies
# These are pre-installed in the GitHub Actions hosted runner machines.
RUN \
apt-get \
--yes \
install \
"libsecret-1-dev=0.18.6-1" \
"libx11-dev=2:1.6.4-3ubuntu0.4" \
"libsecret-1-dev=0.18.6-3" \
"libx11-dev=2:1.6.7-1" \
"libxkbfile-dev=1:1.0.9-2"

# Install Node.js
# It is necessary to use the "unofficial" linux-x64-glibc-217 build because the official Node.js 18.x is dynamically
# linked against glibc 2.28, while Ubuntu 18.04 has glibc 2.27.
ARG node_installation_path="/tmp/node-installation"
ARG artifact_name="node-v${node_version}-linux-x64-glibc-217"
RUN \
mkdir "$node_installation_path" && \
cd "$node_installation_path" && \
\
apt-get \
--yes \
install \
"wget=1.19.4-1ubuntu2.2" && \
\
archive_name="${artifact_name}.tar.xz" && \
wget \
"https://unofficial-builds.nodejs.org/download/release/v${node_version}/${archive_name}" && \
\
apt-get \
--yes \
purge \
"wget" && \
\
tar \
--file="$archive_name" \
--extract && \
rm "$archive_name"
ENV PATH="${PATH}:${node_installation_path}/${artifact_name}/bin"

# Install Yarn
# Yarn is pre-installed in the GitHub Actions hosted runner machines.
# Target python3 symlink to Python 3.7 installation. It would otherwise target version 3.6 due to the installation of
# the `python3` package as a transitive dependency.
RUN \
npm \
install \
--global \
"[email protected]"
ln \
--symbolic \
--force \
"$(which python3.7)" \
"/usr/bin/python3"
39 changes: 15 additions & 24 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ env:
GO_VERSION: '1.21'
# See: https://github.com/actions/setup-node/#readme
NODE_VERSION: '18.17'
YARN_VERSION: '1.22'
JOB_TRANSFER_ARTIFACT_PREFIX: build-artifacts-
CHANGELOG_ARTIFACTS: changelog
STAGED_CHANNEL_FILE_ARTIFACT_PREFIX: staged-channel-file-
Expand Down Expand Up @@ -316,21 +317,26 @@ jobs:
if not exist "${{ matrix.config.working-directory }}" mklink /d "${{ matrix.config.working-directory }}" "C:\actions-runner\_work\arduino-ide\arduino-ide"
- name: Checkout
if: fromJSON(matrix.config.container) == null
uses: actions/checkout@v4

- name: Checkout
# actions/checkout@v4 has dependency on a higher version of glibc than available in the Linux container.
if: fromJSON(matrix.config.container) != null
uses: actions/checkout@v3

- name: Install Node.js
if: fromJSON(matrix.config.container) == null && runner.name != 'WINDOWS-SIGN-PC'
if: runner.name != 'WINDOWS-SIGN-PC'
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
registry-url: 'https://registry.npmjs.org'
cache: 'yarn'
# Yarn is a prerequisite for the action's cache feature, so caching should be disabled when running in the
# container where Yarn is not pre-installed.
cache: ${{ fromJSON(matrix.config.container) == null && 'yarn' || null }}

- name: Install Yarn
if: runner.name != 'WINDOWS-SIGN-PC'
run: |
npm \
install \
--global \
"yarn@${{ env.YARN_VERSION }}"
- name: Install Python 3.x
if: fromJSON(matrix.config.container) == null && runner.name != 'WINDOWS-SIGN-PC'
Expand All @@ -339,33 +345,18 @@ jobs:
python-version: '3.11.x'

- name: Install Go
if: fromJSON(matrix.config.container) == null && runner.name != 'WINDOWS-SIGN-PC'
if: runner.name != 'WINDOWS-SIGN-PC'
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}

- name: Install Go
# actions/setup-go@v5 has dependency on a higher version of glibc than available in the Linux container.
if: fromJSON(matrix.config.container) != null && runner.name != 'WINDOWS-SIGN-PC'
uses: actions/setup-go@v4
with:
go-version: ${{ env.GO_VERSION }}

- name: Install Taskfile
if: fromJSON(matrix.config.container) == null && runner.name != 'WINDOWS-SIGN-PC'
if: runner.name != 'WINDOWS-SIGN-PC'
uses: arduino/setup-task@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
version: 3.x

- name: Install Taskfile
# actions/setup-task@v2 has dependency on a higher version of glibc than available in the Linux container.
if: fromJSON(matrix.config.container) != null && runner.name != 'WINDOWS-SIGN-PC'
uses: arduino/setup-task@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
version: 3.x

- name: Package
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down

0 comments on commit c09b5f7

Please sign in to comment.