Add devcontainer integrations (#451)

* Add devcontainer integrations * Provide two container definitions for defaults and conda-forge * fix mamba support, add some echo's * pre-commit * add news * update docs * add some apt deps * fix mamba dev workflow * add git + ssh * fix failure report * Apply suggestions from code review Co-authored-by: Klaus Zimmermann <[email protected]> * bail instead * rename env vars * update dev docs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Apply suggestions from code review Co-authored-by: Klaus Zimmermann <[email protected]> * Update .devcontainer/conda-forge/devcontainer.json Co-authored-by: Klaus Zimmermann <[email protected]> --------- Co-authored-by: Klaus Zimmermann <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
conda · Mar 22, 2024 · 1f4fccb · 1f4fccb
1 parent 055d5f9
commit 1f4fccb
Show file tree

Hide file tree

Showing 14 changed files with 303 additions and 193 deletions.
diff --git a/.devcontainer/apt-deps.txt b/.devcontainer/apt-deps.txt
@@ -0,0 +1,5 @@
+git
+less
+htop
+nano
+ssh
diff --git a/.devcontainer/conda-forge/devcontainer.json b/.devcontainer/conda-forge/devcontainer.json
@@ -0,0 +1,42 @@
+// For format details, see https://aka.ms/devcontainer.json. For config options, see the
+{
+    "name": "Miniforge (default-channel=conda-forge)",
+    "image": "condaforge/miniforge3:latest",
+
+    // Features to add to the dev container. More info: https://containers.dev/features.
+    // "features": {},
+
+    // Use 'forwardPorts' to make a list of ports inside the container available locally.
+    // "forwardPorts": [],
+    "mounts": [
+        "source=${localWorkspaceFolder}/../conda,target=/workspaces/conda,type=bind,consistency=cached",
+        "source=${localWorkspaceFolder}/../mamba,target=/workspaces/mamba,type=bind,consistency=cached"
+    ],
+    // Use 'postCreateCommand' to run commands after the container is created.
+    "postCreateCommand": "bash /workspaces/conda-libmamba-solver/.devcontainer/post_create.sh",
+    // Use 'postStartCommand' to run commands after the container is started.
+    "postStartCommand": "bash /workspaces/conda-libmamba-solver/.devcontainer/post_start.sh",
+
+    // Configure tool-specific properties.
+    "customizations": {
+        "vscode": {
+            "settings": {
+                "python.defaultInterpreterPath": "/opt/conda/bin/python",
+                "python.testing.pytestArgs": [
+                    "tests"
+                ],
+                "python.testing.unittestEnabled": false,
+                "python.testing.pytestEnabled": true
+            },
+            "extensions": [
+                "charliermarsh.ruff",
+                "eamodio.gitlens",
+                "ms-toolsai.jupyter"
+            ]
+        }
+    }
+
+    // Adjust to connect as non-root instead. More info: https://aka.ms/dev-containers-non-root.
+    // "remoteUser": "root",
+
+}
diff --git a/.devcontainer/defaults/devcontainer.json b/.devcontainer/defaults/devcontainer.json
@@ -0,0 +1,42 @@
+// For format details, see https://aka.ms/devcontainer.json
+{
+    "name": "Miniconda (default-channel=defaults)",
+    "image": "continuumio/miniconda3:latest",
+
+    // Features to add to the dev container. More info: https://containers.dev/features.
+    // "features": {},
+
+    // Use 'forwardPorts' to make a list of ports inside the container available locally.
+    // "forwardPorts": [],
+    "mounts": [
+        "source=${localWorkspaceFolder}/../conda,target=/workspaces/conda,type=bind,consistency=cached",
+        "source=${localWorkspaceFolder}/../mamba,target=/workspaces/mamba,type=bind,consistency=cached"
+    ],
+    // Use 'postCreateCommand' to run commands after the container is created.
+    "postCreateCommand": "bash /workspaces/conda-libmamba-solver/.devcontainer/post_create.sh",
+    // Use 'postStartCommand' to run commands after the container is started.
+    "postStartCommand": "bash /workspaces/conda-libmamba-solver/.devcontainer/post_start.sh",
+
+    // Configure tool-specific properties.
+    "customizations": {
+        "vscode": {
+            "settings": {
+                "python.defaultInterpreterPath": "/opt/conda/bin/python",
+                "python.testing.pytestArgs": [
+                    "tests"
+                ],
+                "python.testing.unittestEnabled": false,
+                "python.testing.pytestEnabled": true
+            },
+            "extensions": [
+                "charliermarsh.ruff",
+                "eamodio.gitlens",
+                "ms-toolsai.jupyter"
+            ]
+        }
+    }
+
+    // Adjust to connect as non-root instead. More info: https://aka.ms/dev-containers-non-root.
+    // "remoteUser": "root",
+
+}
diff --git a/.devcontainer/post_create.sh b/.devcontainer/post_create.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+# This script assumes we are running in a Miniconda container where:
+# - /opt/conda is the Miniconda or Miniforge installation directory
+# - https://github.com/conda/conda is mounted at /workspaces/conda
+# - https://github.com/conda/conda-libmamba-solver is mounted at
+#   /workspaces/conda-libmamba-solver
+# - https://github.com/mamba-org/mamba is (optionally) mounted at
+#   /workspaces/mamba
+
+set -euo pipefail
+
+HERE=$(dirname $0)
+BASE_CONDA=${BASE_CONDA:-/opt/conda}
+SRC_CONDA=${SRC_CONDA:-/workspaces/conda}
+SRC_CONDA_LIBMAMBA_SOLVER=${SRC_CONDA_LIBMAMBA_SOLVER:-/workspaces/conda-libmamba-solver}
+
+if which apt-get > /dev/null; then
+    echo "Installing system dependencies"
+    apt-get update
+    DEBIAN_FRONTEND=noninteractive xargs -a "$HERE/apt-deps.txt" apt-get install -y
+fi
+
+
+if [ ! -f "$SRC_CONDA/pyproject.toml" ]; then
+    echo "https://github.com/conda/conda not found! Please clone or mount to $SRC_CONDA"
+    exit 1
+fi
+
+# Clear history to avoid unneeded conflicts
+echo "Clearing base history..."
+echo '' > "$BASE_CONDA/conda-meta/history"
+
+echo "Installing dev & test dependencies..."
+"$BASE_CONDA/bin/conda" install -n base --yes --quiet \
+    --file="$SRC_CONDA/tests/requirements.txt" \
+    --file="$SRC_CONDA/tests/requirements-ci.txt" \
+    --file="$SRC_CONDA/tests/requirements-Linux.txt" \
+    --file="$SRC_CONDA/tests/requirements-s3.txt" \
+    --file="$SRC_CONDA_LIBMAMBA_SOLVER/dev/requirements.txt" \
+    --file="$SRC_CONDA_LIBMAMBA_SOLVER/tests/requirements.txt"
diff --git a/.devcontainer/post_start.sh b/.devcontainer/post_start.sh
@@ -0,0 +1,76 @@
+#!/bin/bash
+
+# This script assumes we are running in a Miniconda container where:
+# - /opt/conda is the Miniconda or Miniforge installation directory
+# - https://github.com/conda/conda is mounted at /workspaces/conda
+# - https://github.com/conda/conda-libmamba-solver is mounted at
+#   /workspaces/conda-libmamba-solver
+# - https://github.com/mamba-org/mamba is (optionally) mounted at
+#   /workspaces/mamba
+
+set -euo pipefail
+
+BASE_CONDA=${BASE_CONDA:-/opt/conda}
+SRC_CONDA=${SRC_CONDA:-/workspaces/conda}
+SRC_CONDA_LIBMAMBA_SOLVER=${SRC_CONDA_LIBMAMBA_SOLVER:-/workspaces/conda-libmamba-solver}
+SRC_MAMBA=${SRC_MAMBA:-/workspaces/mamba}
+
+cat >> ~/.bashrc <<EOF
+function develop-mamba() (
+  # Runs in a subshell to avoid polluting the current shell
+  set -euo pipefail
+  if ! conda config --show channels | grep -q conda-forge; then
+    echo "Miniconda not compatible with develop-mamba"
+    exit 1
+  fi
+  if [ ! -f "$SRC_MAMBA/mamba/setup.py" ]; then
+    echo "Mamba 1.x not found at $SRC_MAMBA"
+    exit 1
+  fi
+  # Install mamba dev dependencies only once:
+  if [ ! -f ~/.mamba-develop-installed ]; then
+    # remove "sel(win)" in environment yaml hack since conda does not understand
+    # libmamba specific specs
+    echo "Installing mamba 1.x in dev mode..."
+    sed '/sel(.*)/d' "$SRC_MAMBA/mamba/environment-dev.yml" > /tmp/mamba-environment-dev.yml
+    CONDA_QUIET=1 "$BASE_CONDA/condabin/conda" env update -p "$BASE_CONDA" \
+        --file /tmp/mamba-environment-dev.yml
+    "$BASE_CONDA/condabin/conda" install make -yq  # missing in mamba's dev env
+    # Clean build directory to avoid issues with stale build files
+    test -f "$SRC_MAMBA/build/CMakeCache.txt" && rm -rf "$SRC_MAMBA/build"
+  fi
+  # Compile
+  cd "$SRC_MAMBA"
+  "$BASE_CONDA/bin/cmake" -B build/ \
+      -DBUILD_LIBMAMBA=ON \
+      -DBUILD_SHARED=ON \
+      -DCMAKE_INSTALL_PREFIX="$BASE_CONDA" \
+      -DCMAKE_PREFIX_PATH="$BASE_CONDA" \
+      -DBUILD_LIBMAMBAPY=ON
+  "$BASE_CONDA/bin/cmake" --build build/ -j\${NPROC:-2}
+  if [ ! -f ~/.mamba-develop-installed ]; then
+    "$BASE_CONDA/condabin/conda" remove -p "$BASE_CONDA" -yq --force libmambapy libmamba
+  fi
+  make install -C build/
+  cd -
+  "$BASE_CONDA/bin/pip" install -e "$SRC_MAMBA/libmambapy/" --no-deps
+  test -f "$BASE_CONDA/conda-meta/mamba-"*".json" && "$BASE_CONDA/bin/pip" install -e "$SRC_MAMBA/mamba/" --no-deps
+  touch ~/.mamba-develop-installed || true
+)
+EOF
+
+cd "$SRC_CONDA"
+echo "Initializing conda in dev mode..."
+"$BASE_CONDA/bin/python" -m conda init --dev bash
+cd -
+
+echo "Installing conda-libmamba-solver in dev mode..."
+"$BASE_CONDA/bin/python" -m pip install -e "$SRC_CONDA_LIBMAMBA_SOLVER" --no-deps
+
+set -x
+conda list -p "$BASE_CONDA"
+conda info
+conda config --show-sources
+set +x
+test -f "$SRC_MAMBA/mamba/setup.py" \
+  && echo "Mamba mounted at $SRC_MAMBA; source ~/.bashrc and run develop-mamba() for dev-install"
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
@@ -662,6 +662,7 @@ jobs:
     steps:
       - name: Determine Success
         uses: re-actors/[email protected]
+        id: alls-green  # CONDA-LIBMAMBA-SOLVER CHANGE
         with:
           # permit jobs to be skipped if there are no code changes (see changes job)
           allowed-skips: ${{ toJSON(needs) }}

diff --git a/dev/linux/bashrc.sh b/dev/linux/bashrc.sh
diff --git a/dev/linux/upstream_integration.sh b/dev/linux/upstream_integration.sh
diff --git a/dev/linux/upstream_unit.sh b/dev/linux/upstream_unit.sh
diff --git a/docs/dev/implementation.md b/docs/dev/implementation.md
@@ -5,10 +5,11 @@ both within the `conda_libmamba_solver` package itself, and as a `conda` plugin.
 
 ## Repository structure
 
+* `.devcontainer`: Configuration for DevContainer development workflows.
 * `.github/workflows/`: CI pipelines to run unit and upstream tests, as well as linting and performance benchmarks. Some extra workflows might be added by the `conda/infra` settings.
 * `conda_libmamba_solver/`: The Python package. Check sections below for details.
 * `recipe/`: The conda-build recipe used for the PR build previews. It should be kept in sync with `conda-forge` and `defaults`.
-* `dev/`: Supporting scripts and configuration files to set up development environments.
+* `dev/`: Supporting configuration files to set up development environments.
 * `docs/`: Documentation sources.
 * `tests/`: pytest testing infrastructure.
 * `pyproject.toml`: Project metadata. See below for details.
-Original file line number
+Diff line change
@@ -0,0 +1,5 @@
+    git
+    less
+    htop
+    nano
+    ssh