Skip to content

[rhoai-2.25] docs(lockfile): align custom flavor on Dockerfile.konflux.<flavor> #2705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jiridanek wants to merge 2 commits into
base: rhoai-2.25
Choose a base branch
from
Open

[rhoai-2.25] docs(lockfile): align custom flavor on Dockerfile.konflux.<flavor> #2705

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3630278
docs(lockfile): align custom flavor on Dockerfile.konflux.<flavor>
jiridanek Aug 4, 2026
dd12f37
fix(lockfile): auto-select flavor for CUDA/ROCm-only projects
jiridanek Aug 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions scripts/lockfile-generators/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ and a full walkthrough (including jupyter datascience).
| ---------------------- | ---------------------------------------------------------------------- |
| `--component-dir DIR` | Component directory (required), e.g. `codeserver/ubi9-python-3.12` |
| `--rhds` | Use downstream (RHDS) lockfiles instead of upstream (ODH, the default) |
| `--flavor NAME` | Lock file flavor (default: `cpu`) |
| `--flavor NAME` | Lock file flavor (default: `cpu`, or first available `Dockerfile.konflux.{cpu,cuda,rocm}` when `cpu` is absent) |
| `--activation-key KEY` | Red Hat activation key for RHEL RPMs (optional) |
| `--org ORG` | Red Hat organization ID for RHEL RPMs (optional) |

Expand Down Expand Up @@ -850,7 +850,7 @@ The script performs these steps:
| Option | Description |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--pyproject-toml FILE` | Path to `pyproject.toml` (required). Output files are written to the same directory. |
| `--flavor NAME` | Lock file flavor (default: `cpu`). Must match a `Dockerfile.<flavor>` and `build-args/<flavor>.conf` in the project directory. Determines output filenames (`pylock.<flavor>.toml` and `requirements.<flavor>.txt`). |
| `--flavor NAME` | Lock file flavor (default: `cpu`, or the first available `Dockerfile.konflux.{cpu,cuda,rocm}` when `cpu` is absent). Must match `Dockerfile.konflux.<flavor>` in the project directory. The `rh-index` flow also requires `build-args/konflux.<flavor>.conf`. Determines output filenames (`pylock.<flavor>.toml` and `requirements.<flavor>.txt`). |
| `--download` | After generating the lock, download all wheels into `cachi2/output/deps/pip/` (for local testing with podman; not needed in Konflux CI). |


Expand Down Expand Up @@ -878,9 +878,10 @@ This command:
./scripts/lockfile-generators/create-requirements-lockfile.sh \
--pyproject-toml codeserver/ubi9-python-3.12/pyproject.toml

# Custom flavor (e.g. cuda — requires Dockerfile.konflux.cuda and build-args/cuda.conf)
# Custom flavor — rh-index flow (downstream 3.5+): requires
# Dockerfile.konflux.<flavor> and build-args/konflux.<flavor>.conf
./scripts/lockfile-generators/create-requirements-lockfile.sh \
--pyproject-toml codeserver/ubi9-python-3.12/pyproject.toml \
--pyproject-toml jupyter/minimal/ubi9-python-3.12/pyproject.toml \
--flavor cuda
```

Expand Down
47 changes: 41 additions & 6 deletions scripts/lockfile-generators/create-requirements-lockfile.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,18 @@ set -euo pipefail
# ./scripts/lockfile-generators/create-requirements-lockfile.sh \
# --pyproject-toml codeserver/ubi9-python-3.12/pyproject.toml --download
#
# # Custom flavor
# # Custom flavor — rh-index flow (downstream 3.5+):
# # Dockerfile.konflux.<flavor> and build-args/konflux.<flavor>.conf
# ./scripts/lockfile-generators/create-requirements-lockfile.sh \
# --pyproject-toml codeserver/ubi9-python-3.12/pyproject.toml --flavor cuda
# --pyproject-toml jupyter/minimal/ubi9-python-3.12/pyproject.toml --flavor cuda

SCRIPTS_PATH="scripts/lockfile-generators"
PYLOCKS_GENERATOR="scripts/pylocks_generator.py"

# --- Defaults ---
PYPROJECT=""
FLAVOR="cpu"
FLAVOR_EXPLICIT=false
DO_DOWNLOAD=false

# --- Functions ---
Expand All @@ -47,9 +49,11 @@ a pip-compatible requirements.<flavor>.txt with sha256 hashes.
Options:
--pyproject-toml FILE Path to pyproject.toml (required)
(e.g. codeserver/ubi9-python-3.12/pyproject.toml)
--flavor NAME Lock file flavor (default: cpu).
Must match a Dockerfile.<flavor> and
build-args/konflux.<flavor>.conf for the RH-index flow.
--flavor NAME Lock file flavor (default: cpu, or the first available
Dockerfile.konflux.{cpu,cuda,rocm} when cpu is absent).
Must match Dockerfile.konflux.<flavor> in the
project directory. The rh-index flow also requires
build-args/konflux.<flavor>.conf.
--download After generating, download all wheels into
cachi2/output/deps/pip/ for offline builds.
-h, --help Show this help message and exit
Expand All @@ -67,6 +71,35 @@ error_exit() {
exit 1
}

# resolve_konflux_flavor PROJECT_DIR FLAVOR FLAVOR_EXPLICIT
# Echoes the flavor to use. When FLAVOR is the default and its Dockerfile is
# missing, picks the first available cpu/cuda/rocm Dockerfile.konflux.* instead.
resolve_konflux_flavor() {
local project_dir="$1"
local flavor="$2"
local explicit="$3"
local candidate

if [[ -f "${project_dir}/Dockerfile.konflux.${flavor}" ]]; then
echo "$flavor"
return 0
fi

if [[ "$explicit" == true ]]; then
error_exit "Konflux Dockerfile not found: ${project_dir}/Dockerfile.konflux.${flavor}"
fi

for candidate in cpu cuda rocm; do
if [[ -f "${project_dir}/Dockerfile.konflux.${candidate}" ]]; then
echo "Note: auto-selected flavor '${candidate}' (no Dockerfile.konflux.${flavor})" >&2
echo "$candidate"
return 0
fi
done

error_exit "No Dockerfile.konflux.{cpu,cuda,rocm} found in ${project_dir}; use --flavor"
}

# --- Validation ---
if [[ ! -d "$SCRIPTS_PATH" ]]; then
error_exit "This script MUST be run from the repository root."
Expand All @@ -80,7 +113,7 @@ while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) show_help; exit 0 ;;
--pyproject-toml) PYPROJECT="$2"; shift 2 ;;
--flavor) FLAVOR="$2"; shift 2 ;;
--flavor) FLAVOR="$2"; FLAVOR_EXPLICIT=true; shift 2 ;;
--download) DO_DOWNLOAD=true; shift ;;
*) error_exit "Unknown argument: '$1'" ;;
esac
Expand All @@ -91,6 +124,7 @@ done

# Derive paths
PROJECT_DIR="$(dirname "$PYPROJECT")"
FLAVOR="$(resolve_konflux_flavor "$PROJECT_DIR" "$FLAVOR" "$FLAVOR_EXPLICIT")"
REQUIREMENTS_FILE="${PROJECT_DIR}/requirements.${FLAVOR}.txt"

# Use public-index when PROJECT_DIR equals a listed path or is a subdirectory (e.g. .../ubi9-python-3.12).
Expand All @@ -111,6 +145,7 @@ fi

PYLOCK_FILE="${PROJECT_DIR}/uv.lock.d/pylock.${FLAVOR}.toml"
REQUIREMENTS_INDEX_URL=""
KONFLUX_DOCKERFILE="${PROJECT_DIR}/Dockerfile.konflux.${FLAVOR}"
if [[ "$PYLOCKS_MODE" == "public-index" ]]; then
PYLOCK_FILE="${PROJECT_DIR}/pylock.toml"
HERMETO_INDEX_URL="https://pypi.org/simple"
Expand Down
34 changes: 32 additions & 2 deletions scripts/lockfile-generators/prefetch-all.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ SCRIPTS_PATH="scripts/lockfile-generators"
COMPONENT_DIR=""
VARIANT="odh" # "odh" = upstream (CentOS Stream), "rhds" = downstream (RHEL)
FLAVOR="cpu" # selects which pylock/requirements files to use (cpu, cuda, rocm)
FLAVOR_EXPLICIT=false
ACTIVATION_KEY=""
ORG=""

Expand All @@ -60,7 +61,8 @@ Options:
--component-dir DIR Component directory (required)
e.g. codeserver/ubi9-python-3.12
--rhds Use downstream (RHDS) lockfiles instead of upstream (ODH)
--flavor NAME Lock file flavor (default: cpu)
--flavor NAME Lock file flavor (default: cpu, or first available
Dockerfile.konflux.{cpu,cuda,rocm} when cpu is absent)
Comment on lines +64 to +65

@coderabbitai coderabbitai Bot Aug 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Document explicit flavor validation in prefetch-all.sh --help.

The resolver rejects an explicitly supplied flavor when Dockerfile.konflux.<flavor> is missing. The help text only documents the default fallback. Add the exact-match requirement so users can predict validation behavior.

Proposed help-text update 
   --flavor NAME           Lock file flavor (default: cpu, or first available
                           Dockerfile.konflux.{cpu,cuda,rocm} when cpu is absent)
+                          Must match Dockerfile.konflux.<flavor> in the
+                          component directory.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
--flavor NAME Lock file flavor (default: cpu, or first available
Dockerfile.konflux.{cpu,cuda,rocm} when cpu is absent)
--flavor NAME Lock file flavor (default: cpu, or first available
Dockerfile.konflux.{cpu,cuda,rocm} when cpu is absent)
Must match Dockerfile.konflux.<flavor> in the
component directory.
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/lockfile-generators/prefetch-all.sh` around lines 64 - 65, Update the
--flavor option help text in prefetch-all.sh to state that an explicitly
supplied flavor must exactly match an available Dockerfile.konflux.<flavor>;
retain the existing documentation for the cpu default and fallback behavior.

--activation-key KEY Red Hat activation key for RHEL RPMs (optional)
--org ORG Red Hat organization ID for RHEL RPMs (optional)
-h, --help Show this help
Expand All @@ -76,6 +78,33 @@ error_exit() {
exit 1
}

# resolve_konflux_flavor PROJECT_DIR FLAVOR FLAVOR_EXPLICIT
resolve_konflux_flavor() {
local project_dir="$1"
local flavor="$2"
local explicit="$3"
local candidate

if [[ -f "${project_dir}/Dockerfile.konflux.${flavor}" ]]; then
echo "$flavor"
return 0
fi

if [[ "$explicit" == true ]]; then
error_exit "Konflux Dockerfile not found: ${project_dir}/Dockerfile.konflux.${flavor}"
fi

for candidate in cpu cuda rocm; do
if [[ -f "${project_dir}/Dockerfile.konflux.${candidate}" ]]; then
echo "Note: auto-selected flavor '${candidate}' (no Dockerfile.konflux.${flavor})" >&2
echo "$candidate"
return 0
fi
done

error_exit "No Dockerfile.konflux.{cpu,cuda,rocm} found in ${project_dir}; use --flavor"
}

# find_tekton_yaml COMPONENT_DIR VARIANT
# Finds .tekton/*pull-request*.yaml files that build this component for the
# given variant by matching the pipeline's dockerfile param. Requires yq;
Expand Down Expand Up @@ -119,7 +148,7 @@ while [[ $# -gt 0 ]]; do
COMPONENT_DIR="$2"; shift 2 ;;
--rhds) VARIANT="rhds"; shift ;;
--flavor) [[ $# -ge 2 ]] || error_exit "--flavor requires a value"
FLAVOR="$2"; shift 2 ;;
FLAVOR="$2"; FLAVOR_EXPLICIT=true; shift 2 ;;
--activation-key) [[ $# -ge 2 ]] || error_exit "--activation-key requires a value"
ACTIVATION_KEY="$2"; shift 2 ;;
--org) [[ $# -ge 2 ]] || error_exit "--org requires a value"
Expand All @@ -131,6 +160,7 @@ done

[[ -z "$COMPONENT_DIR" ]] && error_exit "--component-dir is required."
[[ -d "$COMPONENT_DIR" ]] || error_exit "Component directory not found: $COMPONENT_DIR"
FLAVOR="$(resolve_konflux_flavor "$COMPONENT_DIR" "$FLAVOR" "$FLAVOR_EXPLICIT")"

@coderabbitai coderabbitai Bot Aug 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -u

rg -n -C 10 \
  'find_tekton_yaml|TEKTON_YAML|FLAVOR=|dockerfile_path' \
  scripts/lockfile-generators/prefetch-all.sh

printf '\nFlavor-specific Tekton references:\n'
rg -n -C 4 \
  'Dockerfile\.konflux\.(cpu|cuda|rocm)|Dockerfile\.konflux\.' \
  .tekton scripts --glob '*.yaml' --glob '*.sh'

Repository: red-hat-data-services/notebooks

Length of output: 50387

🏁 Script executed:

#!/usr/bin/env bash
set -u

printf '%s\n' 'Relevant implementation:'
sed -n '74,140p;292,325p' scripts/lockfile-generators/prefetch-all.sh

printf '\n%s\n' 'NPM downloader input handling:'
fd -i 'download-npm' scripts/lockfile-generators
sed -n '1,220p' scripts/lockfile-generators/download-npm.sh

printf '\n%s\n' 'Pull-request Tekton files grouped by component directory and flavor:'
python3 - <<'PY'
from pathlib import Path
import re
from collections import defaultdict

groups = defaultdict(list)
for path in sorted(Path(".tekton").glob("*pull-request*.yaml")):
    text = path.read_text()
    m = re.search(
        r'(?m)^\s*value:\s*["\']?([^"\']*?/Dockerfile\.konflux\.([A-Za-z0-9_-]+))["\']?\s*$',
        text,
    )
    if not m:
        continue
    dockerfile, flavor = m.groups()
    component = dockerfile.rsplit("/Dockerfile.konflux.", 1)[0]
    groups[component].append((flavor, str(path), dockerfile))

for component, entries in groups.items():
    flavors = sorted({flavor for flavor, _, _ in entries})
    if len(flavors) > 1:
        print(component)
        for flavor, path, dockerfile in entries:
            print(f"  {flavor}: {path} -> {dockerfile}")
PY

Repository: red-hat-data-services/notebooks

Length of output: 12884

🏁 Script executed:

#!/usr/bin/env bash
set -u

python3 - <<'PY'
from pathlib import Path
from collections import defaultdict
import re

# Extract the scalar immediately following a dockerfile parameter.
records = []
for path in sorted(Path(".tekton").glob("*pull-request*.yaml")):
    lines = path.read_text().splitlines()
    for i, line in enumerate(lines):
        if re.match(r"^\s*-\s*name:\s*dockerfile\s*$", line):
            for value_line in lines[i + 1:i + 5]:
                m = re.match(r"^\s*value:\s*[\"']?([^\"']+?)[\"']?\s*$", value_line)
                if m:
                    dockerfile = m.group(1).strip()
                    if "/Dockerfile.konflux." in dockerfile:
                        component, flavor = dockerfile.rsplit("/Dockerfile.konflux.", 1)
                        records.append((component, flavor, str(path), dockerfile))
                    break
            break

by_component = defaultdict(list)
for record in records:
    by_component[record[0]].append(record)

print(f"pull-request YAMLs with Konflux dockerfiles: {len(records)}")
print(f"components represented: {len(by_component)}")
print("\nComponents with multiple matching pull-request YAMLs:")
collision_count = 0
for component in sorted(by_component):
    entries = by_component[component]
    if len(entries) > 1:
        collision_count += 1
        print(component)
        for _, flavor, path, dockerfile in entries:
            print(f"  {flavor}\t{path}\t{dockerfile}")
if collision_count == 0:
    print("  none")

print("\nFlavor distribution:")
flavors = defaultdict(int)
for _, flavor, _, _ in records:
    flavors[flavor] += 1
for flavor, count in sorted(flavors.items()):
    print(f"  {flavor}: {count}")

print("\nFirst matching YAML for each component under lexical glob order:")
for component in sorted(by_component):
    first = by_component[component][0]
    print(f"{component}\t{first[1]}\t{first[2]}")
PY

printf '\nDockerfiles with multiple Konflux flavors:\n'
find . -type f -name 'Dockerfile.konflux.*' -print \
  | sed 's#^\./##' \
  | awk '
    {
      path=$0
      sub(/\/Dockerfile\.konflux\.[^/]+$/, "", path)
      count[path]++
      files[path]=files[path] "\n    " $0
    }
    END {
      for (path in count)
        if (count[path] > 1)
          print path ":" files[path]
    }
  ' | sort

Repository: red-hat-data-services/notebooks

Length of output: 2887

🏁 Script executed:

#!/usr/bin/env bash
set -u

for f in \
  .tekton/odh-workbench-jupyter-minimal-cpu-py312-pull-request.yaml \
  .tekton/odh-workbench-jupyter-minimal-cuda-py312-pull-request.yaml \
  .tekton/odh-workbench-jupyter-minimal-rocm-py312-pull-request.yaml
do
  printf '\n--- %s ---\n' "$f"
  rg -n -C 3 'name: dockerfile|name: prefetch-input|type: npm|path:|build-args-file' "$f"
done

printf '\nFlavor-specific Dockerfiles and package-lock locations:\n'
for f in \
  jupyter/minimal/ubi9-python-3.12/Dockerfile.konflux.cpu \
  jupyter/minimal/ubi9-python-3.12/Dockerfile.konflux.cuda \
  jupyter/minimal/ubi9-python-3.12/Dockerfile.konflux.rocm
do
  printf '\n--- %s ---\n' "$f"
  rg -n '^(FROM|COPY|ADD)|package-lock|package\.json|prefetch-input|npm' "$f" || true
done

printf '\nNPM prefetch paths by YAML (textual YAML extraction):\n'
python3 - <<'PY'
from pathlib import Path
import re

files = [
    Path(".tekton/odh-workbench-jupyter-minimal-cpu-py312-pull-request.yaml"),
    Path(".tekton/odh-workbench-jupyter-minimal-cuda-py312-pull-request.yaml"),
    Path(".tekton/odh-workbench-jupyter-minimal-rocm-py312-pull-request.yaml"),
]
for path in files:
    lines = path.read_text().splitlines()
    dockerfile = next(
        (re.match(r"\s*value:\s*[\"']?([^\"']+?)[\"']?\s*$", lines[i + 1]).group(1)
         for i, line in enumerate(lines[:-1])
         if re.match(r"\s*-\s*name:\s*dockerfile\s*$", line)
         and re.match(r"\s*value:", lines[i + 1])),
        "<not found>",
    )
    npm_paths = []
    in_npm = False
    for i, line in enumerate(lines):
        if re.match(r"\s*-\s*type:\s*npm\s*$", line):
            in_npm = True
        elif in_npm:
            m = re.match(r"\s*path:\s*(\S+)\s*$", line)
            if m:
                npm_paths.append(m.group(1))
                in_npm = False
            elif re.match(r"\s*-\s*type:", line):
                in_npm = False
    print(path)
    print(f"  dockerfile: {dockerfile}")
    print(f"  npm paths: {npm_paths}")
PY

Repository: red-hat-data-services/notebooks

Length of output: 3249

Make find_tekton_yaml flavor-aware.

jupyter/minimal/ubi9-python-3.12 has CPU, CUDA, and ROCm pull-request YAML files. head -1 selects the CPU file for every resolved flavor. The current files have no npm inputs, but this can select incorrect npm inputs when flavor-specific inputs exist. Match Dockerfile.konflux.${FLAVOR} exactly.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/lockfile-generators/prefetch-all.sh` at line 163, Update
find_tekton_yaml to select the Tekton YAML whose Dockerfile declaration matches
Dockerfile.konflux.${FLAVOR} exactly, rather than relying on head -1. Pass the
resolved FLAVOR from the surrounding prefetch flow so CPU, CUDA, and ROCm
configurations resolve their corresponding YAML and inputs.


# CLI args take priority; fall back to env vars so GHA can pass secrets
# without exposing them on the command line. GitHub Actions masks env var
Expand Down
Loading