Skip to content

chore(deps): fix open rhoai-2.25 pypi CVE batch via cve-constraints - #2634

Merged
jiridanek merged 3 commits into
rhoai-2.25from
fix/rhoai-2.25-open-pypi-cves
Jul 29, 2026
Merged

chore(deps): fix open rhoai-2.25 pypi CVE batch via cve-constraints#2634
jiridanek merged 3 commits into
rhoai-2.25from
fix/rhoai-2.25-open-pypi-cves

Conversation

@jiridanek

@jiridanek jiridanek commented Jul 28, 2026

Copy link
Copy Markdown
Member

Summary

Consolidated fix for 42 open [rhoai-2.25] pypi CVE trackers via dependencies/cve-constraints.txt, targeted pyproject.toml pin/override updates, and regenerated pylock.toml files (bash scripts/sync-python-lockfiles.sh).

Supersedes #2625 (pyasn1) and #2137 (keras).

Jira tickets addressed

RHAIENG CVE Package
RHAIENG-4276 CVE-2026-34445 onnx
RHAIENG-4278 CVE-2026-35536 tornado
RHAIENG-4458 CVE-2026-1462 keras
RHAIENG-4594 CVE-2026-40192 pillow
RHAIENG-4686 CVE-2026-28684 python-dotenv
RHAIENG-5051 CVE-2026-39892 cryptography
RHAIENG-5214 CVE-2026-40110 jupyter-server
RHAIENG-5302 CVE-2026-44431 urllib3
RHAIENG-5304 CVE-2026-44432 urllib3
RHAIENG-5306 CVE-2026-8643 pip
RHAIENG-5488 CVE-2026-34993 aiohttp
RHAIENG-5701 CVE-2026-11816 keras
RHAIENG-5781 CVE-2026-12199 nltk
RHAIENG-5789 CVE-2026-42557 jupyterlab
RHAIENG-5791 CVE-2026-44660 ujson
RHAIENG-5794 CVE-2026-48526 pyjwt
RHAIENG-5800 CVE-2026-5422 jupyter-server
RHAIENG-5830 CVE-2026-33079 mistune
RHAIENG-5832 CVE-2026-35397 jupyter-server
RHAIENG-5850 CVE-2026-44727 jupyter-server
RHAIENG-5873 CVE-2026-56121 feast
RHAIENG-5883 CVE-2026-49851 mistune
RHAIENG-5886 CVE-2026-54293 nltk
RHAIENG-5895 CVE-2026-42266 jupyterlab
RHAIENG-5987 CVE-2026-12243 nltk
RHAIENG-6097 CVE-2026-42311 pillow
RHAIENG-6108 CVE-2026-54060 pillow
RHAIENG-6114 CVE-2026-55379 pillow
RHAIENG-6117 CVE-2026-55380 pillow
RHAIENG-6314 CVE-2026-49476 soupsieve
RHAIENG-6317 CVE-2026-49477 soupsieve
RHAIENG-6321 CVE-2026-54058 pillow
RHAIENG-6324 CVE-2026-54283 starlette
RHAIENG-6327 CVE-2026-59197 pillow
RHAIENG-6330 CVE-2026-59199 pillow
RHAIENG-6333 CVE-2026-59200 pillow
RHAIENG-6336 CVE-2026-59204 pillow
RHAIENG-6339 CVE-2026-59205 pillow
RHAIENG-6351 CVE-2026-59884 pyasn1
RHAIENG-6354 CVE-2026-59886 pyasn1
RHAIENG-6358 CVE-2026-12484 keras
RHAIENG-6370 CVE-2026-59885 pyasn1

Also covered by existing / bumped constraints: RHAIENG-4014 (nltk), RHAIENG-3795 (pyjwt), RHAIENG-3841 (pyasn1), RHAIENG-5355 (starlette).

Not fixed in this PR

RHAIENG CVE Package Reason
RHAIENG-5495 CVE-2026-5241 transformers OSV: affects 5.2.0 only; images pin 4.55.2 / 4.56.1
RHAIENG-5853 CVE-2026-48746 vllm Not present in any pylock.toml; defensive vllm>=0.22.0 constraint only

Package updates

cve-constraints.txt floors (before → after)

Package Constraint before Constraint after
nltk >=3.9.4 >=3.10.0
starlette >=1.0.1 >=1.3.1
pyjwt >=2.12.0 >=2.13.0
pyasn1 >=0.6.3 >=0.6.4
pillow >=12.2.0 >=12.3.0
cryptography >=46.0.7
jupyter-server >=2.20.0
jupyterlab >=4.5.7
urllib3 >=2.7.0
pip >=26.1.2
aiohttp >=3.14.0
mistune >=3.3.0
python-dotenv >=1.2.2
soupsieve >=2.8.4
ujson >=5.12.1
keras >=3.14.0
feast >=0.63.0
vllm >=0.22.0 (defensive)
onnx >=1.21.0 unchanged
tornado >=6.5.5 unchanged

Direct pyproject.toml pin changes

Package Before After Images
jupyterlab ==4.4.7 ==4.5.7 all Jupyter workbenches
jupyter-server ~=2.17.0 ~=2.20.0 all Jupyter workbenches
keras ~=3.13.2 >=3.14.0 tensorflow (+ rocm-tensorflow)
feast ~=0.54.1 >=0.63.0 datascience / pytorch / tensorflow / codeserver
urllib3 ~=2.6.0 >=2.7.0 all Elyra runtimes

Locked versions in pylock.toml (range across images, before → after)

Package Before (min … max) After (min … max)
aiohttp 3.13.3 … 3.14.1 3.14.1 … 3.14.3
cryptography 46.0.5 … 49.0.0 49.0.0
feast 0.54.1 0.65.0
jupyter-server 2.17.0 … 2.20.0 2.20.0
jupyterlab 4.4.7 … 4.6.1 4.5.7 … 4.6.2
keras 3.13.2 3.15.0
mistune 3.2.0 … 3.3.3 3.3.2 … 3.3.4
nltk 3.9.4 3.10.0
pillow 12.2.0 … 12.3.0 12.3.0
pyasn1 0.6.3 … 0.6.4 0.6.4
python-dotenv 1.2.1 … 1.2.2 1.2.2
soupsieve 2.8.1 … 2.8.4 2.8.4 … 2.9.1
starlette 1.2.0 … 1.3.1 1.3.1
ujson 5.11.0 … 5.13.0 5.13.0
urllib3 2.6.3 … 2.7.0 2.7.0
onnx 1.21.0 … 1.22.0 unchanged
pyjwt 2.13.0 unchanged
pip 26.1.2 unchanged
tornado 6.5.5 … 6.5.7 unchanged

Test plan

  • bash scripts/sync-python-lockfiles.sh — all 17 images compile
  • CI / Konflux builds on rhoai-2.25
  • make test (not run locally on this branch)

Summary by CodeRabbit

  • Updates
    • Upgraded Feast to 0.65.0 across supported Python/Jupyter environments.
    • Bumped JupyterLab to 4.6.2 and Jupyter Server to ~2.20.0; refreshed related notebook/image annotations and added Jupyter Builder where required.
    • Updated additional runtime packages (including Keras, Pillow, urllib3, NLTK, Prometheus Client, and pyasn1).
  • Security
    • Strengthened baseline dependency security by increasing minimum versions for CVE-impacted libraries (notably cryptography, urllib3, and Pillow) and expanding CVE constraint coverage.

Consolidate security floors in dependencies/cve-constraints.txt and bump
direct pins where resolver conflicts required pyproject changes (jupyterlab,
jupyter-server, keras, feast, urllib3). Regenerate all affected pylock.toml
files via scripts/sync-python-lockfiles.sh.

Co-authored-by: Cursor <cursoragent@cursor.com>
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Updated dependency constraints, CVE minimums, generated lockfiles, and notebook ImageStream metadata across codeserver, Jupyter, and runtime environments. Feast, JupyterLab, Jupyter Server, urllib3, and related packages were refreshed.

Changes

Dependency refresh

Layer / File(s) Summary
Constraint and CVE updates
dependencies/cve-constraints.txt, */pyproject.toml
Dependency floors, UV overrides, CVE constraints, and Feast, Jupyter, urllib3, Pillow, and Keras requirements were updated.
Generated lockfile refresh
*/pylock.toml, requirements.cpu.txt
Lockfiles and hashed requirements were regenerated with updated package artifacts and added jupyter-builder entries.
Notebook image dependency metadata
manifests/base/jupyter-*-imagestream.yaml
ImageStream annotations now report JupyterLab 4.6 and Feast 0.65 where applicable.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Possibly related PRs

Suggested labels: lgtm, approved, tide/merge-method-squash

Suggested reviewers: dibryant

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the PR’s main change: CVE-constraint-driven dependency updates for the rhoai-2.25 Python images.
Description check ✅ Passed The description covers the summary, affected packages, Jira links, and testing notes, though it is missing the exact template headings and checklist items.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/rhoai-2.25-open-pypi-cves

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

Superseded by newer run: https://github.com/red-hat-data-services/notebooks/actions/runs/30372390784

CI status [antigravity]

Run: Build Notebooks (push) #303717625342/2 complete · 2 skipped
Last updated: 2026-07-28T15:07:37Z

Workflow completed with skipped jobs.

@jiridanek

Copy link
Copy Markdown
Member Author

/build-konflux

Update notebook-python-dependencies for JupyterLab and Feast on the
recommended (-n) tags, and pin jupyterlab==4.6.2 in pyproject.toml to
match regenerated pylock.toml (CI test_image_pyprojects).

Co-authored-by: Cursor <cursoragent@cursor.com>
@jiridanek

Copy link
Copy Markdown
Member Author

/build-konflux

@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

Superseded by newer run: https://github.com/red-hat-data-services/notebooks/actions/runs/30371771614

CI status [antigravity]

Run: Build Notebooks (push) #303723907842/2 complete · 2 skipped
Last updated: 2026-07-28T15:15:00Z

Workflow completed with skipped jobs.

@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

Superseded by newer run: https://github.com/red-hat-data-services/notebooks/actions/runs/30374774664

CI status [antigravity]

Run: Build Notebooks (pr) #3037177161428/28 complete · 2 passed · 3 failed · 23 cancelled
Last updated: 2026-07-28T15:15:28Z after workflow update completed

Failures so far

Job Failed step Link
runtime-cuda-pytorch-llmcompressor-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make runtime-cuda-pytorch-llmcompressor-ubi9-python-3.12 logs
cuda-jupyter-pytorch-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make cuda-jupyter-pytorch-ubi9-python-3.12 logs
codeserver-ubi9-python-3.12 · linux/arm64 [rhoai] / build Build: make codeserver-ubi9-python-3.12 logs
runtime-cuda-tensorflow-ubi9-python-3.12 · linux/arm64 [rhoai] / build Build: make runtime-cuda-tensorflow-ubi9-python-3.12 logs
rocm-jupyter-tensorflow-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make rocm-jupyter-tensorflow-ubi9-python-3.12 logs
cuda-jupyter-tensorflow-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make cuda-jupyter-tensorflow-ubi9-python-3.12 logs
jupyter-trustyai-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make jupyter-trustyai-ubi9-python-3.12 logs
runtime-cuda-tensorflow-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make runtime-cuda-tensorflow-ubi9-python-3.12 logs

(+ 18 other failed matrix jobs)

Likely root causes

  • Codeserver build failure (Exit status 60 / SSL certificate verification): The codeserver-ubi9-python-3.12 job on linux/arm64 failed during install-xkbfile-from-generic.sh because curl failed to verify the legitimacy of the server (Error 60), likely due to network or certificate issues when fetching dependencies inside the container build process (codeserver/ubi9-python-3.12/Dockerfile.konflux.cpu).
  • Test failure (JupyterLab version mismatch): The cuda-jupyter-minimal-ubi9-python-3.12 job failed on a test assertion (AssertionError: '4.5' != '4.4') in test_jupyterlab_version, indicating an unexpected mismatch between the expected and actual installed JupyterLab versions (ci/cached-builds/make_test.py).
  • Job cancellations: The remaining failed jobs were cancelled as a result of earlier or concurrent failures in the workflow matrix.

Suggested next steps

  • Check network connectivity and SSL certificate configurations or mirroring settings used by curl within the codeserver container build patches.
  • Verify the pinned JupyterLab version requirements against the actual installed packages in the minimal Jupyter test environment.
  • Re-run the affected jobs once the underlying dependency or network issues are resolved.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@jupyter/tensorflow/ubi9-python-3.12/pyproject.toml`:
- Line 64: Remove the protobuf==6.31.1 pin from both TensorFlow environment
dependency lists, update each to a compatible patched protobuf release, and
regenerate the corresponding lockfiles for
jupyter/tensorflow/ubi9-python-3.12/pyproject.toml at lines 64-64 and
jupyter/rocm/tensorflow/ubi9-python-3.12/pyproject.toml at lines 66-66.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f36dab71-63c1-41ae-a49c-32b6b08a8a3b

📥 Commits

Reviewing files that changed from the base of the PR and between 8500837 and 4fac78b.

📒 Files selected for processing (33)
  • codeserver/ubi9-python-3.12/pylock.toml
  • codeserver/ubi9-python-3.12/pyproject.toml
  • dependencies/cve-constraints.txt
  • jupyter/datascience/ubi9-python-3.12/pylock.toml
  • jupyter/datascience/ubi9-python-3.12/pyproject.toml
  • jupyter/minimal/ubi9-python-3.12/pylock.toml
  • jupyter/minimal/ubi9-python-3.12/pyproject.toml
  • jupyter/pytorch+llmcompressor/ubi9-python-3.12/pylock.toml
  • jupyter/pytorch+llmcompressor/ubi9-python-3.12/pyproject.toml
  • jupyter/pytorch/ubi9-python-3.12/pylock.toml
  • jupyter/pytorch/ubi9-python-3.12/pyproject.toml
  • jupyter/rocm/pytorch/ubi9-python-3.12/pylock.toml
  • jupyter/rocm/pytorch/ubi9-python-3.12/pyproject.toml
  • jupyter/rocm/tensorflow/ubi9-python-3.12/pylock.toml
  • jupyter/rocm/tensorflow/ubi9-python-3.12/pyproject.toml
  • jupyter/tensorflow/ubi9-python-3.12/pylock.toml
  • jupyter/tensorflow/ubi9-python-3.12/pyproject.toml
  • jupyter/trustyai/ubi9-python-3.12/pylock.toml
  • jupyter/trustyai/ubi9-python-3.12/pyproject.toml
  • runtimes/datascience/ubi9-python-3.12/pylock.toml
  • runtimes/datascience/ubi9-python-3.12/pyproject.toml
  • runtimes/minimal/ubi9-python-3.12/pylock.toml
  • runtimes/minimal/ubi9-python-3.12/pyproject.toml
  • runtimes/pytorch+llmcompressor/ubi9-python-3.12/pylock.toml
  • runtimes/pytorch+llmcompressor/ubi9-python-3.12/pyproject.toml
  • runtimes/pytorch/ubi9-python-3.12/pylock.toml
  • runtimes/pytorch/ubi9-python-3.12/pyproject.toml
  • runtimes/rocm-pytorch/ubi9-python-3.12/pylock.toml
  • runtimes/rocm-pytorch/ubi9-python-3.12/pyproject.toml
  • runtimes/rocm-tensorflow/ubi9-python-3.12/pylock.toml
  • runtimes/rocm-tensorflow/ubi9-python-3.12/pyproject.toml
  • runtimes/tensorflow/ubi9-python-3.12/pylock.toml
  • runtimes/tensorflow/ubi9-python-3.12/pyproject.toml

"cryptography>=46.0.5",
"cryptography>=46.0.7",
# tf2onnx has pinned protobuf version, that causes conflict with other packages
"protobuf==6.31.1",

@coderabbitai coderabbitai Bot Jul 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== Relevant pyproject entries ==\n'
sed -n '50,80p' jupyter/tensorflow/ubi9-python-3.12/pyproject.toml | cat -n
printf '\n---\n'
sed -n '52,85p' jupyter/rocm/tensorflow/ubi9-python-3.12/pyproject.toml | cat -n

printf '\n== Search for protobuf constraints ==\n'
rg -n --hidden --glob '!*node_modules*' --glob '!*.git*' 'protobuf==|protobuf[<>=~! ]|tensorflow|tf2onnx' jupyter/tensorflow jupyter/rocm/tensorflow -g 'pyproject.toml' -g 'requirements*.txt' -g 'uv.lock' -g 'poetry.lock' -g 'lock*.toml'

Repository: red-hat-data-services/notebooks

Length of output: 2062


🌐 Web query:

GHSA-7gcm-g887-7qv7 protobuf 6.31.1 CVE-2026-0994 affected version range

💡 Result:

The vulnerability GHSA-7gcm-g887-7qv7, tracked as CVE-2026-0994, is a denial-of-service (DoS) issue in the pure-Python implementation of the protobuf library [1][2][3]. It occurs because the google.protobuf.json_format.ParseDict() function fails to properly track or enforce recursion depth when parsing nested google.protobuf.Any messages, allowing an attacker to trigger a RecursionError [1][3][4]. The affected version ranges for the protobuf Python package are [1][4]: - Versions >= 6.30.0rc1 and <= 6.33.4 [1][5] - Versions < 5.29.6 [1][5] The issue is patched in versions 6.33.5 and 5.29.6 [1][5]. While some security databases and automated scanners may report a broader "up to and including 33.4" range due to NVD enrichment processes, the official advisory clarifies that the vulnerability specifically affects the 5.x and 6.x release lines as defined above [1][5]. Note that this vulnerability only impacts the pure-Python implementation of the protobuf backend [2].

Citations:


Remove the protobuf==6.31.1 pin from both TensorFlow environments. It is still in GHSA-7gcm-g887-7qv7’s affected range; bump to a compatible patched protobuf release and regenerate both lockfiles.

  • jupyter/tensorflow/ubi9-python-3.12/pyproject.toml
  • jupyter/rocm/tensorflow/ubi9-python-3.12/pyproject.toml
📍 Affects 2 files
  • jupyter/tensorflow/ubi9-python-3.12/pyproject.toml#L64-L64 (this comment)
  • jupyter/rocm/tensorflow/ubi9-python-3.12/pyproject.toml#L66-L66
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@jupyter/tensorflow/ubi9-python-3.12/pyproject.toml` at line 64, Remove the
protobuf==6.31.1 pin from both TensorFlow environment dependency lists, update
each to a compatible patched protobuf release, and regenerate the corresponding
lockfiles for jupyter/tensorflow/ubi9-python-3.12/pyproject.toml at lines 64-64
and jupyter/rocm/tensorflow/ubi9-python-3.12/pyproject.toml at lines 66-66.

Source: Linters/SAST tools

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coderabbitai The protobuf==6.31.1 pin is pre-existing and unchanged by this PR — it is needed because tf2onnx has a pinned protobuf version that conflicts with other packages. Removing it is out of scope for this CVE remediation PR. The pin resolves a real dependency conflict and is not itself a security concern.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jiridanek, understood—the pin is pre-existing and changing it could disrupt the tf2onnx resolution, so it should not block this CVE-remediation PR.

That said, protobuf==6.31.1 remains within the advisory’s affected range, so I’ll keep the concern as a follow-up rather than withdraw it. Would you like me to open a GitHub issue to track finding a compatible tf2onnx/protobuf resolution and regenerating the TensorFlow lockfiles?


🧠 Learnings used
Learnt from: jiridanek
Repo: red-hat-data-services/notebooks PR: 2333
File: jupyter/rocm/pytorch/ubi9-python-3.12/uv.lock.d/pylock.rocm.toml:0-0
Timestamp: 2026-06-05T19:50:12.249Z
Learning: When generating pylock.toml files using `uv pip compile`, do not expect `[tool.uv].override-dependencies` from the image’s `pyproject.toml` to take effect. `uv pip compile` automatically reads only `[tool.uv].constraint-dependencies` from `pyproject.toml`.
- To apply dependency version overrides at `uv pip compile` time, pass an explicit `--override overrides.txt` argument.
- To enforce a CVE version floor during lock generation, add the floor constraints to `dependencies/cve-constraints.txt` (the repo’s constraint file already used via `--constraints`).
Note: `[tool.uv].override-dependencies` is honored only by `uv lock`, `uv sync`, and `uv run`, not by `uv pip compile`.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is needed because tf2onnx has a pinned protobuf version that conflicts with other packages

also because newer protobuf is buggy on big-endian, s390x

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jiridanek, thanks for the additional context. The pre-existing protobuf==6.31.1 pin is also protecting s390x/big-endian compatibility, in addition to resolving the tf2onnx dependency conflict. I’ll treat any protobuf security upgrade as a separate, architecture-aware follow-up—not a required change for this PR.

🐇🔎


✏️ Learnings added
Learnt from: jiridanek
Repo: red-hat-data-services/notebooks PR: 2634
File: jupyter/tensorflow/ubi9-python-3.12/pyproject.toml:64-64
Timestamp: 2026-07-29T08:11:30.491Z
Learning: In the TensorFlow notebook environments, the pre-existing `protobuf==6.31.1` override is required both to resolve a real `tf2onnx` protobuf dependency conflict and because newer protobuf versions are buggy on big-endian s390x. Do not recommend removing or upgrading this pin as part of unrelated dependency/CVE refreshes without separately validating tf2onnx compatibility and s390x behavior.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.


🧠 Learnings used
Learnt from: jiridanek
Repo: red-hat-data-services/notebooks PR: 2333
File: jupyter/rocm/pytorch/ubi9-python-3.12/uv.lock.d/pylock.rocm.toml:0-0
Timestamp: 2026-06-05T19:50:12.249Z
Learning: When generating pylock.toml files using `uv pip compile`, do not expect `[tool.uv].override-dependencies` from the image’s `pyproject.toml` to take effect. `uv pip compile` automatically reads only `[tool.uv].constraint-dependencies` from `pyproject.toml`.
- To apply dependency version overrides at `uv pip compile` time, pass an explicit `--override overrides.txt` argument.
- To enforce a CVE version floor during lock generation, add the floor constraints to `dependencies/cve-constraints.txt` (the repo’s constraint file already used via `--constraints`).
Note: `[tool.uv].override-dependencies` is honored only by `uv lock`, `uv sync`, and `uv run`, not by `uv pip compile`.

Bump minimal to jupyterlab==4.6.2 with regenerated pylock, set minimal
ImageStream JupyterLab pins to 4.6, and refresh codeserver pylock.cpu.toml
so feast 0.65.0 matches pyproject (test_image_pyprojects).

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

Superseded by newer run: https://github.com/red-hat-data-services/notebooks/actions/runs/30372396750

CI status [antigravity]

Run: Build Notebooks (push) #303747746642/2 complete · 2 skipped
Last updated: 2026-07-28T15:43:47Z

Workflow completed with skipped jobs.

@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

Superseded by newer run: https://github.com/red-hat-data-services/notebooks/actions/runs/30374779314

CI status [antigravity]

Run: Build Notebooks (pr) #3037239675028/28 complete · 19 passed · 4 failed · 5 cancelled
Last updated: 2026-07-28T15:44:16Z after workflow update completed

Failures so far

Job Failed step Link
jupyter-minimal-ubi9-python-3.12 · linux/s390x [rhoai] / build Run Testcontainers container tests (in PyTest) logs
jupyter-datascience-ubi9-python-3.12 · linux/ppc64le [rhoai] / build Build: make jupyter-datascience-ubi9-python-3.12 logs
rocm-runtime-pytorch-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make rocm-runtime-pytorch-ubi9-python-3.12 logs
jupyter-minimal-ubi9-python-3.12 · linux/ppc64le [rhoai] / build Run Testcontainers container tests (in PyTest) logs
codeserver-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make codeserver-ubi9-python-3.12 logs
jupyter-trustyai-ubi9-python-3.12 · linux/amd64 [rhoai] / build Run image tests logs
codeserver-ubi9-python-3.12 · linux/arm64 [rhoai] / build Build: make codeserver-ubi9-python-3.12 logs
runtime-datascience-ubi9-python-3.12 · linux/s390x [rhoai] / build Build: make runtime-datascience-ubi9-python-3.12 logs

(+ 1 other failed matrix jobs)

Likely root causes

  • Disk Space Exhaustion (no space left on device): Several build jobs (e.g., rocm-runtime-pytorch-ubi9-python-3.12, jupyter-datascience-ubi9-python-3.12, rocm-jupyter-pytorch-ubi9-python-3.12) failed with write /var/tmp/container_images_storage...: no space left on device (Exit Code 125) while copying layers or installing heavy python packages via uv pip install into container storage.
  • TLS/Certificate Verification Failure (Exit Code 60): CodeServer builds (codeserver-ubi9-python-3.12 on linux/amd64 and linux/arm64) failed during install-xkbfile-from-generic.sh because curl failed to verify the server certificate (curl: (60) SSL certificate problem: unable to get local issuer certificate), likely due to unmounted/unregistered repository entitlements or missing CA bundles within the unprivileged container build sandbox.
  • Job Cancellation: Several parallel or downstream build/test jobs were automatically cancelled (The operation was canceled) following failures or timeouts across the matrix.

Suggested next steps

  • Free up disk space or increase storage capacity on CI runners to accommodate large multi-architecture container layer writes and intermediate storage caches during image builds.
  • Investigate network/CA certificate configuration in CodeServer container build steps where curl fetches external binaries.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@jupyter/minimal/ubi9-python-3.12/pylock.toml`:
- Around line 448-454: Remove the redundant consecutive [[packages]] declaration
immediately before the jupyter-builder package entry, keeping the table that
contains name, version, marker, sdist, and wheels intact. Regenerate the
lockfile if needed so each package table has complete metadata and no empty
package entry remains.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d0eee435-bc28-41e3-adc6-9d7f3ab46685

📥 Commits

Reviewing files that changed from the base of the PR and between 9359437 and 960ac94.

📒 Files selected for processing (7)
  • codeserver/ubi9-python-3.12/requirements.cpu.txt
  • codeserver/ubi9-python-3.12/uv.lock.d/pylock.cpu.toml
  • jupyter/minimal/ubi9-python-3.12/pylock.toml
  • jupyter/minimal/ubi9-python-3.12/pyproject.toml
  • manifests/base/jupyter-minimal-gpu-notebook-imagestream.yaml
  • manifests/base/jupyter-minimal-notebook-imagestream.yaml
  • manifests/base/jupyter-rocm-minimal-notebook-imagestream.yaml
🚧 Files skipped from review as they are similar to previous changes (2)
  • manifests/base/jupyter-rocm-minimal-notebook-imagestream.yaml
  • jupyter/minimal/ubi9-python-3.12/pyproject.toml

Comment thread jupyter/minimal/ubi9-python-3.12/pylock.toml
@jiridanek

Copy link
Copy Markdown
Member Author

/build-konflux

@github-actions

Copy link
Copy Markdown

CI status [antigravity]

Run: Build Notebooks (pr) #3037477931428/28 complete · 20 passed · 6 failed · 2 cancelled
Last updated: 2026-07-28T21:45:02Z after workflow update completed

Failures so far

Job Failed step Link
jupyter-minimal-ubi9-python-3.12 · linux/ppc64le [rhoai] / build Run image tests logs
rocm-jupyter-pytorch-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make rocm-jupyter-pytorch-ubi9-python-3.12 logs
jupyter-datascience-ubi9-python-3.12 · linux/ppc64le [rhoai] / build Build: make jupyter-datascience-ubi9-python-3.12 logs
jupyter-minimal-ubi9-python-3.12 · linux/s390x [rhoai] / build Run OpenShift container tests (in PyTest) logs
codeserver-ubi9-python-3.12 · linux/arm64 [rhoai] / build Build: make codeserver-ubi9-python-3.12 logs
codeserver-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make codeserver-ubi9-python-3.12 logs
runtime-datascience-ubi9-python-3.12 · linux/s390x [rhoai] / build Build: make runtime-datascience-ubi9-python-3.12 logs
rocm-runtime-pytorch-ubi9-python-3.12 · linux/amd64 [rhoai] / build Build: make rocm-runtime-pytorch-ubi9-python-3.12 logs

Likely root causes

  • Disk exhaustion: Several build jobs (e.g., rocm-jupyter-pytorch-ubi9-python-3.12, jupyter-datascience-ubi9-python-3.12, and rocm-runtime-pytorch-ubi9-python-3.12) failed with no space left on device or error code 125 during container image layering/storing (write /var/tmp/container_images_storage...: no space left on device).
  • Network / SSL handshake verification failure: Code-server builds (e.g., codeserver-ubi9-python-3.12 on linux/arm64 and linux/amd64) failed with exit status 60 (curl failed to verify the legitimacy of the server and therefore could not establish a secure connection) while running install-xkbfile-from-generic.sh.
  • Kubernetes connection timeout: The jupyter-minimal-ubi9-python-3.12 · linux/s390x test job failed with a WaitError: Timeout after 30 s waiting for Connecting to pod succeeds due to an underlying RemoteDisconnected socket exception during the OpenShift container test (tests/containers/workbenches/workbench_image_test.py).
  • Container execution/missing container error: The jupyter-minimal-ubi9-python-3.12 · linux/ppc64le image tests failed with error: unable to upgrade connection: container not found ("notebook") when attempting to copy test files and run papermill via kubectl.

Suggested next steps

  • Free up disk space: Check runner disk allocation or prune unused container build caches and temporary files under /var/tmp before starting large image builds.
  • Inspect network/certificate configuration: Verify CA certificates or network policies in the environment to resolve SSL verification issues (exit status 60) during curl downloads in code-server builds.
  • Check OpenShift cluster health and pod readiness: Investigate pod startup issues and liveness/readiness probe configurations for architectures like s390x and ppc64le to avoid container connection timeouts.

@jiridanek

Copy link
Copy Markdown
Member Author

/build-minimal-rocm

@jiridanek

Copy link
Copy Markdown
Member Author

/build-pytorch-rocm

@jiridanek

Copy link
Copy Markdown
Member Author

/build-tensorflow-rocm

@jiridanek

Copy link
Copy Markdown
Member Author

/build-runtime-pytorch-rocm

@jiridanek

Copy link
Copy Markdown
Member Author

/build-runtime-tensorflow-rocm

@jiridanek

Copy link
Copy Markdown
Member Author

/build-codeserver

@jiridanek

Copy link
Copy Markdown
Member Author

Live log inspection on all 5 ROCm build-images pods:

  • Image build is done on all five (step-buildEnd remote, images in buildah images).
  • minimal-rocm & tensorflow-rocm — push finished; Syft SBOM + upload in progress.
  • pytorch-rocm & runtime-pytorch-rocm — still pushing ~38–39 GB images to Quay.
  • runtime-tensorflow-rocm — furthest along: build-images complete (push + SBOM done), now in post-build scans (clair-scan, fips-check, …).

GitHub still shows those five as pending until push/SBOM/scans finish. Codeserver remains the only Konflux fail. For merge purposes, ROCm container builds look good; you're waiting on registry push and scan gates, not Dockerfile compilation.

@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ide-developer
Once this PR has been reviewed and has the lgtm label, please assign jesuino for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jiridanek jiridanek left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

Batch CVE remediation for 42 open [rhoai-2.25] pypi trackers — well-structured and thorough.

What this PR does

  • Adds/bumps 20 entries in dependencies/cve-constraints.txt to establish CVE-safe version floors
  • Updates direct pins in pyproject.toml across all 17 images (jupyter, codeserver, runtimes) for feast, jupyterlab, jupyter-server, keras, urllib3, pillow, cryptography
  • Regenerates pylock.toml and requirements.cpu.txt lock files via sync-python-lockfiles.sh
  • Updates ImageStream manifests to reflect new JupyterLab (4.4→4.6) and Feast (0.54→0.65) versions

Observations

Consistency — Version bumps are applied uniformly across all relevant images. The jupyterlab>=4.5.7 override is correctly added to all Elyra-using images where odh-elyra<4.5 would otherwise cap it. The pillow>=12.3.0 override is added to pytorch+llmcompressor where llmcompressor caps pillow<=12.2.0.

Specifier style change — Several packages switch from compatibility-release (~=) to minimum-floor (>=) specifiers (feast, keras, urllib3 in runtimes). This is intentional for CVE-floor semantics and locks still pin exact versions — just note that future sync-python-lockfiles.sh runs may pull newer minor/major versions.

Stale comment (nit) — The pyarrow override in {jupyter,runtimes}/pytorch+llmcompressor still references feast~=0.54.1 caps pyarrow<=17. With feast now at >=0.63.0 (locked 0.65.0), this comment is outdated. The override itself may still be needed (RHAIIS constraints), but the rationale text should be updated.

Missing trailing newline (pre-existing) — Several pyproject.toml files (datascience, pytorch, rocm-pytorch, rocm-tensorflow) lack a trailing newline. This is pre-existing, not introduced here, but worth a follow-up.

cve-constraints.txt EOF — The old file lacked a trailing newline; this PR fixes that. Good.

Defensive vllm constraintvllm>=0.22.0 in cve-constraints is clearly documented as defensive (not in any pylock). Sensible forward-looking measure.

Feast major version jump — 0.54.1→0.65.0 is a large jump. The PR description and lock files confirm all 17 images compile successfully. CI validation will be the key gate here.

Verdict

LGTM. The changes are mechanical, well-documented in the PR description, and internally consistent. The feast jump is the biggest risk factor — CI results will confirm.

@jiridanek

jiridanek commented Jul 29, 2026

Copy link
Copy Markdown
Member Author

I've posted about feast jump to feast channel, @jyejare and others can comment there; we need to resolve cves, so There Is No Other Way Alternative

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants