From 4c883333e4080bc9b238757863724a5b6fcf201a Mon Sep 17 00:00:00 2001 From: Nils Mittler Date: Tue, 24 Jun 2025 16:38:24 +0200 Subject: [PATCH 1/8] Add basic framework for jupyter-based teaching pathway --- learning-pathways/jupyter-based-teaching.md | 38 +++++++++++++++++++ .../jbt-customization-1/tutorial.bib | 0 .../tutorials/jbt-customization-1/tutorial.md | 20 ++++++++++ .../jbt-customization-2/tutorial.bib | 0 .../tutorials/jbt-customization-2/tutorial.md | 20 ++++++++++ .../tutorials/jbt-featured/tutorial.bib | 0 .../tutorials/jbt-featured/tutorial.md | 20 ++++++++++ .../teaching/tutorials/jbt-intro/tutorial.bib | 0 .../teaching/tutorials/jbt-intro/tutorial.md | 20 ++++++++++ 9 files changed, 118 insertions(+) create mode 100644 learning-pathways/jupyter-based-teaching.md create mode 100644 topics/teaching/tutorials/jbt-customization-1/tutorial.bib create mode 100644 topics/teaching/tutorials/jbt-customization-1/tutorial.md create mode 100644 topics/teaching/tutorials/jbt-customization-2/tutorial.bib create mode 100644 topics/teaching/tutorials/jbt-customization-2/tutorial.md create mode 100644 topics/teaching/tutorials/jbt-featured/tutorial.bib create mode 100644 topics/teaching/tutorials/jbt-featured/tutorial.md create mode 100644 topics/teaching/tutorials/jbt-intro/tutorial.bib create mode 100644 topics/teaching/tutorials/jbt-intro/tutorial.md diff --git a/learning-pathways/jupyter-based-teaching.md b/learning-pathways/jupyter-based-teaching.md new file mode 100644 index 00000000000000..54219fead7b849 --- /dev/null +++ b/learning-pathways/jupyter-based-teaching.md @@ -0,0 +1,38 @@ +--- +layout: learning-pathway +title: Jupyter-based teaching +type: use +description: | + An introduction to Jupyter-based teaching and Jupyter customization. + +editorial_board: +- mittler-works +funding: +- deKCD + +tags: [jupyter, teaching] + +pathway: + - section: "Part One: Introduction" + description: Advantages and limitations of Jupyter-based teaching + tutorials: + - topic: teaching + name: jbt-intro + + - section: "Part Two: Customization" + description: Tweak jupyter to your specific needs + tutorials: + - topic: teaching + name: jbt-customization-1 + - topic: teaching + name: jbt-customization-2 + + - section: "Part Three: Featured customizations" + description: Battle-tested and recommended customizations for teaching + tutorials: + - topic: teaching + name: jbt-featured + +--- + +You'll learn about the advantages of using jupyter for teaching and how you can adapt jupyter to the very specific needs of your course or workshop. diff --git a/topics/teaching/tutorials/jbt-customization-1/tutorial.bib b/topics/teaching/tutorials/jbt-customization-1/tutorial.bib new file mode 100644 index 00000000000000..e69de29bb2d1d6 diff --git a/topics/teaching/tutorials/jbt-customization-1/tutorial.md b/topics/teaching/tutorials/jbt-customization-1/tutorial.md new file mode 100644 index 00000000000000..9f7c20a353ba53 --- /dev/null +++ b/topics/teaching/tutorials/jbt-customization-1/tutorial.md @@ -0,0 +1,20 @@ +--- +layout: tutorial_hands_on + +title: An introduction to jupyter customization +subtopic: practises +draft: true +time_estimation: 1h +questions: + - How can I make basic customizations to jupyter? +objectives: + - An introduction is provided on how to make basic jupyter customizations, e.g. installing additional plugins. +key_points: + - Basic customization costs litte effort and makes it easier for your course participants to get started. +contributors: + - mittler-works +--- + +# Introduction + +(WIP) \ No newline at end of file diff --git a/topics/teaching/tutorials/jbt-customization-2/tutorial.bib b/topics/teaching/tutorials/jbt-customization-2/tutorial.bib new file mode 100644 index 00000000000000..e69de29bb2d1d6 diff --git a/topics/teaching/tutorials/jbt-customization-2/tutorial.md b/topics/teaching/tutorials/jbt-customization-2/tutorial.md new file mode 100644 index 00000000000000..4cd8e49e5cbc06 --- /dev/null +++ b/topics/teaching/tutorials/jbt-customization-2/tutorial.md @@ -0,0 +1,20 @@ +--- +layout: tutorial_hands_on + +title: Advanced jupyter customization +subtopic: practises +draft: true +time_estimation: 1h +questions: + - How can I do more with jupyter? +objectives: + - A guide to advanced jupyter customization is provided, e.g. how to install proxy applications. +key_points: + - Advanced customitzations may be time-consuming, but offer great potential and reusability value. +contributors: + - mittler-works +--- + +# Introduction + +(WIP) \ No newline at end of file diff --git a/topics/teaching/tutorials/jbt-featured/tutorial.bib b/topics/teaching/tutorials/jbt-featured/tutorial.bib new file mode 100644 index 00000000000000..e69de29bb2d1d6 diff --git a/topics/teaching/tutorials/jbt-featured/tutorial.md b/topics/teaching/tutorials/jbt-featured/tutorial.md new file mode 100644 index 00000000000000..f7f13110adee0f --- /dev/null +++ b/topics/teaching/tutorials/jbt-featured/tutorial.md @@ -0,0 +1,20 @@ +--- +layout: tutorial_hands_on + +title: Featured jupyter customizations +subtopic: practises +draft: true +time_estimation: 1h +questions: + - Which customizations are battle-tested and recommended for teaching? +objectives: + - An overview about customizations used by university of giessen. +key_points: + - Don't reinvent the wheel. +contributors: + - mittler-works +--- + +# Introduction + +(WIP) \ No newline at end of file diff --git a/topics/teaching/tutorials/jbt-intro/tutorial.bib b/topics/teaching/tutorials/jbt-intro/tutorial.bib new file mode 100644 index 00000000000000..e69de29bb2d1d6 diff --git a/topics/teaching/tutorials/jbt-intro/tutorial.md b/topics/teaching/tutorials/jbt-intro/tutorial.md new file mode 100644 index 00000000000000..2f765297ecfbc7 --- /dev/null +++ b/topics/teaching/tutorials/jbt-intro/tutorial.md @@ -0,0 +1,20 @@ +--- +layout: tutorial_hands_on + +title: An introduction to jupyter based teaching +subtopic: practises +draft: true +time_estimation: 1h +questions: + - What are the general advantages and limitations of jupyter based teaching? +objectives: + - An overview is provided to familiarize teachers with the advantages and limitations of jupyter-based teaching. +key_points: + - Jupyter-based teaching offers an easy entrypoint for students, but deprives them of the need to learn basic setup steps themselves. +contributors: + - mittler-works +--- + +# Introduction + +(WIP) \ No newline at end of file From 2e27c241127ed06915fa7ef3f769da1ca833cf63 Mon Sep 17 00:00:00 2001 From: Nils Mittler Date: Wed, 25 Jun 2025 16:16:46 +0200 Subject: [PATCH 2/8] Include more structure and start with bullet points --- learning-pathways/jupyter-based-teaching.md | 26 +++++--- .../tutorials/jbt-customization-1/tutorial.md | 60 +++++++++++++++++-- .../tutorials/jbt-customization-2/tutorial.md | 50 ++++++++++++++-- .../tutorials/jbt-featured/tutorial.md | 28 ++++++++- .../tutorials/jbt-galaxy/tutorial.bib | 0 .../teaching/tutorials/jbt-galaxy/tutorial.md | 24 ++++++++ .../teaching/tutorials/jbt-intro/tutorial.md | 40 +++++++++++-- .../teaching/tutorials/jbt-jhaas/tutorial.bib | 0 .../teaching/tutorials/jbt-jhaas/tutorial.md | 40 +++++++++++++ 9 files changed, 245 insertions(+), 23 deletions(-) create mode 100644 topics/teaching/tutorials/jbt-galaxy/tutorial.bib create mode 100644 topics/teaching/tutorials/jbt-galaxy/tutorial.md create mode 100644 topics/teaching/tutorials/jbt-jhaas/tutorial.bib create mode 100644 topics/teaching/tutorials/jbt-jhaas/tutorial.md diff --git a/learning-pathways/jupyter-based-teaching.md b/learning-pathways/jupyter-based-teaching.md index 54219fead7b849..affad9b464c0fd 100644 --- a/learning-pathways/jupyter-based-teaching.md +++ b/learning-pathways/jupyter-based-teaching.md @@ -13,26 +13,38 @@ funding: tags: [jupyter, teaching] pathway: - - section: "Part One: Introduction" - description: Advantages and limitations of Jupyter-based teaching + - section: "Learn the basics about Jupyter-based teaching" + description: "Advantages and limitations of Jupyter-based teaching" tutorials: - topic: teaching name: jbt-intro - - section: "Part Two: Customization" - description: Tweak jupyter to your specific needs + - section: "Customize Jupyter to your needs" + description: "Adapt Jupyter to your course or workshop specific needs" tutorials: - topic: teaching name: jbt-customization-1 - topic: teaching name: jbt-customization-2 - - section: "Part Three: Featured customizations" - description: Battle-tested and recommended customizations for teaching + - section: "Featured customizations" + description: "Battle-tested and recommended Jupyter extensions for teaching " tutorials: - topic: teaching name: jbt-featured + - section: "For teachers: Use custom Jupyter Images with JHaaS" + description: "Learn how to teach a course or workshop with your Jupyter customizations on JHaaS" + tutorials: + - topic: teaching + name: jbt-jhaas + + - section: "For admins: Provide your Jupyter Images with Galaxy" + description: "Learn how to provide your custom Jupyter customizations on your Galaxy instance" + tutorials: + - topic: teaching + name: jbt-galaxy + --- -You'll learn about the advantages of using jupyter for teaching and how you can adapt jupyter to the very specific needs of your course or workshop. +You'll learn about the advantages of using Jupyter for teaching and how you can adapt Jupyter to the very specific needs of your course or workshop. As a teacher, you'll learn how to teach a course or workshop using your Jupyter customizations. As a Galaxy admin, you'll learn how to provide your users with a customized Jupyter. diff --git a/topics/teaching/tutorials/jbt-customization-1/tutorial.md b/topics/teaching/tutorials/jbt-customization-1/tutorial.md index 9f7c20a353ba53..5c3194347fdd25 100644 --- a/topics/teaching/tutorials/jbt-customization-1/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-1/tutorial.md @@ -1,20 +1,70 @@ --- layout: tutorial_hands_on -title: An introduction to jupyter customization +title: Basics of Jupyter customization subtopic: practises draft: true time_estimation: 1h questions: - - How can I make basic customizations to jupyter? + - How can I make basic customizations to Jupyter? objectives: - - An introduction is provided on how to make basic jupyter customizations, e.g. installing additional plugins. + - An introduction is provided on how to make basic Jupyter customizations, e.g. installing additional plugins. key_points: - Basic customization costs litte effort and makes it easier for your course participants to get started. contributors: - mittler-works --- -# Introduction +# Prerequisites -(WIP) \ No newline at end of file +* We'll use containerized installation + +* Examples with docker, as it is easy to install on all major platforms + +# Start your first local Notebook Server with docker + +* Show docker stacks + +* Select an image + +* WIP + +## Access your local Notebook Server + +* Windows and Mac users will need verification work... + +* WIP + +## Missing packages + +* Install a simple python package + +* WIP + +# Docker Compose (?) + +* Maybe introduce compose to reduce complexity of terminal interaction + +# Build your first custom Image + +* WIP + +# Use your first custom Image + +* WIP + +## Access your local custom Notebook Server + +* WIP + +## See your custom changes + +* WIP + +# Extend your custom Image + +* Install a simple plugin like `igv-notebook` + +* WIP + +# (WIP) \ No newline at end of file diff --git a/topics/teaching/tutorials/jbt-customization-2/tutorial.md b/topics/teaching/tutorials/jbt-customization-2/tutorial.md index 4cd8e49e5cbc06..fb9d8d10fe8d3b 100644 --- a/topics/teaching/tutorials/jbt-customization-2/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-2/tutorial.md @@ -1,14 +1,14 @@ --- layout: tutorial_hands_on -title: Advanced jupyter customization +title: Advanced Jupyter customization subtopic: practises draft: true time_estimation: 1h questions: - - How can I do more with jupyter? + - How can I tailor Jupyter to my specific teaching needs? objectives: - - A guide to advanced jupyter customization is provided, e.g. how to install proxy applications. + - A guide to advanced Jupyter customization is provided, e.g. how to install proxy applications. key_points: - Advanced customitzations may be time-consuming, but offer great potential and reusability value. contributors: @@ -17,4 +17,46 @@ contributors: # Introduction -(WIP) \ No newline at end of file +Now we know how to include changes, we will build something more complex and look at some pitfalls. + +# Install an additional Kernel + +* e.g. rust kernel, or R + +* WIP + +# Config Changes + +* Where to put them + +* What to put there + +* WIP + +# Persistent Data + +* HomeDir will be overlayed with a mounted Volume + +* WIP + +## External persistent Data + +* WIP + +## Large Data + +* Don't include it into your Image! + +* WIP + +## Sensitive Data (?) + +* Maybe WIP, but maybe put that into `jbt-intro` + +# Application Proxies + +* E.G. installing `code-server` + +# (WIP) + +* WIP \ No newline at end of file diff --git a/topics/teaching/tutorials/jbt-featured/tutorial.md b/topics/teaching/tutorials/jbt-featured/tutorial.md index f7f13110adee0f..3a6c7b8b7d58a8 100644 --- a/topics/teaching/tutorials/jbt-featured/tutorial.md +++ b/topics/teaching/tutorials/jbt-featured/tutorial.md @@ -1,7 +1,7 @@ --- layout: tutorial_hands_on -title: Featured jupyter customizations +title: Featured Jupyter customizations subtopic: practises draft: true time_estimation: 1h @@ -17,4 +17,30 @@ contributors: # Introduction +We present adaptions that the JLU Physics Department has been successfully using in its courses for over a year and consider to be a great addition to the default Jupyter notebook server. + +# nbgitpuller + +## Install + +## Configuration + +## Usage + +# nbgrader + +## Install + +## Configuration + +## Usage + +# Config customizations + +## Dos and Don'ts + +# Setup Scripts + +## Where to place them what to do with them + (WIP) \ No newline at end of file diff --git a/topics/teaching/tutorials/jbt-galaxy/tutorial.bib b/topics/teaching/tutorials/jbt-galaxy/tutorial.bib new file mode 100644 index 00000000000000..e69de29bb2d1d6 diff --git a/topics/teaching/tutorials/jbt-galaxy/tutorial.md b/topics/teaching/tutorials/jbt-galaxy/tutorial.md new file mode 100644 index 00000000000000..7ed28fb25f41b7 --- /dev/null +++ b/topics/teaching/tutorials/jbt-galaxy/tutorial.md @@ -0,0 +1,24 @@ +--- +layout: tutorial_hands_on + +title: Provide your custom Jupyter Image via Galaxy +subtopic: practises +draft: true +time_estimation: 1h +questions: + - As a Galaxy admin, how can I provide my users with a custom Notebook Server? +objectives: + - Learn how to add additional interactive tools for each Jupyter customization. +key_points: + - Each user-base has different needs. It's easy to serve them providing customized Jupyter Notebook Servers. +contributors: + - mittler-works +--- + +# Introduction + +(WIP) + +# Create a new interactive tool definition (xml) + +# Include your new interactive tool definition into your galaxy installation \ No newline at end of file diff --git a/topics/teaching/tutorials/jbt-intro/tutorial.md b/topics/teaching/tutorials/jbt-intro/tutorial.md index 2f765297ecfbc7..1432df11b575ca 100644 --- a/topics/teaching/tutorials/jbt-intro/tutorial.md +++ b/topics/teaching/tutorials/jbt-intro/tutorial.md @@ -1,20 +1,48 @@ --- layout: tutorial_hands_on -title: An introduction to jupyter based teaching +title: An introduction to Jupyter based teaching subtopic: practises draft: true -time_estimation: 1h +time_estimation: 30m questions: - - What are the general advantages and limitations of jupyter based teaching? + - What are the general advantages and limitations of Jupyter based teaching? objectives: - - An overview is provided to familiarize teachers with the advantages and limitations of jupyter-based teaching. + - An overview is provided to familiarize teachers with the advantages and limitations of Jupyter-based teaching. key_points: - Jupyter-based teaching offers an easy entrypoint for students, but deprives them of the need to learn basic setup steps themselves. contributors: - mittler-works --- -# Introduction +# Glossary -(WIP) \ No newline at end of file +* Jupyter + +* Jupyter Notebook + +* Jupyter Notebook Server + +* JupyterLab + +* JupyterHub + +# Advantages for teaching + +* Predefined Environment + +* Web-based + +* BYD + +* Reproducability + +# Disadvantages for teaching + +* Important first steps are not need to be taught + +* Habituation effects + +* Jail effects + +# (WIP) \ No newline at end of file diff --git a/topics/teaching/tutorials/jbt-jhaas/tutorial.bib b/topics/teaching/tutorials/jbt-jhaas/tutorial.bib new file mode 100644 index 00000000000000..e69de29bb2d1d6 diff --git a/topics/teaching/tutorials/jbt-jhaas/tutorial.md b/topics/teaching/tutorials/jbt-jhaas/tutorial.md new file mode 100644 index 00000000000000..f063903631fb0c --- /dev/null +++ b/topics/teaching/tutorials/jbt-jhaas/tutorial.md @@ -0,0 +1,40 @@ +--- +layout: tutorial_hands_on + +title: Teaching a course with custom Jupyter via JHaaS +subtopic: practises +draft: true +time_estimation: 1h +questions: + - How can I use my Jupyter customizations in a course relevant matter? +objectives: + - Learn how to teach a course via JHaaS using your Jupyter customizations. +key_points: + - Don't worry about infrastructure and service deployment, use JHaaS. +contributors: + - mittler-works +--- + +# Getting Started with JHaaS + +## Authenticate + +## Apply for Teacher role + +# Make your customizations available + +* e.g. docker hub + +# Create a new JupyterHub Request + +## Use your custom image + +# Manage your participants + +## Invitations + +## Applications + +# Teach your course + +(WIP) From 83a54ae92da82ec3a7c6321b8767dcfe6da6aeb6 Mon Sep 17 00:00:00 2001 From: Nils Mittler Date: Thu, 26 Jun 2025 23:08:07 +0200 Subject: [PATCH 3/8] Add basic text, exercises and dockerfile examples --- .../tutorials/jbt-customization-1/tutorial.md | 156 ++++++++++++++++-- .../tutorials/jbt-customization-2/tutorial.md | 135 ++++++++++++++- .../teaching/tutorials/jbt-galaxy/tutorial.md | 29 +++- .../teaching/tutorials/jbt-intro/tutorial.md | 53 ++++-- 4 files changed, 340 insertions(+), 33 deletions(-) diff --git a/topics/teaching/tutorials/jbt-customization-1/tutorial.md b/topics/teaching/tutorials/jbt-customization-1/tutorial.md index 5c3194347fdd25..57420c1fd8ef09 100644 --- a/topics/teaching/tutorials/jbt-customization-1/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-1/tutorial.md @@ -29,6 +29,12 @@ contributors: * WIP +```bash +sudo docker run --rm -p 127.0.0.1:8888:8888 quay.io/jupyter/minimal-notebook:2025-06-23 +``` + +* Will print out a link to `http://localhost:8888/lab?token=` + ## Access your local Notebook Server * Windows and Mac users will need verification work... @@ -41,30 +47,152 @@ contributors: * WIP -# Docker Compose (?) +* You are now in a predefined environment. All Packages from mentioned in the docker stacks documentation are there, so you can easily open the `Python 3` Notebook presented on the start page and start working. As you are in a minimal notebook you just do have the python default libraries, but you could run this to get the current weekday: -* Maybe introduce compose to reduce complexity of terminal interaction +```python +import datetime +now = datetime.datetime.now() +print(now.strftime("%A")) +``` -# Build your first custom Image +* For scientific work you will likely need more than just the default python libraries. Reviewing the docker stacks is helpful and maybe an image like the `scipy-notebook` will have most of the packages you need. But of course these default collections cannot have suitable package selections for the very specific needs of all courses out in the world. -* WIP +For example, in this minimal setup you might like to have the option to print out text in a fancier way. Luckily, there is a python package for this: `cowsay`. So let's go ahead, try running -# Use your first custom Image +```python +from cowsay import cow +cow('Muuuuh') +``` -* WIP +As you will see, this fails as cowsay is not available in minimal-notebook. -## Access your local custom Notebook Server -* WIP +# Build your first custom Image -## See your custom changes +In order to customize this Container Image, we need a so-called `Dockerfile`. Go ahead, create a new directory as your playground. Then, create a file called `Dockerfile` in this new directory. -* WIP +Fill it with content: -# Extend your custom Image +```dockerfile +# First, we say from which container image we want to start +# This is called the base image +FROM quay.io/jupyter/minimal-notebook:2025-06-23 -* Install a simple plugin like `igv-notebook` +# Now, we can run arbitrary commands in order to extend the base image +RUN pip install --quiet --no-cache-dir cowsay +``` -* WIP +* Build it, mind the period at the end of the command + +```bash +sudo docker build -t my-custom-jupyterlab . +``` + + +# Use your first custom Image -# (WIP) \ No newline at end of file +```bash +sudo docker run --rm -p 127.0.0.1:8888:8888 my-custom-jupyterlab +``` + +Again, you'll provided with a link to open you jupyterlab. Start a Pyhton 3 Notebook by clicking the tile. + +Again, run: + +```python +from cowsay import cow +cow('Muuuuh') +``` + +As you will see, the command now succeeds and prints a nice graphical text output. + +# Exercise: build your own custom Image + +Now, let's go ahead and try out some real usecase. + +> Create a custom notebook from the `scipy-notebook` base image and install the `igv-browser` extension. + +You succeeded, if you can run following snippet successfully and are provided with a genome browser with loaded track: + +> Make this script succeed in your custom JupyterLab +> +> +> ```python +> # This example snippet is taken from https://github.com/igvteam/igv-notebook/blob/v3.1.4/README.md +> # License: MIT License +> +> import igv_notebook +> igv_notebook.init() +> igv_browser= igv_notebook.Browser( +> { +> "genome": "hg19", +> "locus": "chr22:24,376,166-24,376,456", +> "tracks": [{ +> "name": "BAM", +> "url": "https://s3.amazonaws.com/igv.org.demo/gstt1_sample.bam", +> "indexURL": "https://s3.amazonaws.com/igv.org.demo/gstt1_sample.bam.bai", +> "format": "bam", +> "type": "alignment" +> }], +> "roi": [ +> { +> "name": "ROI set 1", +> "url": "https://s3.amazonaws.com/igv.org.test/data/roi/roi_bed_1.bed", +> "indexed": False, +> "color": "rgba(94,255,1,0.25)" +> } +> ] +> } +> ) +> ``` +{: .question} + +> Proposed approach +> +> Keep in mind, there are many ways to solve this exercise. This proposed one is only one way to get the job done. +> +> 1. Create a new `Dockerfile`: +> ```dockerfile +> FROM quay.io/jupyter/scipy-notebook:2025-06-23 +> RUN pip install --quiet --no-cache-dir igv-notebook +> ``` +> 2. Build the new Container Image: +> ```bash +> sudo docker build -t my-custom-jupyterlab . +> ``` +> 3. Run a container from your image +> ```bash +> sudo docker run --rm -p 127.0.0.1:8888:8888 my-custom-jupyterlab +> ``` +> 4. Access the JupyterLab via URL provided by the container output +> 5. Open a Python 3 Jupyter Notebook, paste in the above code and run it. It should work now. +{: .solution} + +# Docker Compose + +While running all these docker commands into the command line directly works fine, it has significant disadvantages: it's verbose and therefore error-prone, it's difficult to reproduce and it's difficult to share and version. + +A better approach is to use a `docker-compose` setup. All relevant config is written into a file called `compose.yml`, which is sharable, reproducable and versionizable. + +```yaml +services: + jupyterlab: + image: my-custom-jupyterlab + build: + context: . + dockerfile: Dockerfile + ports: + - 127.0.0.1:8888:8888 +``` + +You may now build the image simply by executing: + +``` +sudo docker compose build +``` + +An you may run it easily by executing: + +``` +sudo docker compose up +``` diff --git a/topics/teaching/tutorials/jbt-customization-2/tutorial.md b/topics/teaching/tutorials/jbt-customization-2/tutorial.md index fb9d8d10fe8d3b..49922a33c04b2f 100644 --- a/topics/teaching/tutorials/jbt-customization-2/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-2/tutorial.md @@ -21,17 +21,66 @@ Now we know how to include changes, we will build something more complex and loo # Install an additional Kernel -* e.g. rust kernel, or R +https://github.com/jupyter/jupyter/wiki/Jupyter-kernels -* WIP +Rust: https://github.com/evcxr/evcxr/tree/main/evcxr_jupyter + + +``` +FROM quay.io/jupyter/minimal-notebook:2025-06-23 + +USER root +RUN apt update \ + && apt install -y build-essential \ + && apt clean + +USER ${NB_USER} +ADD --chmod=755 https://sh.rustup.rs /tmp/rustup.sh +RUN /tmp/rustup.sh -v -y +RUN . "$HOME/.cargo/env" && cargo install --locked evcxr_jupyter +RUN . "$HOME/.cargo/env" && evcxr_jupyter --install +``` + +Build It. + +Run it. + +Try it out: + +```rust +println!("Hello World!"); +``` + +HINT: + +This method installs all the kernel related resources inside the default homedir. + +Therefore this will not work inside deployments where an empty volume is mounted as homedir, e.g. in default z2jh JupyterHub Deployments. # Config Changes * Where to put them +You may put it in `~/.jupyter/jupyter_lab_config.py` + +But you should put it in `/etc/jupyter/jupyter_lab_config.py` because homedir will be overriden in some infrastructures. + * What to put there +https://docs.jupyter.org/en/latest/use/config.html -* WIP +You find all configuration options here: https://jupyter-server.readthedocs.io/en/latest/other/full-config.html + +* How to put there +Create a new local file `jupyter_lab_config.py` and fill it with content. + +Then, in your Dockerfile add it either as system wide config (/etc) or user config (~/) + +```dockerfile +FROM quay.io/jupyter/minimal-notebook:2025-06-23 + +COPY --chown=root:root jupyter_lab_config.py /etc/jupyter/jupyter_lab_config.py + +``` # Persistent Data @@ -41,12 +90,16 @@ Now we know how to include changes, we will build something more complex and loo ## External persistent Data +* Depends on your environment + * WIP ## Large Data * Don't include it into your Image! +* S3 + * WIP ## Sensitive Data (?) @@ -55,8 +108,74 @@ Now we know how to include changes, we will build something more complex and loo # Application Proxies -* E.G. installing `code-server` - -# (WIP) - -* WIP \ No newline at end of file +Install the pip `jupyter_server_proxy` + +Download code-server from Github https://github.com/coder/code-server/releases + +``` +wget -qO- 'https://github.com/coder/code-server/releases/download/v4.101.2/code-server-4.101.2-linux-amd64.tar.gz' | tar xzvf - -C code-server --strip-components 1 +``` + +Configure the proxy server application in `/etc/jupyter/jupyter_lab_config.py`: + +```python +c.ServerProxy.servers = { + "code-server": { + "command": [ + "/opt/code-server/bin/code-server", + "--auth=none", + "--socket={unix_socket}", + "--disable-telemetry", + "--disable-update-check" + ], + "unix_socket": True, + "timeout": 30, + "absolute_url": False, + "raw_socket_proxy": False, + "launcher_entry": { + "enabled": True, + "title": "Code", + "icon_path": "/opt/code-server/src/browser/media/favicon.svg" + } + } +} +``` + +Extend your Dockerfile: + +```dockerfile +FROM quay.io/jupyter/minimal-notebook:2025-06-23 + +USER root +RUN apt update \ + && apt install -y build-essential \ + && apt clean + +USER ${NB_USER} +ADD --chmod=755 https://sh.rustup.rs /tmp/rustup.sh +RUN /tmp/rustup.sh -v -y +RUN . "$HOME/.cargo/env" && cargo install --locked evcxr_jupyter +RUN . "$HOME/.cargo/env" && evcxr_jupyter --install + +USER root +ARG CS_URL=https://github.com/coder/code-server/releases/download/v4.101.2/code-server-4.101.2-linux-amd64.tar.gz +ARG CS_PATH=/opt/code-server +RUN mkdir "${CS_PATH}" +RUN wget -qO- "${CS_URL}" | tar xzvf - -C "${CS_PATH}" --strip-components 1 +COPY --chown=root:root jupyter_lab_config.py /etc/jupyter/jupyter_lab_config.py + +USER ${NB_UID} +RUN pip install jupyter_server_proxy +``` + +You may install any other arbitrary server applications this way too, e.g. RStudio. + +# Exercise + +Why is the rust installation in step 1 of this tutorial problematic in case this notebook is served via JupyterHub? + +--> it installs rust in homedir + +Build a Customized JupyterLab with an available rust kernel even in JupyterHub szenario. + +--> Dockerfile here diff --git a/topics/teaching/tutorials/jbt-galaxy/tutorial.md b/topics/teaching/tutorials/jbt-galaxy/tutorial.md index 7ed28fb25f41b7..f6f558d308a047 100644 --- a/topics/teaching/tutorials/jbt-galaxy/tutorial.md +++ b/topics/teaching/tutorials/jbt-galaxy/tutorial.md @@ -19,6 +19,33 @@ contributors: (WIP) +# Make your Image available + +E.g. on Docker Hub, Quay, GitLab or any other public accessible Registry + # Create a new interactive tool definition (xml) -# Include your new interactive tool definition into your galaxy installation \ No newline at end of file +--> You may take the available Jupyter Notebook tool definition as starting point: https://github.com/galaxyproject/galaxy/blob/dev/tools/interactive/interactivetool_jupyter_notebook.xml + +--> Tweak it to your needs + +--> Adjust the container image to point to your container image on the registry. + +# Include your new interactive tool definition into your galaxy installation + +Official docs: https://docs.galaxyproject.org/en/master/admin/special_topics/interactivetools.html + +--> Include your newly created definition in your `config/tool_conf.xml` like described in the docs: + +```xml +... + + ... +
+ ... + + ... +
+ ... + +``` diff --git a/topics/teaching/tutorials/jbt-intro/tutorial.md b/topics/teaching/tutorials/jbt-intro/tutorial.md index 1432df11b575ca..a8d0c388951c27 100644 --- a/topics/teaching/tutorials/jbt-intro/tutorial.md +++ b/topics/teaching/tutorials/jbt-intro/tutorial.md @@ -4,7 +4,7 @@ layout: tutorial_hands_on title: An introduction to Jupyter based teaching subtopic: practises draft: true -time_estimation: 30m +time_estimation: 15m questions: - What are the general advantages and limitations of Jupyter based teaching? objectives: @@ -15,25 +15,52 @@ contributors: - mittler-works --- -# Glossary +* Central Platform for interactive teaching -* Jupyter +* All in one place, Code, Text, Math, Plots and more -* Jupyter Notebook +* No installation required -* Jupyter Notebook Server +* Low entry access -* JupyterLab +* Easy scalable + +* Custom Courses in predefined Environments + +# Jupyter Terminology + +In the context of Jupyter, there are some components with similar terminology involved. It is important to know how to keep them apart, which is why this tutorial begins with a list of common Jupyter wording. + +### Jupyter +is the name of the umbrella open source project that provides Jupyter Notebook, JupyterLab, JupyterHub and many more projects. + +### Jupyter Notebook +is a computer program that serves a webbased interactive tool for writing, documenting, and sharing code. Please note, that the `.ipnb` document itself is often reffered to as the Jupyter Notebook. Please always mind the context to distinguish between the Jupyter Notebook program and the Jupyter Notebook document. + +### JupyterLab +is a computer program that serves an extensible webbased environment for interactive coding, that combines Jupyter Notebook with other components like text editors, terminals, kernels, plots and more. + +### JupyterHub +is a centralized organizational platform for JupyterLab on a server. It allows to provide many users with JupyterLab instances and handles authentication and authorization. -* JupyterHub # Advantages for teaching +* User management + +* Integratable with LMS (Moodle etc) + +* Interactive teaching + +* Code, Data, Documentation, Results --> All in one place + +* Customizable for your course needs + * Predefined Environment -* Web-based +* Web-based, BYD --> No onboarding time, instant teaching, Easy to use -* BYD +* Localization * Reproducability @@ -41,8 +68,14 @@ contributors: * Important first steps are not need to be taught +* Bad Code Practices can easily creep in + +* Less debugging possibilities + +* Dependecies + * Habituation effects * Jail effects -# (WIP) \ No newline at end of file +# (WIP) From 250743c319c001acaafaf89c047cb5f8b0332b14 Mon Sep 17 00:00:00 2001 From: Nils Mittler Date: Fri, 27 Jun 2025 11:42:59 +0200 Subject: [PATCH 4/8] Add content to advanced customization and nbgrader and minor changes to other --- learning-pathways/jupyter-based-teaching.md | 8 +- .../tutorials/jbt-customization-1/tutorial.md | 2 +- .../tutorials/jbt-customization-2/tutorial.md | 99 ++++++++++++++++++- .../tutorials/jbt-featured/tutorial.md | 60 ++++++++++- .../teaching/tutorials/jbt-galaxy/tutorial.md | 4 +- .../teaching/tutorials/jbt-jhaas/tutorial.md | 4 +- 6 files changed, 159 insertions(+), 18 deletions(-) diff --git a/learning-pathways/jupyter-based-teaching.md b/learning-pathways/jupyter-based-teaching.md index affad9b464c0fd..da230eaef3d9d1 100644 --- a/learning-pathways/jupyter-based-teaching.md +++ b/learning-pathways/jupyter-based-teaching.md @@ -33,15 +33,11 @@ pathway: - topic: teaching name: jbt-featured - - section: "For teachers: Use custom Jupyter Images with JHaaS" - description: "Learn how to teach a course or workshop with your Jupyter customizations on JHaaS" + - section: "Use your customized JupyterLab for Courses" + description: "Learn how to teach a course or workshop with your custom JupyterLab" tutorials: - topic: teaching name: jbt-jhaas - - - section: "For admins: Provide your Jupyter Images with Galaxy" - description: "Learn how to provide your custom Jupyter customizations on your Galaxy instance" - tutorials: - topic: teaching name: jbt-galaxy diff --git a/topics/teaching/tutorials/jbt-customization-1/tutorial.md b/topics/teaching/tutorials/jbt-customization-1/tutorial.md index 57420c1fd8ef09..92d4c889785153 100644 --- a/topics/teaching/tutorials/jbt-customization-1/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-1/tutorial.md @@ -30,7 +30,7 @@ contributors: * WIP ```bash -sudo docker run --rm -p 127.0.0.1:8888:8888 quay.io/jupyter/minimal-notebook:2025-06-23 +sudo docker run --rm -it -p 127.0.0.1:8888:8888 quay.io/jupyter/minimal-notebook:2025-06-23 ``` * Will print out a link to `http://localhost:8888/lab?token=` diff --git a/topics/teaching/tutorials/jbt-customization-2/tutorial.md b/topics/teaching/tutorials/jbt-customization-2/tutorial.md index 49922a33c04b2f..ccc86724e0c472 100644 --- a/topics/teaching/tutorials/jbt-customization-2/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-2/tutorial.md @@ -4,7 +4,7 @@ layout: tutorial_hands_on title: Advanced Jupyter customization subtopic: practises draft: true -time_estimation: 1h +time_estimation: 90m questions: - How can I tailor Jupyter to my specific teaching needs? objectives: @@ -170,7 +170,7 @@ RUN pip install jupyter_server_proxy You may install any other arbitrary server applications this way too, e.g. RStudio. -# Exercise +# Exercise 1: Persistent Data Why is the rust installation in step 1 of this tutorial problematic in case this notebook is served via JupyterHub? @@ -178,4 +178,97 @@ Why is the rust installation in step 1 of this tutorial problematic in case this Build a Customized JupyterLab with an available rust kernel even in JupyterHub szenario. ---> Dockerfile here +```dockerfile +FROM quay.io/jupyter/minimal-notebook:2025-06-23 + +USER root +RUN apt update \ + && apt install -y build-essential \ + && apt clean + +ARG RUSTUP_URL=https://sh.rustup.rs +ARG RUSTUP_INIT=/tmp/rustup.sh + +ENV RUSTUP_HOME=/opt/rustup +ENV CARGO_HOME=/opt/cargo +ENV JUPYTER_PATH=/usr/local/share/jupyter +ENV PATH="${PATH}:${CARGO_HOME}/bin" + +ADD --chmod=755 "${RUSTUP_URL}" "${RUSTUP_INIT}" +RUN "${RUSTUP_INIT}" -v -y +RUN cargo install --locked evcxr_jupyter +RUN evcxr_jupyter --install + +RUN rm "${RUSTUP_INIT}" + +USER ${NB_UID} +``` + +# Exercise 2: Proxy Application + +Create a simple html page like this: + +```html + + + + Hello World + + + +

Hello World!

+

I am served by your python server application!

+ + +``` + +Include it as `index.html` in a separate folder in your container image. + +Create a proxy application serving this index.html file with the simple python builtin `http.server` package. + +Tip: + +```bash +python3 -m http.server -d {path_to_dir_containing_the_html_file} {PORT} +``` + + + +Serve it via simple python + +Solution: + +Create the index.html as explained above. + +Create a JupyterLab config `jupyter_lab_config.py` like this: + +```python +c.ServerProxy.servers = { + "hello-world-server": { + "command": [ + "python3", + "-m", "http.server", + "-d", "/srv/html", + "{port}" + ], + "launcher_entry": { + "enabled": True, + "title": "Hello World Server" + } + } +} +``` + +Create a Dockerfile like this: + +```dockerfile +FROM quay.io/jupyter/minimal-notebook:2025-06-23 + +USER root + +COPY --chown=root:root jupyter_lab_config.py /etc/jupyter/jupyter_lab_config.py +COPY --chown=root:root index.html /srv/html/index.html + +USER ${NB_UID} +RUN pip install jupyter_server_proxy +``` diff --git a/topics/teaching/tutorials/jbt-featured/tutorial.md b/topics/teaching/tutorials/jbt-featured/tutorial.md index 3a6c7b8b7d58a8..1b0dde38339eb4 100644 --- a/topics/teaching/tutorials/jbt-featured/tutorial.md +++ b/topics/teaching/tutorials/jbt-featured/tutorial.md @@ -4,7 +4,7 @@ layout: tutorial_hands_on title: Featured Jupyter customizations subtopic: practises draft: true -time_estimation: 1h +time_estimation: 30m questions: - Which customizations are battle-tested and recommended for teaching? objectives: @@ -21,14 +21,68 @@ We present adaptions that the JLU Physics Department has been successfully using # nbgitpuller +https://github.com/jupyterhub/nbgitpuller + +Allows a teacher to host his teaching material in a git repository. + +Teacher may update the repo while course is running in order to provide new exercises or to provide solutions for older exercises. + +nbgitpuller will will sync if either + +- the git puller url is opened +- the notebook is started and git puller is configured as default url + ## Install +```dockerfile +FROM quay.io/jupyter/minimal-notebook:2025-06-23 + +RUN pip install nbgitpuller +``` + ## Configuration -## Usage +Link generator: https://nbgitpuller.readthedocs.io/en/latest/link.html + +No special config needed, you just generate a link and call it + +Hint: when using nbgitpuller with a standalone JupyterLab and not JupyterHub, you need to remove the `hub/user-redirect/` part from the resulting URL! + +So, instead of `http://localhost:8888/hub/user-redirect/git-pull?repo=...` you should use `http://localhost:8888/git-pull?repo=...` + +## Use it + +## Exercise: create a git puller url for the igv-notebook repo + +From the earlier tutorial, expand the JupyterLab with igv-notebook installed: install nbgitpuller inside this JupyterLab. + +Then goto link generator and create a git puller link for https://github.com/igvteam/igv-notebook + +Configure it so the `BamFiles_Lab.ipynb` example openes automatically in the JupyterLab. + +solution: configure the link generator like this: + +| --- | --- | +| JupyterHub URL | http://localhost:8888 | +| Git Repository URL | https://github.com/igvteam/igv-notebook.git | +| branch | main | +| File to open | examples/BamFiles_Lab.ipynb | +| Application to Open | JupyterLab | +| Named Server to open | (leave it blank) | + + +Strip off the `hub/user-redirect/` part. + +The link will be: + +http://localhost:8888/git-pull?repo=https%3A%2F%2Fgithub.com%2Figvteam%2Figv-notebook.git&urlpath=tree%2Figv-notebook.git%2Fexamples%2FBamFiles_Lab.ipynb&branch=main + +It should pull the repo and opens the example automatically. You should be able to run the example if you extended the igv-notebook tutorial. # nbgrader +https://github.com/jupyter/nbgrader + ## Install ## Configuration @@ -42,5 +96,3 @@ We present adaptions that the JLU Physics Department has been successfully using # Setup Scripts ## Where to place them what to do with them - -(WIP) \ No newline at end of file diff --git a/topics/teaching/tutorials/jbt-galaxy/tutorial.md b/topics/teaching/tutorials/jbt-galaxy/tutorial.md index f6f558d308a047..5d0f7ad4ff0581 100644 --- a/topics/teaching/tutorials/jbt-galaxy/tutorial.md +++ b/topics/teaching/tutorials/jbt-galaxy/tutorial.md @@ -1,10 +1,10 @@ --- layout: tutorial_hands_on -title: Provide your custom Jupyter Image via Galaxy +title: Provide your custom JupyterLab via Galaxy subtopic: practises draft: true -time_estimation: 1h +time_estimation: 15m questions: - As a Galaxy admin, how can I provide my users with a custom Notebook Server? objectives: diff --git a/topics/teaching/tutorials/jbt-jhaas/tutorial.md b/topics/teaching/tutorials/jbt-jhaas/tutorial.md index f063903631fb0c..7e0b56cf009aa4 100644 --- a/topics/teaching/tutorials/jbt-jhaas/tutorial.md +++ b/topics/teaching/tutorials/jbt-jhaas/tutorial.md @@ -1,10 +1,10 @@ --- layout: tutorial_hands_on -title: Teaching a course with custom Jupyter via JHaaS +title: Provide your custom JupyterLab via JHaaS subtopic: practises draft: true -time_estimation: 1h +time_estimation: 15m questions: - How can I use my Jupyter customizations in a course relevant matter? objectives: From c1c7ac63579cd87e94029d9ec7dab46c6f444ad5 Mon Sep 17 00:00:00 2001 From: Nils Mittler Date: Fri, 27 Jun 2025 15:24:44 +0200 Subject: [PATCH 5/8] Expand bullets to text in intro and customization-1 --- learning-pathways/jupyter-based-teaching.md | 16 ++-- .../tutorials/jbt-customization-1/tutorial.md | 82 +++++++++++++------ .../tutorials/jbt-featured/tutorial.md | 2 + .../teaching/tutorials/jbt-intro/tutorial.md | 39 +++++---- 4 files changed, 91 insertions(+), 48 deletions(-) diff --git a/learning-pathways/jupyter-based-teaching.md b/learning-pathways/jupyter-based-teaching.md index da230eaef3d9d1..1eb5d566fb21f4 100644 --- a/learning-pathways/jupyter-based-teaching.md +++ b/learning-pathways/jupyter-based-teaching.md @@ -3,7 +3,7 @@ layout: learning-pathway title: Jupyter-based teaching type: use description: | - An introduction to Jupyter-based teaching and Jupyter customization. + An introduction to Jupyter-based teaching and JupyterLab customization. editorial_board: - mittler-works @@ -14,13 +14,13 @@ tags: [jupyter, teaching] pathway: - section: "Learn the basics about Jupyter-based teaching" - description: "Advantages and limitations of Jupyter-based teaching" + description: "An Overview on the Jupyter project components as well as advantages and limitations of Jupyter-based teaching and learning." tutorials: - topic: teaching name: jbt-intro - - section: "Customize Jupyter to your needs" - description: "Adapt Jupyter to your course or workshop specific needs" + - section: "Customize JupyterLab to your needs" + description: "Adapt JupyterLab to your course or workshop specific needs by installing custom packages, kernels and proxy applications." tutorials: - topic: teaching name: jbt-customization-1 @@ -28,13 +28,13 @@ pathway: name: jbt-customization-2 - section: "Featured customizations" - description: "Battle-tested and recommended Jupyter extensions for teaching " + description: "Battle-tested and recommended JupyterLab extensions for frustration-free teaching and learning." tutorials: - topic: teaching name: jbt-featured - - section: "Use your customized JupyterLab for Courses" - description: "Learn how to teach a course or workshop with your custom JupyterLab" + - section: "Productive use of your customized JupyterLab" + description: "Learn how to teach a course or workshop, providing you students with your customized JupyterLab." tutorials: - topic: teaching name: jbt-jhaas @@ -43,4 +43,4 @@ pathway: --- -You'll learn about the advantages of using Jupyter for teaching and how you can adapt Jupyter to the very specific needs of your course or workshop. As a teacher, you'll learn how to teach a course or workshop using your Jupyter customizations. As a Galaxy admin, you'll learn how to provide your users with a customized Jupyter. +You'll learn about the advantages of using Jupyter for teaching and how you can adapt JupyterLab to the very specific needs of your courses or workshops. As a teacher, you'll learn how to provide your students with your customized JupyterLab for individual courses or workshops. As a Galaxy admin, you'll learn how to provide your users with a customized JupyterLab each as new interactive tool. diff --git a/topics/teaching/tutorials/jbt-customization-1/tutorial.md b/topics/teaching/tutorials/jbt-customization-1/tutorial.md index 92d4c889785153..b2419fdbd3ac72 100644 --- a/topics/teaching/tutorials/jbt-customization-1/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-1/tutorial.md @@ -1,14 +1,14 @@ --- layout: tutorial_hands_on -title: Basics of Jupyter customization +title: Basics of JupyterLab customization subtopic: practises draft: true time_estimation: 1h questions: - - How can I make basic customizations to Jupyter? + - How can I make basic customizations to JupyterLab? objectives: - - An introduction is provided on how to make basic Jupyter customizations, e.g. installing additional plugins. + - An introduction is provided on how to make basic JupyterLab customizations, e.g. installing additional plugins. key_points: - Basic customization costs litte effort and makes it easier for your course participants to get started. contributors: @@ -17,55 +17,87 @@ contributors: # Prerequisites -* We'll use containerized installation +One common way to serve JupyterLab is the container-based approach, especially when using a JupyterHub infrastructure provider. -* Examples with docker, as it is easy to install on all major platforms +This is why this tutorial also follows the container-based approach. As a teacher who wants to customize a container-based JupyterLab you need to install a container runtime as well as some related tools for building container images and more. -# Start your first local Notebook Server with docker +One of the easier ways to install all the required dependencies is the installation of docker. Docker is available on all major platforms, is very easy to use and has also the advantage, that your produced images will be likely working with any infrastructure provider. -* Show docker stacks +All examples in this tutorial are therefore developed with docker in mind, but they should of course also work with other container platforms. -* Select an image +All commands that you should run in a terminal are given with linux in mind. On windows and mac, you might adapt them, e.g. it might not be neccessary (or possible) to use sudo on windows. -* WIP +## Install docker -```bash -sudo docker run --rm -it -p 127.0.0.1:8888:8888 quay.io/jupyter/minimal-notebook:2025-06-23 -``` +In order to install docker, please follow the official installation steps. Please read through the whole installation documentation first before starting the installation. -* Will print out a link to `http://localhost:8888/lab?token=` +- for Linux, I recommend to install docker engine only\ +[https://docs.docker.com/engine/install/](https://docs.docker.com/engine/install/) -## Access your local Notebook Server +- for Windows, you need docker dekstop - Please use the WSL 2 backend to ensure proper function\ +[https://docs.docker.com/desktop/setup/install/windows-install/](https://docs.docker.com/desktop/setup/install/windows-install/) -* Windows and Mac users will need verification work... +- for Mac, you also need docker desktop\ +[https://docs.docker.com/desktop/setup/install/mac-install/](https://docs.docker.com/desktop/setup/install/mac-install/) -* WIP +# Start your first local JupyterLab with docker -## Missing packages +When you have installed docker successfully, you may now start your first containerized JupyterLab. + +In order to run a container you need a container image to start from. The container image defines all the content of the running container. For JupyterLab, there are some prebuild container images for many usecases: + +[https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html) + +You may select an image that fits closest to your usecase. For this tutorial, we'll use the `jupyter/minimal-notebook` image. + +To run this image, open up a terminal and execute: + +```bash +sudo docker run --rm -p 127.0.0.1:8888:8888 quay.io/jupyter/minimal-notebook:2025-06-23 +``` -* Install a simple python package +Let's tear this command apart: -* WIP +|---|---| +| sudo | Executes the command with elevated privileges. This is neccassary as docker server is installed as privileged daemon by default. | +| docker | The docker binary. Allows to talk to the docker daemon. | +| run | Tells docker to create and start a new container | +| --rm | Tells docker to remove the container as soon as it's stopped | +| -p | Tells docker to setup a port forwarding | +| 127.0.0.1:8888:8888 | The argument for `-p`. A port forwarding on the local interface 127.0.0.1 from port 8888 to the container port 8888 | +| quay.io/jupyter/minimal-notebook:2025-06-23 | The image to create the container from | -* You are now in a predefined environment. All Packages from mentioned in the docker stacks documentation are there, so you can easily open the `Python 3` Notebook presented on the start page and start working. As you are in a minimal notebook you just do have the python default libraries, but you could run this to get the current weekday: +## Access your local Notebook Server + +The above command will print out a link to `http://localhost:8888/lab?token=`. Copy this link into your browser or just right-click-open the link. You will then see your JupyterLab loading. + +When the interface is booted, click on the `Python 3` tile in the Notebook tab to open up a new Notebook. Verify, that everything is working by printing out today's weekday: ```python +# Hint: press shift + enter to execute code in Jupyter Notebook import datetime now = datetime.datetime.now() print(now.strftime("%A")) ``` -* For scientific work you will likely need more than just the default python libraries. Reviewing the docker stacks is helpful and maybe an image like the `scipy-notebook` will have most of the packages you need. But of course these default collections cannot have suitable package selections for the very specific needs of all courses out in the world. +## Missing packages + +You are now in a predefined environment. The packages that are installed are defined by the container image you started, i.e. the `minimal-notebook`. All Packages from this image are available, which are the python default libraries. so you can easily succeed following above steps to print out the weekday, as the used packages are default packages. + +For scientific work however, you will likely need more than just the default python libraries. Reviewing the docker stacks is helpful and maybe an image like the `scipy-notebook` will have most of the packages you need. But of course these default collections cannot have perfect fit package selections for the very specific needs of all courses that you could imagine of. -For example, in this minimal setup you might like to have the option to print out text in a fancier way. Luckily, there is a python package for this: `cowsay`. So let's go ahead, try running +For example, in this minimal setup you might like to have the option to print out the weekday in a fancier way. Luckily, there is a python package for this: `cowsay`. So let's go ahead, try running: ```python +# Hint: press shift + enter to execute code in Jupyter Notebook +import datetime from cowsay import cow -cow('Muuuuh') -``` -As you will see, this fails as cowsay is not available in minimal-notebook. +now = datetime.datetime.now() +cow(now.strftime("%A")) +``` +As you will see, this fails with a `ModuleNotFoundError` as cowsay is not available in minimal-notebook. # Build your first custom Image diff --git a/topics/teaching/tutorials/jbt-featured/tutorial.md b/topics/teaching/tutorials/jbt-featured/tutorial.md index 1b0dde38339eb4..45c2d909894a38 100644 --- a/topics/teaching/tutorials/jbt-featured/tutorial.md +++ b/topics/teaching/tutorials/jbt-featured/tutorial.md @@ -83,6 +83,8 @@ It should pull the repo and opens the example automatically. You should be able https://github.com/jupyter/nbgrader +More: https://nbgrader.readthedocs.io/en/stable/ + ## Install ## Configuration diff --git a/topics/teaching/tutorials/jbt-intro/tutorial.md b/topics/teaching/tutorials/jbt-intro/tutorial.md index a8d0c388951c27..74b3e690e7457e 100644 --- a/topics/teaching/tutorials/jbt-intro/tutorial.md +++ b/topics/teaching/tutorials/jbt-intro/tutorial.md @@ -1,12 +1,12 @@ --- layout: tutorial_hands_on -title: An introduction to Jupyter based teaching -subtopic: practises +title: An introduction to Jupyter-based teaching +subtopic: introduction draft: true time_estimation: 15m questions: - - What are the general advantages and limitations of Jupyter based teaching? + - What are the general advantages and limitations of Jupyter-based teaching? objectives: - An overview is provided to familiarize teachers with the advantages and limitations of Jupyter-based teaching. key_points: @@ -15,34 +15,43 @@ contributors: - mittler-works --- -* Central Platform for interactive teaching +Jupyter Notebooks are a promising way to provide participants of a course or workshop with an easy-to-use, webbased, interactive and reproducible environment to learn and work with. -* All in one place, Code, Text, Math, Plots and more +Jupyter Notebooks makes it possible to Code in various programming languages, writing documentation and other text, doing math, printing plots and much more in one single environment. All relevant resources are ready to be used. -* No installation required +Jupyter Notebooks can be run on an individual machine as well as centrally managed by a JupyterHub server. If provided via JupyterHub, the webbased access truly allows participants to bring their own device, as their devices only need to satisfy common web standards and do not need to satisfy the requirements of the software running inside the notebook, as this will be run on the server. No installation on the participants device required. -* Low entry access +This allows for an low-level access especially for beginner courses or workshops. -* Easy scalable +As a teacher you may provide a predefined learning environment, i.e. a predefined software stack, data, configuration and more. This allows you to cover all the necessary requirements in advance, so your participants do not have to carry out further setup steps during the course, as these tend to be frustrating in many szenarios. -* Custom Courses in predefined Environments +If you use an infrastructure provider for your course, you only have to deal with your JupyterLab customizations in order to adapt JupyterLab to your course. The deployment of these JupyterLabs in order to provide them to your participants is the infrastructure providers' job. With this in mind it makes little difference if you plan your JupyterLab for three or three hundred participants, as it is horizontally scalable. # Jupyter Terminology -In the context of Jupyter, there are some components with similar terminology involved. It is important to know how to keep them apart, which is why this tutorial begins with a list of common Jupyter wording. +To get started, we have to elaborate on some terminology. Especially the term `Jupyter Notebook` is heavily overloaded and may be used in many szanarios while having very different meanings. Over all, in the context of Jupyter, there are some components with similar terminology involved. It is important to know how to keep them apart. + +> Which Jupyter-related terms do you now? +> +> Just take a moment and think about what Jupyter-related terms you have already heard about. For each term that comes to your mind, can you explain how it fits into the Jupyter landscape? +{: .question} + + +Let's start this tutorial with a list of a few very common and important Jupyter terms. ### Jupyter -is the name of the umbrella open source project that provides Jupyter Notebook, JupyterLab, JupyterHub and many more projects. +Jupyter itself is the name of the umbrella open source project that provides software components like Jupyter Notebook, JupyterLab, JupyterHub and many more projects. Jupyter itself is not a software nor a file format. ### Jupyter Notebook -is a computer program that serves a webbased interactive tool for writing, documenting, and sharing code. Please note, that the `.ipnb` document itself is often reffered to as the Jupyter Notebook. Please always mind the context to distinguish between the Jupyter Notebook program and the Jupyter Notebook document. +Jupyter Notebook is a computer program that serves a webbased interactive tool for writing, documenting, and sharing code. It is commonly used on a local machine. Note, this term is heavily overloaded and it is common that the `.ipnb` files themselves are reffered to as Jupyter Notebooks too. + +Please always mind the context to distinguish between the Jupyter Notebook program and the Jupyter Notebook file. ### JupyterLab -is a computer program that serves an extensible webbased environment for interactive coding, that combines Jupyter Notebook with other components like text editors, terminals, kernels, plots and more. +JupyterLab is a computer program that serves an extensible webbased environment for interactive coding, that combines Jupyter Notebook with other components like text editors, terminals, kernels, plots and more. ### JupyterHub -is a centralized organizational platform for JupyterLab on a server. It allows to provide many users with JupyterLab instances and handles authentication and authorization. - +JupyterHub is a centralized organizational platform for managing JupyterLab instances on a server. It allows to provide many users with JupyterLab instances and handles authentication and authorization. # Advantages for teaching From c5448f42c797680039e871e7223bac26878d1026 Mon Sep 17 00:00:00 2001 From: Nils Mittler Date: Sat, 28 Jun 2025 21:29:46 +0200 Subject: [PATCH 6/8] Expand basic customizations tutorial --- .../tutorials/jbt-customization-1/tutorial.md | 162 ++++++++++++------ 1 file changed, 111 insertions(+), 51 deletions(-) diff --git a/topics/teaching/tutorials/jbt-customization-1/tutorial.md b/topics/teaching/tutorials/jbt-customization-1/tutorial.md index b2419fdbd3ac72..e8a67f3d881843 100644 --- a/topics/teaching/tutorials/jbt-customization-1/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-1/tutorial.md @@ -67,6 +67,10 @@ Let's tear this command apart: | 127.0.0.1:8888:8888 | The argument for `-p`. A port forwarding on the local interface 127.0.0.1 from port 8888 to the container port 8888 | | quay.io/jupyter/minimal-notebook:2025-06-23 | The image to create the container from | +> Docker CLI +> The complete docker cli reference can be found at [https://docs.docker.com/reference/cli/docker/](https://docs.docker.com/reference/cli/docker/) +{: .comment} + ## Access your local Notebook Server The above command will print out a link to `http://localhost:8888/lab?token=`. Copy this link into your browser or just right-click-open the link. You will then see your JupyterLab loading. @@ -101,28 +105,58 @@ As you will see, this fails with a `ModuleNotFoundError` as cowsay is not availa # Build your first custom Image -In order to customize this Container Image, we need a so-called `Dockerfile`. Go ahead, create a new directory as your playground. Then, create a file called `Dockerfile` in this new directory. +Of course, you could run `pip install cowsay` in order to have the package available in your notebook. For a simple package installation like this, adding an install step inside the Notebook document itself might be feasible. But in general the installation of tooling should be done on beforehand. Not only is it more convinient, it also is also helps reproducing. -Fill it with content: +So we'll add the `cowsay` package as a simple example for customization. In order to customize the `minimal-notebook` Container Image, we need a so-called `Dockerfile`. -```dockerfile -# First, we say from which container image we want to start -# This is called the base image -FROM quay.io/jupyter/minimal-notebook:2025-06-23 +> +> +> Create a new directory as your playground. Then, create a file called `Dockerfile` in this new directory. +> +> Fill the Dockerfile with this content: +> +> ```dockerfile +> # First, we say from which container image we want to start +> # This is called the base image +> FROM quay.io/jupyter/minimal-notebook:2025-06-23 +> +> # Now, we can run arbitrary commands in order to extend the base image +> RUN pip install --no-cache-dir cowsay +> ``` +{: .hands_on} -# Now, we can run arbitrary commands in order to extend the base image -RUN pip install --quiet --no-cache-dir cowsay -``` +> Let's tear this command apart +> +> |---|---| +> | FROM | This tells docker from which image to start building | +> | RUN | Runs an arbitrary command on top of the image and thus creates a new image layer | +{: .comment} -* Build it, mind the period at the end of the command +> Let's tear this command apart +> +> The complete Dockerfile reference can be found at [https://docs.docker.com/reference/dockerfile/](https://docs.docker.com/reference/dockerfile/). +{: .comment} + +> +> +> In order to build the customized image, run the following command. Please mind the period dot at the end of the command. +> +> ```bash +> sudo docker build -t my-custom-jupyterlab . +> ``` +{: .hands_on} + +> docker build +> +> The `-t` option tells docker to tag the built image with provided tag (`my-custom-jupyterlab`). The `docker build` command needs a build context as the last parameter. We tell docker to use the current directory (`.`) as build context. +{: .comment} -```bash -sudo docker build -t my-custom-jupyterlab . -``` # Use your first custom Image +You may use the newly build image just the same way you have used the `minimal-notebook` image beforehand, just use the tag you have provided with your build command: + ```bash sudo docker run --rm -p 127.0.0.1:8888:8888 my-custom-jupyterlab ``` @@ -132,23 +166,28 @@ Again, you'll provided with a link to open you jupyterlab. Start a Pyhton 3 Note Again, run: ```python +# Hint: press shift + enter to execute code in Jupyter Notebook +import datetime from cowsay import cow -cow('Muuuuh') -``` - -As you will see, the command now succeeds and prints a nice graphical text output. -# Exercise: build your own custom Image +now = datetime.datetime.now() +cow(now.strftime("%A")) +``` -Now, let's go ahead and try out some real usecase. +As you will see, the command now succeeds and prints a ascii-cow with the current weekday. -> Create a custom notebook from the `scipy-notebook` base image and install the `igv-browser` extension. +# Build your own customized JupyterLab -You succeeded, if you can run following snippet successfully and are provided with a genome browser with loaded track: +You now know how to make basic customizations to your JupyterLab. Let's go ahead and try out some real usecase: > Make this script succeed in your custom JupyterLab -> -> +> +> Create a custom notebook from the `scipy-notebook` base image and install the `igv-browser` extension. +> +> You find all relevant information about this extension at [https://github.com/igvteam/igv-notebook](https://github.com/igvteam/igv-notebook). +> +> You have succeeded, if you can run following snippet in your JupyterLab and are provided with a genome browser with loaded track: +> > ```python > # This example snippet is taken from https://github.com/igvteam/igv-notebook/blob/v3.1.4/README.md > # License: MIT License @@ -177,54 +216,75 @@ You succeeded, if you can run following snippet successfully and are provided wi > } > ) > ``` -{: .question} +{: .hands_on} -> Proposed approach +> Possible solution > -> Keep in mind, there are many ways to solve this exercise. This proposed one is only one way to get the job done. +> Keep in mind, there are many ways to solve this exercise. The proposed solution is only one way to succeed. > > 1. Create a new `Dockerfile`: > ```dockerfile > FROM quay.io/jupyter/scipy-notebook:2025-06-23 -> RUN pip install --quiet --no-cache-dir igv-notebook +> RUN pip install --no-cache-dir igv-notebook > ``` > 2. Build the new Container Image: > ```bash -> sudo docker build -t my-custom-jupyterlab . +> sudo docker build -t my-igv-jupyterlab . > ``` > 3. Run a container from your image > ```bash -> sudo docker run --rm -p 127.0.0.1:8888:8888 my-custom-jupyterlab +> sudo docker run --rm -p 127.0.0.1:8888:8888 my-igv-jupyterlab > ``` > 4. Access the JupyterLab via URL provided by the container output -> 5. Open a Python 3 Jupyter Notebook, paste in the above code and run it. It should work now. +> 5. Open a Python 3 Jupyter Notebook, paste in the above snippet and run it +> 6. You should be provided with a genome browser with loaded track {: .solution} # Docker Compose -While running all these docker commands into the command line directly works fine, it has significant disadvantages: it's verbose and therefore error-prone, it's difficult to reproduce and it's difficult to share and version. +Even though running all these docker commands directly from the command line works fine, it has significant disadvantages: +- it's verbose and therefore error-prone +- it's difficult to reproduce +- it's difficult to share and version -A better approach is to use a `docker-compose` setup. All relevant config is written into a file called `compose.yml`, which is sharable, reproducable and versionizable. - -```yaml -services: - jupyterlab: - image: my-custom-jupyterlab - build: - context: . - dockerfile: Dockerfile - ports: - - 127.0.0.1:8888:8888 -``` +A better approach is to use a container composition, e.g. with dockers `compose` plugin. All relevant config is written into a file called `compose.yml`, which is reviewable, sharable, reproducable and versionizable. -You may now build the image simply by executing: +> +> +> Create a file called `compose.yml` inside the directory created above. +> +> Fill the compose file with following content: +> +> ```yaml +> services: +> jupyterlab: +> image: my-custom-jupyterlab +> build: +> context: . +> dockerfile: Dockerfile +> ports: +> - 127.0.0.1:8888:8888 +> ``` +{: .hands_on} -``` -sudo docker compose build -``` +> Docker Compose +> +> A complete reference to the compose format can be found at [https://docs.docker.com/reference/compose-file/](https://docs.docker.com/reference/compose-file/) +{: .comment} -An you may run it easily by executing: +> +> +> Now, build the image by simply executing: +> +> ``` +> sudo docker compose build +> ``` +> +> An then run it by simply executing: +> +> ``` +> sudo docker compose up +> ``` +{: .hands_on} -``` -sudo docker compose up -``` +Using a container composition is the preferred way of using containers on your local machine. However, following tutorials will guide you without compose files. Feel free to practise composition and solve the exercises using container compositions. From 6de9ff6629a95b274071b7a52fb336438b51b3f4 Mon Sep 17 00:00:00 2001 From: Nils Mittler Date: Sat, 28 Jun 2025 23:24:13 +0200 Subject: [PATCH 7/8] Extend advanced JupyterLab customizations --- .../tutorials/jbt-customization-1/tutorial.md | 2 +- .../tutorials/jbt-customization-2/tutorial.md | 224 ++++++++++++------ 2 files changed, 150 insertions(+), 76 deletions(-) diff --git a/topics/teaching/tutorials/jbt-customization-1/tutorial.md b/topics/teaching/tutorials/jbt-customization-1/tutorial.md index e8a67f3d881843..115d4ae3027501 100644 --- a/topics/teaching/tutorials/jbt-customization-1/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-1/tutorial.md @@ -1,7 +1,7 @@ --- layout: tutorial_hands_on -title: Basics of JupyterLab customization +title: Basic JupyterLab customizations subtopic: practises draft: true time_estimation: 1h diff --git a/topics/teaching/tutorials/jbt-customization-2/tutorial.md b/topics/teaching/tutorials/jbt-customization-2/tutorial.md index ccc86724e0c472..566fb226f29dff 100644 --- a/topics/teaching/tutorials/jbt-customization-2/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-2/tutorial.md @@ -1,98 +1,162 @@ --- layout: tutorial_hands_on -title: Advanced Jupyter customization +title: Advanced JupyterLab customizations subtopic: practises draft: true time_estimation: 90m questions: - - How can I tailor Jupyter to my specific teaching needs? + - How can I tailor JupyterLab to my specific teaching needs? objectives: - - A guide to advanced Jupyter customization is provided, e.g. how to install proxy applications. + - A guide to advanced JupyterLab customization is provided, e.g. how to install kernels and proxy applications. key_points: - - Advanced customitzations may be time-consuming, but offer great potential and reusability value. + - Advanced customitzations may be time-consuming, but offer many possibilities and reusability value. contributors: - mittler-works --- -# Introduction - -Now we know how to include changes, we will build something more complex and look at some pitfalls. - -# Install an additional Kernel - -https://github.com/jupyter/jupyter/wiki/Jupyter-kernels - -Rust: https://github.com/evcxr/evcxr/tree/main/evcxr_jupyter - - -``` -FROM quay.io/jupyter/minimal-notebook:2025-06-23 - -USER root -RUN apt update \ - && apt install -y build-essential \ - && apt clean - -USER ${NB_USER} -ADD --chmod=755 https://sh.rustup.rs /tmp/rustup.sh -RUN /tmp/rustup.sh -v -y -RUN . "$HOME/.cargo/env" && cargo install --locked evcxr_jupyter -RUN . "$HOME/.cargo/env" && evcxr_jupyter --install -``` - -Build It. - -Run it. - -Try it out: - -```rust -println!("Hello World!"); -``` - -HINT: - -This method installs all the kernel related resources inside the default homedir. - -Therefore this will not work inside deployments where an empty volume is mounted as homedir, e.g. in default z2jh JupyterHub Deployments. - -# Config Changes - -* Where to put them -You may put it in `~/.jupyter/jupyter_lab_config.py` - -But you should put it in `/etc/jupyter/jupyter_lab_config.py` because homedir will be overriden in some infrastructures. - - -* What to put there -https://docs.jupyter.org/en/latest/use/config.html - -You find all configuration options here: https://jupyter-server.readthedocs.io/en/latest/other/full-config.html - -* How to put there -Create a new local file `jupyter_lab_config.py` and fill it with content. - -Then, in your Dockerfile add it either as system wide config (/etc) or user config (~/) - -```dockerfile -FROM quay.io/jupyter/minimal-notebook:2025-06-23 - -COPY --chown=root:root jupyter_lab_config.py /etc/jupyter/jupyter_lab_config.py - -``` +# Prerequisites + +It is assumed that you either completed the previous tutorial "Basic JupyterLab customizations" or that you already have basic knowledge of containerization and JupyterLab customizing. + +With this knowledge, we'll add some more advanced customizations to JupyterLab and look at some pitfalls. + +# Installing an additional Kernel + +One common szenario for customization is the need for an additional kernel, e.g. for Julia or R. This tutorial guides you through installing a rust kernel on top of `minimal-notebook` as an example. + +> Jupyter Kernels +> +> A list of availible Jupyter Kernels can be found at [https://github.com/jupyter/jupyter/wiki/Jupyter-kernels](https://github.com/jupyter/jupyter/wiki/Jupyter-kernels) +{: .comment} + +If you look for `Rust` you'll find the `Evcxr` Jupyter Kernel which is the one we are going to add. + +> Evcxr Kernel +> +> All relevant information for the Evcxr Kernel can be found at [https://github.com/evcxr/evcxr/tree/main/evcxr_jupyter](https://github.com/evcxr/evcxr/tree/main/evcxr_jupyter) +{: .comment} + +> +> +> Create a new directory. Inside, create a `Dockerfile` with the following content: +> +> ``` +> FROM quay.io/jupyter/minimal-notebook:2025-06-23 +> +> USER root +> RUN apt update \ +> && apt install -y build-essential \ +> && apt clean +> +> USER ${NB_UID} +> ADD --chmod=755 https://sh.rustup.rs /tmp/rustup.sh +> RUN /tmp/rustup.sh -v -y +> +> ENV PATH="${PATH}:${HOME}/.cargo/bin" +> RUN cargo install --locked evcxr_jupyter +> RUN evcxr_jupyter --install +> ``` +{: .hands_on} + +> Breakdown +> +> Again, we are using `minimal-notebook` as base image. But this time we need to do more than just installing a python package: +> 1. we need to install system packages to meet requirements for Rust +> 2. we need to install Rust itself +> 3. we need to install and register the Evcxr kernel +> +> In order to install system packages, we need a privileged user inside the docker build, which is why we are changing to the root user using the `USER` keyword. When installing packages, keep in mind that docker builds are non-interactive, thus you need to use the `-y` flag to bypass the confirmation prompt. +> +> After installing system files, we can return to the JupyterLab default user, who is usually called `jovyan`. The variable `NB_UID` is inherited from the base image. +> +> Next, the rustup utility will be downloaded and executed. Note, with the `--chmod` portion you can tell `ADD` in which filemod to save the downloaded file, thus you do not need to make the file executable in a separate step. +> The rustup script installs the binaries for rust in `${HOME}/.cargo/bin` which is why we need to add it to the path. We can do that simply by using the `ENV` keyword. +> +> At least, we install and register the Evcxr as explained on the projects github page. +{: .question} + +> Build your JupyterLab +> +> ```bash +> sudo docker build -t my-rust-jupyterlab . +> ``` +{: .hands_on} + +> Run your JupyterLab +> +> ```bash +> sudo docker run --rm -p 127.0.0.1:8888:8888 my-rust-jupyterlab . +> ``` +{: .hands_on} + +> Try your JupyterLab +> +> Again, you'll be provided with a link to open your JupyterLab. +> +> Open it, click on the Rust kernel and run following snippet: +> ```rust +> println!("Hello World!"); +> ``` +{: .hands_on} + +Congratulations, you have successfully installed a custom kernel to your JupyterLab! + +# Configuration Changes + +There are a few files for configuring JupyterLab. One important one is the `jupyter_lab_config.py` file, where you can configure various different settings, e.g. the log level of the server application and much more. + +> +> +> All relevant configuration options for `jupyter_lab_config.py` can be found at [https://jupyter-server.readthedocs.io/en/latest/other/full-config.html](https://jupyter-server.readthedocs.io/en/latest/other/full-config.html) +{: .comment} + +You might include this file either as a user configuration in the user's home directory like this: `${HOME}/.jupyter/jupyter_lab_config.py`. But you better include it as a system-wide configuration by including it as `/etc/jupyter/jupyter_lab_config.py`. You can find out why this is the better solution in the [persistent data](#persistent-data) section. + +> Include your custom configuration +> +> First, create a snippet of custom config, e.g. let's edit the default name for Notebooks from "Untitled" to "HelloWorld". Save it as `jupyter_lab_config.py`: +> +> ```python +> c.ContentsManager.untitled_notebook = 'HelloWorld' +> ``` +> +> Then, add it as system wide config to your Dockerfile. Again, we're using the `minimal-notebook` as our base image: +> +> ```dockerfile +> FROM quay.io/jupyter/minimal-notebook:2025-06-23 +> +> COPY --chown=root:root jupyter_lab_config.py /etc/jupyter/jupyter_lab_config.py +> ``` +> +> You may now build it as usual... +> +> ```bash +> sudo docker build -t my-configured-jupyterlab . +> ``` +> +> ...run it as usual... +> +> ```bash +> sudo docker run --rm -p 127.0.0.1:8888:8888 my-configured-jupyterlab . +> ``` +> +> ...and access it with the printed URL. Now, open a new Notebook Document and you'll see it is now called `HelloWorld.ipynb` instead of `Untitled.ipynb`. +{: .hands_on} + +Congratulations, you have successfully configured your JupyterLab with a custom configuration! While this example handles a very low impact customization, please review the available configuration options! You may also configure plugins that way, as you will learn later in the [Application Proxies](#application-proxies) section. # Persistent Data -* HomeDir will be overlayed with a mounted Volume +Even though persistent data is not strictly a topic of customization, it is a relevant topic that you have to keep in mind when customizing your JupyterLab. -* WIP +Running JupyterLab in a container, means that your data may be lost when the container is removed. To avoid data loss, volumes can be used. -## External persistent Data +--> On local machine, mount it wherever you want -* Depends on your environment +--> Bind mount vs actual volume -* WIP +--> With JupyterHub it's common to overlay homedir ## Large Data @@ -104,6 +168,8 @@ COPY --chown=root:root jupyter_lab_config.py /etc/jupyter/jupyter_lab_config.py ## Sensitive Data (?) +* Just avoid at all cost + * Maybe WIP, but maybe put that into `jbt-intro` # Application Proxies @@ -176,6 +242,14 @@ Why is the rust installation in step 1 of this tutorial problematic in case this --> it installs rust in homedir + +HINT: + +This method installs all the kernel related resources inside the default homedir. + +Therefore this will not work inside deployments where an empty volume is mounted as homedir, e.g. in default z2jh JupyterHub Deployments. + + Build a Customized JupyterLab with an available rust kernel even in JupyterHub szenario. ```dockerfile From 647f6a439d9af11ead98028c82761711487fdb24 Mon Sep 17 00:00:00 2001 From: Nils Mittler Date: Sun, 29 Jun 2025 22:53:16 +0200 Subject: [PATCH 8/8] Further extend advanced JupyterLab customizations --- .../tutorials/jbt-customization-2/tutorial.md | 402 +++++++++++------- 1 file changed, 243 insertions(+), 159 deletions(-) diff --git a/topics/teaching/tutorials/jbt-customization-2/tutorial.md b/topics/teaching/tutorials/jbt-customization-2/tutorial.md index 566fb226f29dff..abc970982c99bb 100644 --- a/topics/teaching/tutorials/jbt-customization-2/tutorial.md +++ b/topics/teaching/tutorials/jbt-customization-2/tutorial.md @@ -150,199 +150,283 @@ Congratulations, you have successfully configured your JupyterLab with a custom Even though persistent data is not strictly a topic of customization, it is a relevant topic that you have to keep in mind when customizing your JupyterLab. -Running JupyterLab in a container, means that your data may be lost when the container is removed. To avoid data loss, volumes can be used. - ---> On local machine, mount it wherever you want - ---> Bind mount vs actual volume - ---> With JupyterHub it's common to overlay homedir - -## Large Data - -* Don't include it into your Image! +## Volumes -* S3 - -* WIP +Running JupyterLab in a container, means that your data may be lost when the container is removed. To avoid data loss, volumes can be used. -## Sensitive Data (?) +Volumes are only loosly coupled with containers at runtime and thus they are not tailored to a containers lifecycle. Volumes can be mounted into a container on an arbitrary path and multiple Volumes may by mounted. -* Just avoid at all cost +Please note that data in the directory to which a volume is being mounted will be overlaid by the data in the volume in will wherefore not be easy accessible. That means, if you install user packages or configuration in your home directory when building the image and if you are mounting a volume as your home directory for persistence, all prebuild data in this home directory will not be available in the running container. -* Maybe WIP, but maybe put that into `jbt-intro` +This is especially an issue if you are using a container-based JupyterHub Provider, as it is very likely that an empty volume will be mounted as users homedirectory by default. On your local machine you have the free choice on which directory a volume should be mounted. You could use `/mnt/data` or `/home/jovyan/data` in order not to override your complete home directory. -# Application Proxies +For your local machine, you have two basic options for using volumes with your containers: Named Volumes and Bind Mounts. -Install the pip `jupyter_server_proxy` +### Named Volumes -Download code-server from Github https://github.com/coder/code-server/releases +Named Volumes are managed by docker and will be automatically created on demand as soon as it is requested. The created Volume will be just empty. You may reference a named volume with it's name or id. -``` -wget -qO- 'https://github.com/coder/code-server/releases/download/v4.101.2/code-server-4.101.2-linux-amd64.tar.gz' | tar xzvf - -C code-server --strip-components 1 -``` - -Configure the proxy server application in `/etc/jupyter/jupyter_lab_config.py`: - -```python -c.ServerProxy.servers = { - "code-server": { - "command": [ - "/opt/code-server/bin/code-server", - "--auth=none", - "--socket={unix_socket}", - "--disable-telemetry", - "--disable-update-check" - ], - "unix_socket": True, - "timeout": 30, - "absolute_url": False, - "raw_socket_proxy": False, - "launcher_entry": { - "enabled": True, - "title": "Code", - "icon_path": "/opt/code-server/src/browser/media/favicon.svg" - } - } -} +```bash +# This mounts the named volume my-data-volume to /data. If it does not exist, it will be created in first place. +sudo docker run --rm -v "my-data-volume:/data" -p 127.0.0.1:8888:8888 my-configured-jupyterlab ``` -Extend your Dockerfile: +### Bind Mounts -```dockerfile -FROM quay.io/jupyter/minimal-notebook:2025-06-23 +Instead of having a volume managed by docker, you may also bind mount an existing directory into the container. This is useful when you have data to be shared on your local drive, e.g. for live coding or for analysis. -USER root -RUN apt update \ - && apt install -y build-essential \ - && apt clean +Please note, bind mounting data into the container may lead to problems with uid and gid numbers for the mounted directory, as the user inside docker not neccessarly has the same uid and gid numbers as your local machine user. -USER ${NB_USER} -ADD --chmod=755 https://sh.rustup.rs /tmp/rustup.sh -RUN /tmp/rustup.sh -v -y -RUN . "$HOME/.cargo/env" && cargo install --locked evcxr_jupyter -RUN . "$HOME/.cargo/env" && evcxr_jupyter --install - -USER root -ARG CS_URL=https://github.com/coder/code-server/releases/download/v4.101.2/code-server-4.101.2-linux-amd64.tar.gz -ARG CS_PATH=/opt/code-server -RUN mkdir "${CS_PATH}" -RUN wget -qO- "${CS_URL}" | tar xzvf - -C "${CS_PATH}" --strip-components 1 -COPY --chown=root:root jupyter_lab_config.py /etc/jupyter/jupyter_lab_config.py - -USER ${NB_UID} -RUN pip install jupyter_server_proxy +```bash +# This bind mounts the current directory to /data +sudo docker run --rm -v "$(pwd):/data" -p 127.0.0.1:8888:8888 my-configured-jupyterlab ``` -You may install any other arbitrary server applications this way too, e.g. RStudio. +## Large Data -# Exercise 1: Persistent Data +Large Data must not be added to a container image, but should be mounted from a (shared) volume into the container. -Why is the rust installation in step 1 of this tutorial problematic in case this notebook is served via JupyterHub? +Having big images wastes time and storage. It increases bulid, pull and push times and replicates the data on each host using this image. ---> it installs rust in homedir +On your local machine, you may use a bind mount for sharing. On a JupyterHub Provider your better use shared volumes or s3 in order to provide your JupyterLabs with data. +## Sensitive Data -HINT: +As well as large data, sensitive data must not be added to a container image. -This method installs all the kernel related resources inside the default homedir. +When it comes to sensitive data, it is not about resource consumption, but about data protection rules. It is very hard to use sensitive data inside a container image without violating GDPR, and especially in JupyterHub Provider szenarios it is hardly possible at all. -Therefore this will not work inside deployments where an empty volume is mounted as homedir, e.g. in default z2jh JupyterHub Deployments. +# Application Proxies +Sometimes it might be useful to start a server software with a web interface from within your JupyterLab. Great examples are RStudio or a Code Server. To access this web interface in a containerized setup, a proxy can and should be used. The proxy runs in your JupyterLab and hands all relevant traffic to your server application. -Build a Customized JupyterLab with an available rust kernel even in JupyterHub szenario. +> Jupyter Server Proxy +> +> All relevant info for Jupyter Server Proxy can be found at [https://github.com/jupyterhub/jupyter-server-proxy](https://github.com/jupyterhub/jupyter-server-proxy). +> +{: .comment} -```dockerfile -FROM quay.io/jupyter/minimal-notebook:2025-06-23 +In this tutorial we'll install `code-server` and register it as a proxy application to be accessed directly from your JupyterLab's home screen. The roadmap for implementation is straight forward: +1. Install `code-server` +2. Install `jupyter_server_proxy` +3. Add Application Proxy to your JupyterLab configuration -USER root -RUN apt update \ - && apt install -y build-essential \ - && apt clean +> code server +> +> All relevant info for code-server can be found at [https://github.com/coder/code-server](https://github.com/coder/code-server). +> +{: .comment} -ARG RUSTUP_URL=https://sh.rustup.rs -ARG RUSTUP_INIT=/tmp/rustup.sh +> Installing code server with application proxy +> +> Create a new directory, inside create a `Dockerfile` with following content: +> +> ```dockerfile +> FROM quay.io/jupyter/minimal-notebook:2025-06-23 +> +> USER root +> ARG CS_URL=https://github.com/coder/code-server/releases/download/v4.101.2/code-server-4.101.2-linux-amd64.tar.gz +> ARG CS_PATH=/opt/code-server +> RUN mkdir "${CS_PATH}" +> RUN wget -qO- "${CS_URL}" | tar xzvf - -C "${CS_PATH}" --strip-components 1 +> COPY --chown=root:root jupyter_lab_config.py /etc/jupyter/jupyter_lab_config.py +> +> USER ${NB_UID} +> RUN pip install jupyter_server_proxy +> ``` +> +> Next to the Dockerfile, create a configuration file `jupyter_lab_config.py` with following content: +> +> ```python +> c.ServerProxy.servers = { +> "code-server": { +> "command": [ +> "/opt/code-server/bin/code-server", +> "--auth=none", +> "--socket={unix_socket}", +> "--disable-telemetry", +> "--disable-update-check" +> ], +> "unix_socket": True, +> "timeout": 30, +> "absolute_url": False, +> "raw_socket_proxy": False, +> "launcher_entry": { +> "enabled": True, +> "title": "Code", +> "icon_path": "/opt/code-server/src/browser/media/favicon.svg" +> } +> } +> } +> ``` +> +> Now, build it... +> +> ```bash +> sudo docker build -t my-code-jupyterlab . +> ``` +> +> ...and run it... +> +> ```bash +> sudo docker run --rm -p 127.0.0.1:8888:8888 my-code-jupyterlab . +> ``` +> +> ...and access it with the printed URL. You can now see a new Launcher "Code". Try it out and you'll be provided with a web-based IDE. +{: .hands_on} -ENV RUSTUP_HOME=/opt/rustup -ENV CARGO_HOME=/opt/cargo -ENV JUPYTER_PATH=/usr/local/share/jupyter -ENV PATH="${PATH}:${CARGO_HOME}/bin" +Congratulations, you have successfully installed a `code-server` applications proxy into your JupyterLab. You may install any other arbitrary server applications this way too, e.g. Shiny Server or RStudio. -ADD --chmod=755 "${RUSTUP_URL}" "${RUSTUP_INIT}" -RUN "${RUSTUP_INIT}" -v -y -RUN cargo install --locked evcxr_jupyter -RUN evcxr_jupyter --install +# Exercise 1: Persistent Data -RUN rm "${RUSTUP_INIT}" +> +> +> Why might be the rust installation above, in section 1 of this tutorial, problematic? +> +> > +> > +> > With the installation above, all rust related tools will be installed and configured in the users homedir and thus might be overriden when using volumes. Try it out: +> > +> > ```bash +> > sudo docker run --rm -v "jovyan-home:/home/jovyan" -p 127.0.0.1:8888:8888 my-rust-jupyterlab . +> > ``` +> > +> > Access the notebook and see that your kernel and all rust related packages are missing. +> {: .solution} +> +{: .question} -USER ${NB_UID} -``` +> +> +> Build a customized JupyterLab container that solves this problem. +> +> > +> > +> > Keep in mind, that the proposed solution is only one possible way of solving the problem. You might find other solutions as well. +> > +> > First, create a Dockerfile with following content: +> > +> > ```dockerfile +> > FROM quay.io/jupyter/minimal-notebook:2025-06-23 +> > +> > USER root +> > RUN apt update \ +> > && apt install -y build-essential \ +> > && apt clean +> > +> > ARG RUSTUP_URL=https://sh.rustup.rs +> > ARG RUSTUP_INIT=/tmp/rustup.sh +> > +> > ENV RUSTUP_HOME=/opt/rustup +> > ENV CARGO_HOME=/opt/cargo +> > ENV JUPYTER_PATH=/usr/local/share/jupyter +> > ENV PATH="${PATH}:${CARGO_HOME}/bin" +> > +> > ADD --chmod=755 "${RUSTUP_URL}" "${RUSTUP_INIT}" +> > RUN "${RUSTUP_INIT}" -v -y +> > RUN cargo install --locked evcxr_jupyter +> > RUN evcxr_jupyter --install +> > +> > RUN rm "${RUSTUP_INIT}" +> > +> > USER ${NB_UID} +> > ``` +> > +> > Build it, run it with volume mounted on homedir... +> > +> > ```bash +> > sudo docker build -t my-rust-jupyterlab . +> > sudo docker run --rm -v "jovyan-home:/home/jovyan" -p 127.0.0.1:8888:8888 my-rust-jupyterlab . +> > ``` +> > +> > Access it and verify that it's working. +> {: .solution} +> +{: .question} # Exercise 2: Proxy Application -Create a simple html page like this: - -```html - - - - Hello World - - - -

Hello World!

-

I am served by your python server application!

- - -``` - -Include it as `index.html` in a separate folder in your container image. - -Create a proxy application serving this index.html file with the simple python builtin `http.server` package. - -Tip: - -```bash -python3 -m http.server -d {path_to_dir_containing_the_html_file} {PORT} -``` - +In this exercise, you will configure your own Proxy Application in order to access a static web page served from within your JupyterLab. +> +> +> Create a simple html page like this: +> +> ```html +> +> +> +> Hello World +> +> +> +>

Hello World!

+>

I am served by your python server application!

+> +> +> ``` +{: .hands_on} -Serve it via simple python - -Solution: - -Create the index.html as explained above. - -Create a JupyterLab config `jupyter_lab_config.py` like this: - -```python -c.ServerProxy.servers = { - "hello-world-server": { - "command": [ - "python3", - "-m", "http.server", - "-d", "/srv/html", - "{port}" - ], - "launcher_entry": { - "enabled": True, - "title": "Hello World Server" - } - } -} -``` - -Create a Dockerfile like this: - -```dockerfile -FROM quay.io/jupyter/minimal-notebook:2025-06-23 - -USER root - -COPY --chown=root:root jupyter_lab_config.py /etc/jupyter/jupyter_lab_config.py -COPY --chown=root:root index.html /srv/html/index.html +> +> +> Include the above html snippet as `index.html` in a custom folder in your container image. +> +> Create a proxy application serving this index.html file with the simple python builtin `http.server` package. +> +> > http.server +> > +> > All relevant information about the http-server package can be found at [https://docs.python.org/3/library/http.server.html](https://docs.python.org/3/library/http.server.html). +> {: .comment} +> +> > +> > +> > ```bash +> > python3 -m http.server -d {path_to_dir_containing_the_html_file} {PORT} +> > ``` +> {: .tip} +> +{: .question} -USER ${NB_UID} -RUN pip install jupyter_server_proxy -``` +> +> +> Create an index.html file according to example above. +> +> Create a JupyterLab config `jupyter_lab_config.py` like this: +> +> ```python +> c.ServerProxy.servers = { +> "hello-world-server": { +> "command": [ +> "python3", +> "-m", "http.server", +> "-d", "/srv/html", +> "{port}" +> ], +> "launcher_entry": { +> "enabled": True, +> "title": "Hello World Server" +> } +> } +> } +> ``` +> +> Create a Dockerfile like this: +> +> ```dockerfile +> FROM quay.io/jupyter/minimal-notebook:2025-06-23 +> +> COPY --chown=root:root jupyter_lab_config.py /etc/jupyter/jupyter_lab_config.py +> COPY --chown=root:root index.html /srv/html/index.html +> +> RUN pip install jupyter_server_proxy +> ``` +> +> Build and run it: +> +> ```bash +> sudo docker build -t my-rust-jupyterlab . +> sudo docker run --rm -v "jovyan-home:/home/jovyan" -p 127.0.0.1:8888:8888 my-rust-jupyterlab . +> ``` +> +> Access it and verify that it's working. +{: .solution}