Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions mkdocs/docs/HPC/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,8 +319,8 @@ Please send an e-mail to {{hpcinfo}} that includes:

{% endif %}

If the software is a Python package, you can manually install it in a virtual environment.
More information can be found [here](./setting_up_python_virtual_environments.md).
If the software is a Python package, you can manually
[install it in a virtual environment](./setting_up_python_virtual_environments.md).
Note that it is still preferred to submit a software installation request,
as the software installed by the HPC team will be optimized for the HPC environment.
This can lead to dramatic performance improvements.
Expand Down
8 changes: 4 additions & 4 deletions mkdocs/docs/HPC/alphafold.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ It is therefore recommended to first familiarize yourself with AlphaFold. The fo
- VSC webpage about AlphaFold: <https://www.vscentrum.be/alphafold>
- Introductory course on AlphaFold by VIB: <https://elearning.vib.be/courses/alphafold>
- "Getting Started with AlphaFold" presentation by Kenneth Hoste (HPC-UGent)
- recording available [on YouTube](https://www.youtube.com/watch?v=jP9Qg1yBGcs)
- slides available [here (PDF)](https://www.vscentrum.be/_files/ugd/5446c2_f19a8723f7f7460ebe990c28a53e56a2.pdf?index=true)
- [recording available](https://www.youtube.com/watch?v=jP9Qg1yBGcs) (on YouTube)
- [slides available](https://www.vscentrum.be/_files/ugd/5446c2_f19a8723f7f7460ebe990c28a53e56a2.pdf?index=true) (PDF)
- see also <https://www.vscentrum.be/alphafold>


Expand Down Expand Up @@ -130,8 +130,8 @@ Likewise for `jackhmmer`, the core count can be controlled via `$ALPHAFOLD_JACKH

### CPU/GPU comparison

The provided timings were obtained by executing the `T1050.fasta` example, as outlined in the Alphafold [README]({{readme}}).
For the corresponding jobscripts, they are available [here](./example-jobscripts).
The provided timings were obtained by executing the `T1050.fasta` example, as outlined in the [Alphafold README]({{readme}}).
The [corresponding jobscripts](#example-jobscripts) are available.

Using `--db_preset=full_dbs`, the following runtime data was collected:

Expand Down
8 changes: 6 additions & 2 deletions mkdocs/docs/HPC/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,12 @@ Make sure you can get to a shell access to the {{hpcinfra}} before proceeding wi

Now that you can login, it is time to transfer files from your local computer to your **home directory** on the {{hpcinfra}}.

Download [tensorflow_mnist.py](https://raw.githubusercontent.com/hpcugent/vsc_user_docs/main/{{exampleloc}}/tensorflow_mnist.py)
and [run.sh](https://raw.githubusercontent.com/hpcugent/vsc_user_docs/main/{{exampleloc}}/run.sh) example scripts to your computer (from [here](https://github.com/hpcugent/vsc_user_docs/tree/main/{{exampleloc}})).
Download following the example scripts to your computer:

- [tensorflow_mnist.py](https://raw.githubusercontent.com/hpcugent/vsc_user_docs/main/{{exampleloc}}/tensorflow_mnist.py)
- [run.sh](https://raw.githubusercontent.com/hpcugent/vsc_user_docs/main/{{exampleloc}}/run.sh)

You can also find the example scripts in our git repo: [https://github.com/hpcugent/vsc_user_docs/](https://github.com/hpcugent/vsc_user_docs/tree/main/mkdocs/docs/HPC/examples/Getting_Started/tensorflow_mnist).

{%- if OS == windows %}

Expand Down
4 changes: 2 additions & 2 deletions mkdocs/docs/HPC/infrastructure.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ Science and Innovation (EWI).
Log in to the HPC-UGent Tier-2 infrastructure via [https://login.hpc.ugent.be](https://login.hpc.ugent.be)
or using SSH via `login.hpc.ugent.be`.

More info on using the web portal you can find [here](web_portal.md),
and about connection with SSH [here](connecting.md).
Read more info on [using the web portal](web_portal.md),
and [about making a connection with SSH](connecting.md).

## Tier-2 compute clusters

Expand Down
4 changes: 3 additions & 1 deletion mkdocs/docs/HPC/jupyter.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,9 @@ $ module load SciPy-bundle/2023.11-gfbf-2023b
```
This throws no errors, since this module uses a toolchain that is compatible with the toolchain used by the notebook

If we use a different SciPy module that uses an incompatible toolchain, we will get a module load conflict when trying to load it (For more info on these errors, see [here](troubleshooting.md#module-conflicts)).
If we use a different SciPy module that uses an incompatible toolchain,
we will get a module load conflict when trying to load it
(for more info on these errors, consult the [troubleshooting page](troubleshooting.md#module-conflicts)).

```shell
$ module load JupyterNotebook/7.2.0-GCCcore-13.2.0
Expand Down
2 changes: 1 addition & 1 deletion mkdocs/docs/HPC/multi_core_jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ MPI.

!!! warning
Just requesting more nodes and/or cores does not mean that your job will automatically run faster.
You can find more about this [here](troubleshooting.md#job_does_not_run_faster).
This is explained on the [troubleshooting page](troubleshooting.md#job_does_not_run_faster).

## Parallel Computing with threads

Expand Down
6 changes: 5 additions & 1 deletion mkdocs/docs/HPC/multi_job_submission.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,11 @@ a parameter instance is called a work item in Worker parlance.
```
module swap cluster/donphan
```
We recommend using a `module swap cluster` command after submitting the jobs. Additional information about this as well as more comprehensive details concerning the 'Illegal instruction' error can be accessed [here](troubleshooting.md#multi-job-submissions-on-a-non-default-cluster).

We recommend using a `module swap cluster` command after submitting the jobs.
Additional information about this as well as more comprehensive details
concerning the 'Illegal instruction' error can be found
on [the troubleshooting page](troubleshooting.md#multi-job-submissions-on-a-non-default-cluster).

## The Worker framework: Job arrays
[//]: # (sec:worker-framework-job-arrays)
Expand Down
6 changes: 4 additions & 2 deletions mkdocs/docs/HPC/only/gent/2023/donphan-gallade.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ For software installation requests, please use the [request form](https://www.ug

`donphan` is the new debug/interactive cluster.

It replaces `slaking`, which will be retired on **Monday 22 May 2023**.
It replaces `slaking`, which was retired on **Monday 22 May 2023**.

It is primarily intended for interactive use: interactive shell sessions, using GUI applications through the
[HPC-UGent web portal](../../../web_portal.md), etc.
Expand Down Expand Up @@ -135,4 +135,6 @@ a `gallade` workernode has 128 cores (so ~7.3 GiB per core on average), while a
(so ~20.5 GiB per core on average).

It is important to take this aspect into account when submitting jobs to `gallade`, especially when requesting
all cores via `ppn=all`. You may need to explictly request more memory (see also [here](../../../fine_tuning_job_specifications#pbs_mem)).
all cores via `ppn=all`.
You may need to explictly request more memory by
[setting the memory parameter](../../../fine_tuning_job_specifications#pbs_mem).
4 changes: 3 additions & 1 deletion mkdocs/docs/HPC/running_batch_jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -833,7 +833,9 @@ The output of the various commands interacting with jobs (`qsub`,
It is possible to submit jobs from a job to a cluster different than the one your job is running on.
This could come in handy if, for example, the tool used to submit jobs only works on a particular cluster
(or only on the login nodes), but the jobs can be run on several clusters.
An example of this is the `wsub` command of `worker`, see also [here](troubleshooting.md#multi-job-submissions-on-a-non-default-cluster).
An example of this is the `wsub` command of `worker`.
More info on these commands is in the document on [multi job submission](multi_job_submission.md)
or on the [troubleshooting page](troubleshooting.md#multi-job-submissions-on-a-non-default-cluster).

To submit jobs to the `{{othercluster}}` cluster, you can change only what is needed in your session environment
to submit jobs to that particular cluster by using `module swap env/slurm/{{othercluster}}` instead of using
Expand Down
3 changes: 2 additions & 1 deletion mkdocs/docs/HPC/setting_up_python_virtual_environments.md
Original file line number Diff line number Diff line change
Expand Up @@ -363,7 +363,8 @@ $ python
Illegal instruction (core dumped)
```

we are presented with the illegal instruction error. More info on this [here](troubleshooting.md#illegal-instruction-error)
we are presented with the illegal instruction error.
More info on this on the [troubleshooting page](troubleshooting.md#illegal-instruction-error)

### Error: GLIBC not found

Expand Down
4 changes: 2 additions & 2 deletions mkdocs/docs/HPC/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ or because the pinning is done incorrectly and several threads/processes are bei
- **Lack of sufficient memory**: When there is not enough memory available, or not enough memory bandwidth,
it is likely that you will not see a significant speedup when using more cores (since each thread or process most likely requires additional memory).

More info on running multi-core workloads on the {{ hpcinfra }} can be found [here](multi_core_jobs.md).
There is more info on [running multi-core workloads](multi_core_jobs.md) on the {{ hpcinfra }}.

### Using multiple nodes
When trying to use multiple (worker)nodes to improve the performance of your workloads, you may not see (significant) speedup.
Expand All @@ -74,7 +74,7 @@ Actually using additional nodes is not as straightforward as merely asking for m
Using the resources of multiple nodes is often done using a [Message Passing Interface (MPI)](https://en.wikipedia.org/wiki/Message_Passing_Interface) library.
MPI allows nodes to communicate and coordinate, but it also introduces additional complexity.

An example of how you can make beneficial use of multiple nodes can be found [here](multi_core_jobs.md#parallel-computing-with-mpi).
We have an example of [how you can make beneficial use of multiple nodes](multi_core_jobs.md#parallel-computing-with-mpi).

You can also use MPI in Python, some useful packages that are also available on the HPC are:

Expand Down
8 changes: 3 additions & 5 deletions mkdocs/docs/HPC/web_portal.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,9 @@ Through this web portal, you can:

- open a terminal session directly in your web browser;

More detailed information is available below, as well as in the [Open
OnDemand
documentation](https://osc.github.io/ood-documentation/latest/). A
walkthrough video is available on YouTube
[here](https://www.youtube.com/watch?v=4-w-4wjlnPk).
More detailed information is available below, as well as in the
[Open OnDemand documentation](https://osc.github.io/ood-documentation/latest/).
A [walkthrough video](https://www.youtube.com/watch?v=4-w-4wjlnPk) is available on YouTube.

## Pilot access

Expand Down