OSG AI

Build OSG AI for class submission

Prerequisites:

• Obtain the Operating system knowledge from the text books
• Provide some knowledge to users about operating system of the class OSG
• English is required. Vietnamese is optional.

Requirements

• An NVIDIA GPU with Studio driver installed and above version 550.
• Minimum available memory (not total memory and exclude swap memory) is 12gb.
• A GNU/Linux operating system that run based-on AMD64 architecture.
• Anaconda (or Miniconda) environment. You can download from here.

Note for WSL2 users:

1. NVIDIA GPU should not be installed normally within the distro, since installing the actual driver requires the Linux header (linux-headers), which is not available in the WSL. Instead, please review Getting Started with CUDA on WSL 2
2. In .wslconfig, please enable GPU settings (set to true or default) and memory in the Main WSL Settings

Getting started

To pull all models, and submodules for tunning the LLM, using

git clone https://github.com/anhvlt-2k6/osg-ai.git
cd osg-ai 
git submodule update --init --recursive
cd osg-ai && git lfs fetch --all

Prepare conda environment

Import and create a new conda environment using this command

cd ..
conda env create -f .envs/anaconda-env-linux.yaml

Then wait until the environment is ready. Once it's done, running

conda activate osg-ai

After that, prepare docling of the conda environment

docling-tools models download

Prepare the tunning environment

Test GPU

To avoid using CPU from tunning the LLM, run

cd prepare-env
python3 test_gpu.py

If it shows something like...

GPU is available: ....

...then it is good to see. However, if it shows

Only CPU is available

Then you should stop from doing the next step. In that case, review your Linux distro or WSL instance.

Tunning the LLM

cd ../training-parse
python3 train.py
python3 convert.py

The output model is in ./osg-ai/osg-ai.gguf

License

The repo follows the Apache License