Implemented base transformers, structured folders, etc

.gitignore

@@ -1,4 +1,14 @@
 .settings
 .vscode
 .idea
-AI_overview.drawio
+AI_overview.drawio
+.conda
+.settings/
+.idea
+.vscode
+transformers/from_scratch/src/__pycache__
+transformers/from_scratch/decoder/src/__pycache__
+transformers/from_scratch/encoder/src/__pycache__
+transformers/from_scratch/encoder-decoder/src/__pycache__
+supervised/base-trainer/.ipynb_checkpoints
+.project

.gitpod.yml

@@ -38,8 +38,8 @@ vscode:
     - bierner.markdown-preview-github-styles
     - oderwat.indent-rainbow
     - mongodb.mongodb-vscode
-    - 2gua.rainbow-brackets
-    - dzhavat.bracket-pair-toggler
+    # - 2gua.rainbow-brackets
+    # - dzhavat.bracket-pair-toggler
     - IBM.output-colorizer
     - GitHub.vscode-pull-request-github
     - cweijan.git-graph-history

.settings/org.eclipse.core.resources.prefs

@@ -1,2 +1,4 @@
 eclipse.preferences.version=1
+encoding//misc_files/bilinear_examples.py=utf-8
+encoding//supervised/base-trainer/src/data/make_dataset.py=utf-8
 encoding/bilinear_examples.py=utf-8

README.md

@@ -3,7 +3,8 @@
 This repository contains examples, tutorials, tools and frameworks on how to prepare training environments, train and apply machine learning (ML) models to problems in various settings. 
 The training environments and selected tools will be based on popularity and personal choice with focus on their overall performance. Emphasis will also be laid on examples where performace will affect model training and implementation. Though my aim is not to recommend any particular tools and frameworks but I am hopeful that you may gain from my personal experience in using these tools. Besides, I will also select a number of ML model examples which are mostly suited to my use cases. Specifically, I will mention why using a particular tool may be suitable in a given scenario. Next, I will begin by listing the main tools and discuss the training methods and application of the ML models o interest.
 
-### Preparing the Training Environment 
+### Preparing the Training Environment
+
 In order to begin training or fine-tuning any model, we must prepare the training environment. This is necessary in order to facilitate training  and manage different Python versions. This also provides a virtual representation of the libraries and enables us to effectively manage tools and frameworks. It also helps us to prevent potential issues that may arise with using incompatible tools or frameworks which may affect the settings of your operating system.
 In python, I will usually download and install the latest stable Python version.
 

TODO.md

@@ -0,0 +1,11 @@
+# Highlights
+
+ We cover the following topics:
+
+- Vectors, Matrices and Tensors
+- Vectors = magnetude and direction
+- Essential bits of Neural Network Models
+- Structure of Neural Networks Models
+- Essentials of Python Programming
+- Preparing the AI Training Tools (Tensorflow, JAX, PyTorch)
+- Implementation and Applications of AI Models

compose-llms.yaml

@@ -0,0 +1,44 @@
+version: '3'
+
+services:
+  tgi:
+    image: ghcr.io/huggingface/text-generation-inference:latest
+    container_name: tgi
+    ports:
+      - 8080:80
+    volumes:
+      - ${LOCAL_MODEL_CACHE_DIR}:/model_cache
+    environment:
+      - HUGGING_FACE_HUB_TOKEN=${LLAMA_TOKEN}
+    # need this to access GPU
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: 1
+              capabilities: [gpu]
+    command: 
+      - '--huggingface-hub-cache' 
+      - '/model_cache' 
+      - '--model-id'
+      - '${MODEL_ID}' 
+      - '--max-batch-prefill-tokens'
+      - '${MAX_PREFILL_TOKENS}' 
+      - '--quantize' 
+      - '${QUANT}'
+      - '--max-total-tokens'
+      - '${MAX_TOTAL_TOKENS}'
+      - '--max-input-length'
+      - '${MAX_INPUT_LENGTH}'
+    shm_size: 1gb
+  ui:
+    image: localllm-ui:latest
+    container_name: ui
+    build:
+      context: ./chat_ui/
+    ports:
+      - 7000:7000
+
+  # api:
+  #   image:

docker-compose.yml → infra/docker/docker-compose.yml

@@ -1,5 +1,3 @@
----
-version: '2'
 services:
   zookeeper:
     image: confluentinc/cp-zookeeper:7.1.0