Add support for newer LLM models (#1334)

* Bump tqdm in /api in the pip group across 1 directory Bumps the pip group with 1 update in the /api directory: [tqdm](https://github.com/tqdm/tqdm). Updates `tqdm` from 4.66.3 to 4.66.4 - [Release notes](https://github.com/tqdm/tqdm/releases) - [Commits](tqdm/tqdm@v4.66.3...v4.66.4) --- updated-dependencies: - dependency-name: tqdm dependency-type: indirect dependency-group: pip ... Signed-off-by: dependabot[bot] <[email protected]> * Add support for Codestral 22B v0.1 * Update README.md * Update README.md * Update models.json Add Phi-3-mini-4k-instruct-v0.3 Add Phi-3-medium-4k-instruct Add Phi-3-medium-128k-instruct * Update models.json Add SOLAR-10.7B-v1.0 Add SOLAR-10.7B-instruct-v1.0 * Update README.md * Update README.md * Update models.json --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
serge-chat · Jun 15, 2024 · d48fc7d · d48fc7d
1 parent 0dc3097
commit d48fc7d
Show file tree

Hide file tree

Showing 3 changed files with 84 additions and 5 deletions.
diff --git a/README.md b/README.md
@@ -47,7 +47,7 @@ Then, just visit http://localhost:8008, You can find the API documentation at ht
 
 ## 🖥️ Windows
 
-Ensure you have Docker Desktop installed, WSL2 configured, and enough free RAM to run models. 
+Ensure you have Docker Desktop installed, WSL2 configured, and enough free RAM to run models.
 
 ## ☁️ Kubernetes
 
@@ -61,6 +61,7 @@ Instructions for setting up Serge on Kubernetes can be found in the [wiki](https
 | **BioMistral** | 7B |
 | **Code** | 13B, 33B |
 | **CodeLLaMA** | 7B, 7B-Instruct, 7B-Python, 13B, 13B-Instruct, 13B-Python, 34B, 34B-Instruct, 34B-Python |
+| **Codestral** | 22B v0.1 |
 | **Gemma** | 2B, 1.1-2B-Instruct, 7B, 1.1-7B-Instruct |
 | **Gorilla** | Falcon-7B-HF-v0, 7B-HF-v1, Openfunctions-v1, Openfunctions-v2 |
 | **Falcon** | 7B, 7B-Instruct, 40B, 40B-Instruct |
@@ -84,9 +85,11 @@ Instructions for setting up Serge on Kubernetes can be found in the [wiki](https
 | **OpenLLaMA** | 3B-v2, 7B-v2, 13B-v2 |
 | **Orca 2** | 7B, 13B |
 | **Phi 2** | 2.7B |
+| **Phi 3** | mini-4k-instruct, medium-4k-instruct, medium-128k-instruct |
 | **Python Code** | 13B, 33B |
 | **PsyMedRP** | 13B-v1, 20B-v1 |
 | **Starling LM** | 7B-Alpha |
+| **SOLAR** | 10.7B-v1.0, 10.7B-instruct-v1.0 |
 | **TinyLlama** | 1.1B |
 | **Vicuna** | 7B-v1.5, 13B-v1.5, 33B-v1.3, 33B-Coder |
 | **WizardLM** | 2-7B, 13B-v1.2, 70B-v1.0 |

diff --git a/api/poetry.lock b/api/poetry.lock
diff --git a/api/src/serge/data/models.json b/api/src/serge/data/models.json
@@ -161,7 +161,23 @@
                 ]
             }             
         ]
-    },    
+    },
+    {   
+        "name": "Codesstral",
+        "models": [
+            {
+                "name": "Codestral-22B-v0.1",
+                "repo": "bartowski/Codestral-22B-v0.1-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "Codestral-22B-v0.1-Q4_K_M.gguf",
+                        "disk_space": 15722553696.0
+                    }
+                ]
+            }           
+        ]
+    },
     {
         "name": "Falcon",
         "models": [   
@@ -956,6 +972,39 @@
                         "disk_space": 1789239136.0
                     }
                 ]
+            },
+            {
+                "name": "Phi-3-mini-4k-instruct-v0_3",
+                "repo": "bartowski/Phi-3-mini-4k-instruct-v0.3-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "Phi-3-mini-4k-instruct-v0.3-Q4_K_M.gguf",
+                        "disk_space": 2393231456.0
+                    }
+                ]
+            },
+            {
+                "name": "Phi-3-medium-4k-instruct",
+                "repo": "bartowski/Phi-3-medium-4k-instruct-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "Phi-3-medium-4k-instruct-Q4_K_M.gguf",
+                        "disk_space": 8566820736.0
+                    }
+                ]
+            },
+            {
+                "name": "Phi-3-medium-128k-instruct",
+                "repo": "bartowski/Phi-3-medium-128k-instruct-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "Phi-3-medium-128k-instruct-Q4_K_M.gguf",
+                        "disk_space": 8566821408.0
+                    }
+                ]
             }           
         ]
     },    
@@ -1045,6 +1094,33 @@
             }
         ]
     },
+    {   
+        "name": "SOLAR",
+        "models": [
+            {
+                "name": "SOLAR-10.7B-V1_0",
+                "repo": "TheBloke/SOLAR-10.7B-v1.0-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "solar-10.7b-v1.0.Q4_K_M.gguf",
+                        "disk_space": 6461667488.0
+                    }
+                ]
+            },
+            {
+                "name": "SOLAR-10.7B-instruct-V1_0",
+                "repo": "TheBloke/SOLAR-10.7B-Instruct-v1.0-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "solar-10.7b-instruct-v1.0.Q4_K_M.gguf",
+                        "disk_space": 6461667936.0
+                    }
+                ]
+            }            
+        ]
+    },
     {   
         "name": "Tinyllama",
         "models": [