Add support for Vicuna 7B and 13B (#903)

serge-chat · Nov 26, 2023 · 05224d2 · 05224d2
1 parent 6e58157
commit 05224d2
Show file tree

Hide file tree

Showing 2 changed files with 28 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -60,6 +60,7 @@ Instructions for setting up Serge on Kubernetes can be found in the [wiki](https
 | **CodeLLaMA** | 7B, 13B |
 | **LLaMA**  | 7B, 13B, 70B |
 | **Mistral** | 7B-Instruct, 7B-OpenOrca |
+| **Vicuna** | 7B-v1.5, 13B-v1.5 |
 | **Zephyr** | 7B-Alpha, 7B-Beta |
 
 Additional weights can be added to the `serge_weights` volume using `docker cp`:

diff --git a/api/src/serge/data/models.json b/api/src/serge/data/models.json
@@ -1,4 +1,31 @@
 [
+    {
+        "name": "Vicuna",
+        "models": [
+            {
+                "name": "Vicuna-7B",
+                "repo": "TheBloke/vicuna-7B-v1.5-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "vicuna-7b-v1.5.Q4_K_M.gguf",
+                        "disk_space": 4080000000.0
+                    }
+                ]
+            },
+            {
+                "name": "Vicuna-13B",
+                "repo": "TheBloke/vicuna-13B-v1.5-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "vicuna-13b-v1.5.Q4_K_M.gguf",
+                        "disk_space": 7870000000.0
+                    }
+                ]
+            }
+        ]
+    },
     {
         "name": "CodeLLaMA",
         "models": [