pushed docker image, updated documentation

meta-llama · Nov 22, 2024 · 8878396 · 8878396
1 parent a034a0b
commit 8878396
Show file tree

Hide file tree

Showing 4 changed files with 51 additions and 25 deletions.
diff --git a/README.md b/README.md
@@ -69,6 +69,7 @@ A Distribution is where APIs and Providers are assembled together to provide a c
 |        TGI       	|         [llamastack/distribution-tgi](https://hub.docker.com/repository/docker/llamastack/distribution-tgi/general)        	|       [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/tgi.html)       	|
 |        Together       	|         [llamastack/distribution-together](https://hub.docker.com/repository/docker/llamastack/distribution-together/general)        	|       [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/remote_hosted_distro/together.html)       	|
 |        Fireworks       	|         [llamastack/distribution-fireworks](https://hub.docker.com/repository/docker/llamastack/distribution-fireworks/general)        	|       [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/remote_hosted_distro/fireworks.html)       	|
+|        Nutanix       	|         [distribution-nutanix](https://hub.docker.com/repository/docker/jinanz/distribution-nutanix/general)        	|       [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/remote_hosted_distro/nutanix.html)       	|
 
 ## Installation
 

diff --git a/docs/source/distributions/index.md b/docs/source/distributions/index.md
@@ -39,6 +39,7 @@ If so, we suggest:
 - **Do you have an API key for a remote inference provider like Fireworks, Together, etc.?** If so, we suggest:
   - [distribution-together](./remote_hosted_distro/together.md)
   - [distribution-fireworks](./remote_hosted_distro/fireworks.md)
+  - [distribution-nutanix](./remote_hosted_distro/nutanix.md)
 
 - **Do you want to run Llama Stack inference on your iOS / Android device** If so, we suggest:
   - [iOS](./ondevice_distro/ios_sdk.md)

diff --git a/docs/source/distributions/remote_hosted_distro/index.md b/docs/source/distributions/remote_hosted_distro/index.md
@@ -13,6 +13,7 @@ Remote-Hosted distributions are available endpoints serving Llama Stack API that
 |-------------|----------|-----------|---------|---------|---------|------------|
 | Together | [https://llama-stack.together.ai](https://llama-stack.together.ai) | remote::together | meta-reference | remote::weaviate | meta-reference | meta-reference |
 | Fireworks | [https://llamastack-preview.fireworks.ai](https://llamastack-preview.fireworks.ai) | remote::fireworks | meta-reference | remote::weaviate | meta-reference | meta-reference |
+| Nutanix | [https://llamastack-preview.nutanix.ai](https://llamastack-preview.nutanix.ai) | remote::nutanix | meta-reference | meta-reference  | meta-reference | meta-reference |
 
 ## Connecting to Remote-Hosted Distributions
 

diff --git a/llama_stack/templates/nutanix/doc_template.md b/llama_stack/templates/nutanix/doc_template.md
@@ -1,40 +1,63 @@
 # Nutanix Distribution
 
-The `llamastack/distribution-nutanix` distribution consists of the following provider configurations.
+```{toctree}
+:maxdepth: 2
+:hidden:
 
+self
+```
 
-| **API**         	| **Inference** 	| **Agents**     	| **Memory**                                       	| **Safety**     	| **Telemetry**  	|
-|-----------------	|---------------	|----------------	|--------------------------------------------------	|----------------	|----------------	|
-| **Provider(s)** 	| remote::nutanix   	| meta-reference 	| meta-reference 	| meta-reference 	| meta-reference 	|
+The `llamastack/distribution-{{ name }}` distribution consists of the following provider configurations.
 
+{{ providers_table }}
 
-### Start the Distribution (Hosted remote)
+{% if run_config_env_vars %}
+### Environment Variables
 
-> [!NOTE]
-> This assumes you have an hosted Nutanix AI endpoint and an API Key.
+The following environment variables can be configured:
 
-1. Clone the repo
-```
-git clone [email protected]:meta-llama/llama-stack.git
-cd llama-stack
-```
+{% for var, (default_value, description) in run_config_env_vars.items() %}
+- `{{ var }}`: {{ description }} (default: `{{ default_value }}`)
+{% endfor %}
+{% endif %}
 
-2. Config the model name
+{% if default_models %}
+### Models
 
-Please adjust the `NUTANIX_SUPPORTED_MODELS` variable at line 29 in `llama_stack/providers/adapters/inference/nutanix/nutanix.py` according to your deployment.
+The following models are available by default:
 
-3. Build the distrbution
-```
-pip install -e .
-llama stack build --template nutanix --image-type conda
-```
+{% for model in default_models %}
+- `{{ model.model_id }} ({{ model.provider_model_id }})`
+{% endfor %}
+{% endif %}
 
-4. Edit the yaml file
-```
-vim
-```
 
-5. Serve and enjoy!
+### Prerequisite: API Keys
+Make sure you have a Nutanix AI Endpoint deployed and a API key.  
+
+
+## Running Llama Stack with Nutanix
+
+You can do this via Conda (build code) or Docker.
+
+### Via Docker
+
+```bash
+llama stack build --template nutanix --image-type docker
+
+LLAMA_STACK_PORT=1740
+llama stack run nutanix \
+    --port $LLAMA_STACK_PORT \
+    --env NUTANIX_API_KEY=$NUTANIX_API_KEY
 ```
-llama stack run ntnx --port 174
+
+### Via Conda
+
+```bash
+llama stack build --template nutanix --image-type conda
+
+LLAMA_STACK_PORT=1740
+llama stack run ./run.yaml \
+    --port $LLAMA_STACK_PORT \
+    --env NUTANIX_API_KEY=$NUTANIX_API_KEY
 ```