-
Notifications
You must be signed in to change notification settings - Fork 766
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
pushed docker image, updated documentation
- Loading branch information
1 parent
cb82b1e
commit e1c6a2c
Showing
4 changed files
with
51 additions
and
25 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,40 +1,63 @@ | ||
# Nutanix Distribution | ||
|
||
The `llamastack/distribution-nutanix` distribution consists of the following provider configurations. | ||
```{toctree} | ||
:maxdepth: 2 | ||
:hidden: | ||
self | ||
``` | ||
|
||
| **API** | **Inference** | **Agents** | **Memory** | **Safety** | **Telemetry** | | ||
|----------------- |--------------- |---------------- |-------------------------------------------------- |---------------- |---------------- | | ||
| **Provider(s)** | remote::nutanix | meta-reference | meta-reference | meta-reference | meta-reference | | ||
The `llamastack/distribution-{{ name }}` distribution consists of the following provider configurations. | ||
|
||
{{ providers_table }} | ||
|
||
### Start the Distribution (Hosted remote) | ||
{% if run_config_env_vars %} | ||
### Environment Variables | ||
|
||
> [!NOTE] | ||
> This assumes you have an hosted Nutanix AI endpoint and an API Key. | ||
The following environment variables can be configured: | ||
|
||
1. Clone the repo | ||
``` | ||
git clone [email protected]:meta-llama/llama-stack.git | ||
cd llama-stack | ||
``` | ||
{% for var, (default_value, description) in run_config_env_vars.items() %} | ||
- `{{ var }}`: {{ description }} (default: `{{ default_value }}`) | ||
{% endfor %} | ||
{% endif %} | ||
|
||
2. Config the model name | ||
{% if default_models %} | ||
### Models | ||
|
||
Please adjust the `NUTANIX_SUPPORTED_MODELS` variable at line 29 in `llama_stack/providers/adapters/inference/nutanix/nutanix.py` according to your deployment. | ||
The following models are available by default: | ||
|
||
3. Build the distrbution | ||
``` | ||
pip install -e . | ||
llama stack build --template nutanix --image-type conda | ||
``` | ||
{% for model in default_models %} | ||
- `{{ model.model_id }} ({{ model.provider_model_id }})` | ||
{% endfor %} | ||
{% endif %} | ||
|
||
4. Edit the yaml file | ||
``` | ||
vim | ||
``` | ||
|
||
5. Serve and enjoy! | ||
### Prerequisite: API Keys | ||
Make sure you have a Nutanix AI Endpoint deployed and a API key. | ||
|
||
|
||
## Running Llama Stack with Nutanix | ||
|
||
You can do this via Conda (build code) or Docker. | ||
|
||
### Via Docker | ||
|
||
```bash | ||
llama stack build --template nutanix --image-type docker | ||
|
||
LLAMA_STACK_PORT=1740 | ||
llama stack run nutanix \ | ||
--port $LLAMA_STACK_PORT \ | ||
--env NUTANIX_API_KEY=$NUTANIX_API_KEY | ||
``` | ||
llama stack run ntnx --port 174 | ||
|
||
### Via Conda | ||
|
||
```bash | ||
llama stack build --template nutanix --image-type conda | ||
|
||
LLAMA_STACK_PORT=1740 | ||
llama stack run ./run.yaml \ | ||
--port $LLAMA_STACK_PORT \ | ||
--env NUTANIX_API_KEY=$NUTANIX_API_KEY | ||
``` |