Skip to content

Commit 0947853

Browse files
sdgilleysdgilley
committed
Merge branch 'sdg-rename-notebooks' of github.com:Azure/azureml-examples into sdg-rename-notebooks
2 parents 367a49c + 574bf0e commit 0947853

File tree

30 files changed

+3531
-36
lines changed

30 files changed

+3531
-36
lines changed

‎.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
/sdk/python/foundation-models/cohere/cohere-aisearch-langchain-rag.ipynb @stewart-co @kseniia-cohere
1818
sdk/python/foundation-models/cohere/command_faiss_langchain.ipynb @stewart-co @kseniia-cohere
1919
sdk/python/foundation-models/cohere/command_tools-langchain.ipynb @stewart-co @kseniia-cohere
20+
/sdk/python/foundation-models/nixtla/ @AzulGarza
2021

2122
#### files referenced in docs (DO NOT EDIT, except for Docs team!!!) #############################################################################################
2223
/cli/assets/component/train.yml @sdgilley @msakande @Blackmist @ssalgadodev @lgayhardt @fbsolo-ms1

‎cli/foundation-models/system/finetune/image-classification/multiclass-classification/readme.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@ For using this component, run the shell script file `bash ./hftransformers-fridg
77
Currently following models are supported:
88
| Model Name | Source |
99
| ------ | ---------- |
10-
| [microsoft-beit-base-patch16-224-pt22k-ft22k](https://ml.azure.com/registries/azureml/models/microsoft-beit-base-patch16-224-pt22k-ft22k/version/11) | azureml registry |
11-
| [microsoft-swinv2-base-patch4-window12-192-22k](https://ml.azure.com/registries/azureml/models/microsoft-swinv2-base-patch4-window12-192-22k/version/11) | azureml registry |
12-
| [facebook-deit-base-patch16-224](https://ml.azure.com/registries/azureml/models/facebook-deit-base-patch16-224/version/10) | azureml registry |
13-
| [google-vit-base-patch16-224](https://ml.azure.com/registries/azureml/models/google-vit-base-patch16-224/version/10) | azureml registry |
10+
| [microsoft-beit-base-patch16-224-pt22k-ft22k](https://ml.azure.com/registries/azureml/models/microsoft-beit-base-patch16-224-pt22k-ft22k/version/19) | azureml registry |
11+
| [microsoft-swinv2-base-patch4-window12-192-22k](https://ml.azure.com/registries/azureml/models/microsoft-swinv2-base-patch4-window12-192-22k/version/20) | azureml registry |
12+
| [facebook-deit-base-patch16-224](https://ml.azure.com/registries/azureml/models/facebook-deit-base-patch16-224/version/19) | azureml registry |
13+
| [google-vit-base-patch16-224](https://ml.azure.com/registries/azureml/models/google-vit-base-patch16-224/version/17) | azureml registry |
1414
| [Image classification models from Huggingface's Transformer library](https://huggingface.co/models?pipeline_tag=image-classification&library=transformers)| HuggingFace |

‎cli/foundation-models/system/finetune/image-classification/multilabel-classification/readme.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@ For using this component, run the shell script file `bash ./hftransformers-fridg
77
Currently following models are supported:
88
| Model Name | Source |
99
| ------ | ---------- |
10-
| [microsoft-beit-base-patch16-224-pt22k-ft22k](https://ml.azure.com/registries/azureml/models/microsoft-beit-base-patch16-224-pt22k-ft22k/version/11) | azureml registry |
11-
| [microsoft-swinv2-base-patch4-window12-192-22k](https://ml.azure.com/registries/azureml/models/microsoft-swinv2-base-patch4-window12-192-22k/version/11) | azureml registry |
12-
| [facebook-deit-base-patch16-224](https://ml.azure.com/registries/azureml/models/facebook-deit-base-patch16-224/version/10) | azureml registry |
13-
| [google-vit-base-patch16-224](https://ml.azure.com/registries/azureml/models/google-vit-base-patch16-224/version/10) | azureml registry |
10+
| [microsoft-beit-base-patch16-224-pt22k-ft22k](https://ml.azure.com/registries/azureml/models/microsoft-beit-base-patch16-224-pt22k-ft22k/version/19) | azureml registry |
11+
| [microsoft-swinv2-base-patch4-window12-192-22k](https://ml.azure.com/registries/azureml/models/microsoft-swinv2-base-patch4-window12-192-22k/version/20) | azureml registry |
12+
| [facebook-deit-base-patch16-224](https://ml.azure.com/registries/azureml/models/facebook-deit-base-patch16-224/version/19) | azureml registry |
13+
| [google-vit-base-patch16-224](https://ml.azure.com/registries/azureml/models/google-vit-base-patch16-224/version/17) | azureml registry |
1414
| [Image classification models from Huggingface's Transformer library](https://huggingface.co/models?pipeline_tag=image-classification&library=transformers)| HuggingFace |

‎cli/foundation-models/system/finetune/image-instance-segmentation/readme.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,5 +7,5 @@ For using this component for instance segmentation, run the shell script file `b
77
Currently following models are supported:
88
| Model Name | Source |
99
| :------------: | :-------: |
10-
| [mask-rcnn_swin-t-p4-w7_fpn_1x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-mask-rcnn_swin-t-p4-w7_fpn_1x_coco/version/8) | azureml registry |
10+
| [mmd-3x-mask-rcnn_swin-t-p4-w7_fpn_1x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-mask-rcnn_swin-t-p4-w7_fpn_1x_coco/version/14) | azureml registry |
1111
| [Image instance-segmentation models from MMDetection](https://github.com/open-mmlab/mmdetection/blob/v3.1.0/docs/en/model_zoo.md) | MMDetection |

‎cli/foundation-models/system/finetune/image-object-detection/readme.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,13 @@ You can launch a sample pipeline for image object detection using `mmdetection_i
55
For using this component for object detection, run the shell script file `bash ./mmdetection-fridgeobjects-detection.sh`.
66

77
Currently following models are supported:
8+
89
| Model Name | Source |
910
| :------------: | :-------: |
10-
| [deformable-detr_refine_twostage_r50_16xb2-50e_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-deformable-detr_refine_twostage_r50_16xb2-50e_coco/version/8) | azureml registry |
11-
| [sparse-rcnn_r50_fpn_300-proposals_crop-ms-480-800-3x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-sparse-rcnn_r50_fpn_300-proposals_crop-ms-480-800-3x_coco/version/8) | azureml registry |
12-
| [sparse-rcnn_r101_fpn_300-proposals_crop-ms-480-800-3x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-sparse-rcnn_r101_fpn_300-proposals_crop-ms-480-800-3x_coco/version/8) | azureml registry |
13-
| [vfnet_r50-mdconv-c3-c5_fpn_ms-2x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-vfnet_r50-mdconv-c3-c5_fpn_ms-2x_coco/version/8) | azureml registry |
14-
| [vfnet_x101-64x4d-mdconv-c3-c5_fpn_ms-2x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-vfnet_x101-64x4d-mdconv-c3-c5_fpn_ms-2x_coco/version/8) | azureml registry |
15-
| [yolof_r50_c5_8x8_1x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-yolof_r50_c5_8x8_1x_coco/version/8) | azureml registry |
11+
| [mmd-3x-deformable-detr_refine_twostage_r50_16xb2-50e_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-deformable-detr_refine_twostage_r50_16xb2-50e_coco/version/12) | azureml registry |
12+
| [mmd-3x-sparse-rcnn_r50_fpn_300-proposals_crop-ms-480-800-3x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-sparse-rcnn_r50_fpn_300-proposals_crop-ms-480-800-3x_coco/version/12) | azureml registry |
13+
| [mmd-3x-sparse-rcnn_r101_fpn_300-proposals_crop-ms-480-800-3x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-sparse-rcnn_r101_fpn_300-proposals_crop-ms-480-800-3x_coco/version/12) | azureml registry |
14+
| [mmd-3x-vfnet_r50-mdconv-c3-c5_fpn_ms-2x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-vfnet_r50-mdconv-c3-c5_fpn_ms-2x_coco/version/12) | azureml registry |
15+
| [mmd-3x-vfnet_x101-64x4d-mdconv-c3-c5_fpn_ms-2x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-vfnet_x101-64x4d-mdconv-c3-c5_fpn_ms-2x_coco/version/12) | azureml registry |
16+
| [mmd-3x-yolof_r50_c5_8x8_1x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-yolof_r50_c5_8x8_1x_coco/version/12) | azureml registry |
1617
| [Image object detection models from MMDetection](https://github.com/open-mmlab/mmdetection/blob/v3.1.0/docs/en/model_zoo.md) | MMDetection |

‎cli/foundation-models/system/inference/visual-chat-completion/deploy.yml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
2+
name: phi-3-vision
3+
instance_type: Standard_NC48ads_A100_v4
4+
instance_count: 1
5+
liveness_probe:
6+
initial_delay: 180
7+
period: 180
8+
failure_threshold: 49
9+
timeout: 299
10+
request_settings:
11+
request_timeout_ms: 180000

‎cli/foundation-models/system/inference/visual-chat-completion/online-endpoint-deployment.sh

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
set -x
2+
3+
# script inputs
4+
subscription_id="<SUBSCRIPTION_ID>"
5+
resource_group_name="<RESOURCE_GROUP>"
6+
workspace_name="<WORKSPACE_NAME>"
7+
8+
# This is the model from system registry that needs to be deployed
9+
registry_name="azureml"
10+
model_name="Phi-3-vision-128k-instruct"
11+
deployment_sku="Standard_NC48ads_A100_v4"
12+
13+
# Validate the existence of the model in the registry and get the latest version
14+
model_list=$(az ml model list --name ${model_name} --registry-name ${registry_name} 2>&1)
15+
if [[ ${model_list} == *"[]"* ]]; then
16+
echo "Model doesn't exist in registry. Check the model list and try again."; exit 1;
17+
fi
18+
version_temp=${model_list#*\"version\": \"}
19+
model_version=${version_temp%%\"*}
20+
21+
version=$(date +%s)
22+
endpoint_name="phi-3-mini-v-instruct-$version"
23+
24+
# scoring_file
25+
scoring_file="./sample_chat_completions_score.json"
26+
27+
# 1. Setup pre-requisites
28+
if [ "$subscription_id" = "<SUBSCRIPTION_ID>" ] || \
29+
["$resource_group_name" = "<RESOURCE_GROUP>" ] || \
30+
[ "$workspace_name" = "<WORKSPACE_NAME>" ]; then
31+
echo "Please update the script with the subscription_id, resource_group_name and workspace_name"
32+
exit 1
33+
fi
34+
35+
az account set -s $subscription_id
36+
workspace_info="--resource-group $resource_group_name --workspace-name $workspace_name"
37+
38+
# 2. Check if the model exists in the registry
39+
# need to confirm model show command works for registries outside the tenant (aka system registry)
40+
if ! az ml model show --name $model_name --version $model_version --registry-name $registry_name
41+
then
42+
echo "Model $model_name:$model_version does not exist in registry $registry_name"
43+
exit 1
44+
fi
45+
46+
# 3. Deploy the model to an endpoint
47+
# create online endpoint
48+
az ml online-endpoint create --name $endpoint_name $workspace_info || {
49+
echo "endpoint create failed"; exit 1;
50+
}
51+
52+
# deploy model from registry to endpoint in workspace
53+
az ml online-deployment create --file deploy.yml $workspace_info --all-traffic --set \
54+
endpoint_name=$endpoint_name model=azureml://registries/$registry_name/models/$model_name/versions/$model_version \
55+
instance_type=$deployment_sku || {
56+
echo "deployment create failed"; exit 1;
57+
}
58+
59+
# 4. Try a sample scoring request
60+
61+
# Check if scoring data file exists
62+
if [ -f $scoring_file ]; then
63+
echo "Invoking endpoint $endpoint_name with following input:\n\n"
64+
cat $scoring_file
65+
echo "\n\n"
66+
else
67+
echo "Scoring file $scoring_file does not exist"
68+
exit 1
69+
fi
70+
71+
az ml online-endpoint invoke --name $endpoint_name --request-file $scoring_file $workspace_info || {
72+
echo "endpoint invoke failed"; exit 1;
73+
}
74+
75+
# 6. Delete the endpoint
76+
az ml online-endpoint delete --name $endpoint_name $workspace_info --yes || {
77+
echo "endpoint delete failed"; exit 1;
78+
}
79+

‎cli/foundation-models/system/inference/visual-chat-completion/sample_chat_completions_score.json

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"input_data": {
3+
"input_string": [
4+
{
5+
"role": "user",
6+
"content": [
7+
{
8+
"type": "image_url",
9+
"image_url": {
10+
"url": "https://www.ilankelman.org/stopsigns/australia.jpg"
11+
}
12+
},
13+
{
14+
"type": "text",
15+
"text": "What is shown in this image? Be extremely detailed and specific."
16+
}
17+
]
18+
}
19+
],
20+
"parameters": { "temperature": 0.7, "max_new_tokens": 2048 }
21+
}
22+
}

‎sdk/python/foundation-models/jais/litellm.ipynb

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Use litellm with JAIS in Azure AI and Azure ML\n",
8+
"\n",
9+
"Use `litellm` to consume JAIS deployments in Azure AI and Azure ML. Notice that JAIS in Azure only supports chat completions API.\n",
10+
"\n",
11+
"> Review the [documentation](https://aka.ms/jais-azure-ai-studio-docs) for the JAIS 30b Chat model in AI Studio and for ML Studio for details on how to provision inference endpoints, regional availability, pricing and inference schema reference."
12+
]
13+
},
14+
{
15+
"cell_type": "markdown",
16+
"metadata": {},
17+
"source": [
18+
"## Prerequisites\n",
19+
"\n",
20+
"Before we start, there are certain steps we need to take to deploy the models:\n",
21+
"\n",
22+
"* Register for a valid Azure account with subscription \n",
23+
"* Make sure you have access to [Azure AI Studio](https://learn.microsoft.com/en-us/azure/ai-studio/what-is-ai-studio?tabs=home)\n",
24+
"* Create a project and resource group\n",
25+
"* Select `Jais-30b-Chat` in the model catalog.\n",
26+
"\n",
27+
" > Notice that some models may not be available in all the regions in Azure AI and Azure Machine Learning. On those cases, you can create a workspace or project in the region where the models are available and then consume it with a connection from a different one. To learn more about using connections see [Consume models with connections](https://learn.microsoft.com/en-us/azure/ai-studio//deployments-connections)\n",
28+
"\n",
29+
"* Deploy with \"Serverless APIs\"\n",
30+
"\n",
31+
"Once deployed successfully, you should be assigned for an API endpoint and a security key for inference.\n",
32+
"\n",
33+
"For more information, you should consult Azure's official documentation [here](https://aka.ms/jais-azure-ai-studio-docs) for model deployment and inference.\n",
34+
"\n",
35+
"To complete this tutorial, you will need to:\n",
36+
"\n",
37+
"* Install `litellm`:\n",
38+
"\n",
39+
" ```bash\n",
40+
" pip install litellm\n",
41+
" ```"
42+
]
43+
},
44+
{
45+
"cell_type": "markdown",
46+
"metadata": {},
47+
"source": [
48+
"## Example\n",
49+
"\n",
50+
"The following is an example about how to use `litellm` with a JAIS model deployed in Azure AI and Azure ML:"
51+
]
52+
},
53+
{
54+
"cell_type": "code",
55+
"execution_count": null,
56+
"metadata": {
57+
"name": "imports"
58+
},
59+
"outputs": [],
60+
"source": [
61+
"import litellm"
62+
]
63+
},
64+
{
65+
"cell_type": "markdown",
66+
"metadata": {},
67+
"source": [
68+
"You will need to have a Endpoint url and Authentication Key associated with that endpoint. This can be acquired from previous steps. To work with `litellm`, configure the client as follows:\n",
69+
"\n",
70+
"- `base_url`: Use the endpoint URL from your deployment. Include the `/v1` in the URL.\n",
71+
"- `api_key`: Use your API key."
72+
]
73+
},
74+
{
75+
"cell_type": "code",
76+
"execution_count": null,
77+
"metadata": {
78+
"name": "chat_client"
79+
},
80+
"outputs": [],
81+
"source": [
82+
"client = litellm.LiteLLM(\n",
83+
" base_url=\"https://<endpoint-name>.<region>.inference.ai.azure.com/v1\",\n",
84+
" api_key=\"<key>\",\n",
85+
")"
86+
]
87+
},
88+
{
89+
"cell_type": "markdown",
90+
"metadata": {},
91+
"source": [
92+
"Use the client to create chat completions requests:"
93+
]
94+
},
95+
{
96+
"cell_type": "code",
97+
"execution_count": null,
98+
"metadata": {
99+
"name": "chat_invoke"
100+
},
101+
"outputs": [],
102+
"source": [
103+
"response = client.chat.completions.create(\n",
104+
" messages=[{\"content\": \"List the emirates of the UAE.\", \"role\": \"user\"}],\n",
105+
" model=\"openai\",\n",
106+
" custom_llm_provider=\"custom_openai\",\n",
107+
")"
108+
]
109+
},
110+
{
111+
"cell_type": "markdown",
112+
"metadata": {},
113+
"source": [
114+
"The generated text can be accessed as follows:"
115+
]
116+
},
117+
{
118+
"cell_type": "code",
119+
"execution_count": null,
120+
"metadata": {
121+
"name": "chat_response"
122+
},
123+
"outputs": [],
124+
"source": [
125+
"print(response.choices[0].message.content)"
126+
]
127+
},
128+
{
129+
"cell_type": "markdown",
130+
"metadata": {},
131+
"source": [
132+
"## Aditional resources\n",
133+
"\n",
134+
"Here are some additional reference: \n",
135+
"\n",
136+
"* [Plan and manage costs (marketplace)](https://learn.microsoft.com/azure/ai-studio//costs-plan-manage#monitor-costs-for-models-offered-through-the-azure-marketplace)"
137+
]
138+
}
139+
],
140+
"metadata": {
141+
"kernelspec": {
142+
"display_name": "Python 3.10 - SDK v2",
143+
"language": "python",
144+
"name": "python310-sdkv2"
145+
},
146+
"language_info": {
147+
"codemirror_mode": {
148+
"name": "ipython",
149+
"version": 3
150+
},
151+
"file_extension": ".py",
152+
"mimetype": "text/x-python",
153+
"name": "python",
154+
"nbconvert_exporter": "python",
155+
"pygments_lexer": "ipython3",
156+
"version": "3.10.11"
157+
}
158+
},
159+
"nbformat": 4,
160+
"nbformat_minor": 2
161+
}

0 commit comments

Comments
 (0)