Skip to content

Commit 398ef1d

Browse files
ravi9ravi9
committed
Updated to fix sagemaker space issue
1 parent 1ac6174 commit 398ef1d

File tree

3 files changed

+207
-29
lines changed

3 files changed

+207
-29
lines changed

‎aws/mo-utility/create_ir_for_keras.ipynb

+64-9
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,34 @@
11
{
22
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"## Example 1: Convert Keras App Model to OpenVINO IR"
8+
]
9+
},
10+
{
11+
"cell_type": "markdown",
12+
"metadata": {},
13+
"source": [
14+
"This notebook shows how to convert Keras App Model to OpenVINO IR format.\n",
15+
"\n",
16+
"See list of supported models in Keras-SupportedModelList\n",
17+
"\n",
18+
"Following are the steps\n",
19+
"- Install necessary pip packages.\n",
20+
"- In AWS Sagemaker environment, remove few unused conda envs. This will ensure we have enought space for OpenVINO docker images.\n",
21+
"- Select a model from the supported list\n",
22+
"- Run the conversion"
23+
]
24+
},
25+
{
26+
"cell_type": "markdown",
27+
"metadata": {},
28+
"source": [
29+
"### Install necessary pip packages"
30+
]
31+
},
332
{
433
"cell_type": "code",
534
"execution_count": null,
@@ -14,7 +43,23 @@
1443
"cell_type": "markdown",
1544
"metadata": {},
1645
"source": [
17-
"## Example 1: Convert Keras App Model to OpenVINO IR"
46+
"### Removed few unused conda envs. "
47+
]
48+
},
49+
{
50+
"cell_type": "code",
51+
"execution_count": null,
52+
"metadata": {},
53+
"outputs": [],
54+
"source": [
55+
"!bash remove-unused-conda-envs.sh"
56+
]
57+
},
58+
{
59+
"cell_type": "markdown",
60+
"metadata": {},
61+
"source": [
62+
"## Convert Keras App Model to OpenVINO IR"
1863
]
1964
},
2065
{
@@ -33,11 +78,21 @@
3378
"source": [
3479
"### Provide below details:\n",
3580
"\n",
36-
"#### model name to the variable \"keras_app_model_name\"\n",
37-
"#### S3 bucket name with \"sagemaker\" as part of the name. (important for Sagemaker access) to the variable \"bucket_name\". Ex: sagemaker-ir-creation\n",
38-
"#### precision_type can be \"FP32\" or \"FP16\".\n",
81+
"`bucket_name` - S3 bucket name with \"sagemaker\" as part of the name.\n",
82+
"Example: sagemaker-ir-creation\n",
83+
"\n",
84+
"`keras_app_model_name` - Keras app model name. See supported list in Keras-SupportedModelList\n",
85+
"\n",
86+
"`precision_type` - Use either FP32 or FP16\n",
87+
"\n",
88+
"#### NOTE: If the bucket does not exist then it will be created.\n",
3989
"\n",
40-
"#### NOTE: If the bucket does not exist then it will be created."
90+
"#### Example:\n",
91+
"```\n",
92+
"bucket_name = \"sagemaker-ir-creation\"\n",
93+
"keras_app_model_name = \"EfficientNetB0\"\n",
94+
"precision_type = \"FP32\"\n",
95+
"```"
4196
]
4297
},
4398
{
@@ -46,9 +101,9 @@
46101
"metadata": {},
47102
"outputs": [],
48103
"source": [
49-
"keras_app_model_name = \"Your_model_name\"\n",
50-
"bucket_name = \"Your_bucket_name\"\n",
51-
"precision_type = \"Your_precision\""
104+
"bucket_name = \"sagemaker-ir-creation\" # must have sagemaker as part of the name\n",
105+
"keras_app_model_name = \"Pick-supported-model-name\"\n",
106+
"precision_type = \"FP32\" # FP16 also supported"
52107
]
53108
},
54109
{
@@ -112,7 +167,7 @@
112167
"name": "python",
113168
"nbconvert_exporter": "python",
114169
"pygments_lexer": "ipython3",
115-
"version": "3.6.10"
170+
"version": "3.6.13"
116171
}
117172
},
118173
"nbformat": 4,

‎aws/mo-utility/create_ir_for_obj_det.ipynb

+68-10
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,34 @@
11
{
22
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"## Example 3: Convert Object Detection models to OpenVINO IR"
8+
]
9+
},
10+
{
11+
"cell_type": "markdown",
12+
"metadata": {},
13+
"source": [
14+
"This notebook shows how to convert Object Detection models to OpenVINO IR format.\n",
15+
"\n",
16+
"See list of supported models in ObjDet-SupportedModelList\n",
17+
"\n",
18+
"Following are the steps\n",
19+
"- Install necessary pip packages.\n",
20+
"- In AWS Sagemaker environment, remove few unused conda envs. This will ensure we have enought space for OpenVINO docker images.\n",
21+
"- Select a model from the supported list\n",
22+
"- Run the conversion"
23+
]
24+
},
25+
{
26+
"cell_type": "markdown",
27+
"metadata": {},
28+
"source": [
29+
"### Install necessary pip packages"
30+
]
31+
},
332
{
433
"cell_type": "code",
534
"execution_count": null,
@@ -14,7 +43,23 @@
1443
"cell_type": "markdown",
1544
"metadata": {},
1645
"source": [
17-
"## Example 3: Convert Object Detection models to OpenVINO IR"
46+
"### Removed few unused conda envs. "
47+
]
48+
},
49+
{
50+
"cell_type": "code",
51+
"execution_count": null,
52+
"metadata": {},
53+
"outputs": [],
54+
"source": [
55+
"!bash remove-unused-conda-envs.sh"
56+
]
57+
},
58+
{
59+
"cell_type": "markdown",
60+
"metadata": {},
61+
"source": [
62+
"## Convert Object Detection models to OpenVINO IR"
1863
]
1964
},
2065
{
@@ -33,11 +78,24 @@
3378
"source": [
3479
"### Provide below details:\n",
3580
"\n",
36-
"#### URL and json for your model to the variables \"objdet_model_url\" and \"input_json\" respectively.\n",
37-
"#### S3 bucket name with \"sagemaker\" as part of the name (important for Sagemaker access) to the variable \"bucket_name\". Ex: sagemaker-ir-creation\n",
38-
"#### precision_type can be \"FP32\" or \"FP16\".\n",
81+
"`bucket_name` - S3 bucket name with \"sagemaker\" as part of the name.\n",
82+
"Example: sagemaker-ir-creation\n",
83+
"\n",
84+
"`objdet_model_url` - Object detection model URL. See supported list in ObjDet-SupportedModelList\n",
85+
"\n",
86+
"`input_json` - Input json of the selected model. See ObjDet-SupportedModelList\n",
87+
"\n",
88+
"`precision_type` - Use either FP32 or FP16\n",
89+
"\n",
90+
"#### NOTE: If the bucket does not exist then it will be created.\n",
3991
"\n",
40-
"#### NOTE: If the bucket does not exist then it will be created."
92+
"#### Example:\n",
93+
"```\n",
94+
"bucket_name = \"sagemaker-ir-creation\"\n",
95+
"objdet_model_url = \"http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_2018_01_28.tar.gz\"\n",
96+
"input_json = \"ssd_v2_support.json\"\n",
97+
"precision_type = \"FP32\"\n",
98+
"```"
4199
]
42100
},
43101
{
@@ -46,10 +104,10 @@
46104
"metadata": {},
47105
"outputs": [],
48106
"source": [
49-
"objdet_model_url = \"Your_model_URL\"\n",
50-
"input_json = \"Your_json\"\n",
51-
"bucket_name = \"Your_bucket_name\"\n",
52-
"precision_type = \"Your_precision\""
107+
"bucket_name = \"sagemaker-ir-creation\" # must have sagemaker as part of the name\n",
108+
"objdet_model_url = \"Pick-supported-model-url\"\n",
109+
"input_json = \"input_json\"\n",
110+
"precision_type = \"FP32\" # FP16 also supported"
53111
]
54112
},
55113
{
@@ -117,7 +175,7 @@
117175
"name": "python",
118176
"nbconvert_exporter": "python",
119177
"pygments_lexer": "ipython3",
120-
"version": "3.6.10"
178+
"version": "3.6.13"
121179
}
122180
},
123181
"nbformat": 4,

‎aws/mo-utility/create_ir_for_tfhub.ipynb

+75-10
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,34 @@
11
{
22
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"## Example 2: Convert TFHub Models to OpenVINO IR"
8+
]
9+
},
10+
{
11+
"cell_type": "markdown",
12+
"metadata": {},
13+
"source": [
14+
"This notebook shows how to convert TFHub models to OpenVINO IR format.\n",
15+
"\n",
16+
"See list of supported models in TFHub-SupportedModelList\n",
17+
"\n",
18+
"Following are the steps\n",
19+
"- Install necessary pip packages.\n",
20+
"- In AWS Sagemaker environment, remove few unused conda envs. This will ensure we have enought space for OpenVINO docker images.\n",
21+
"- Select a model from the supported list\n",
22+
"- Run the conversion"
23+
]
24+
},
25+
{
26+
"cell_type": "markdown",
27+
"metadata": {},
28+
"source": [
29+
"### Install necessary pip packages"
30+
]
31+
},
332
{
433
"cell_type": "code",
534
"execution_count": null,
@@ -14,7 +43,23 @@
1443
"cell_type": "markdown",
1544
"metadata": {},
1645
"source": [
17-
"## Example 2: Convert TFHUB Models to OpenVINO IR"
46+
"### Removed few unused conda envs. "
47+
]
48+
},
49+
{
50+
"cell_type": "code",
51+
"execution_count": null,
52+
"metadata": {},
53+
"outputs": [],
54+
"source": [
55+
"!bash remove-unused-conda-envs.sh"
56+
]
57+
},
58+
{
59+
"cell_type": "markdown",
60+
"metadata": {},
61+
"source": [
62+
"## Convert TFHub Models to OpenVINO IR"
1863
]
1964
},
2065
{
@@ -33,11 +78,24 @@
3378
"source": [
3479
"### Provide below details:\n",
3580
"\n",
36-
"#### URL and shape for your model to the variables \"tfhub_model_url\" and \"input_shape\" respectively.\n",
37-
"#### S3 bucket name with \"sagemaker\" as part of the name (important for Sagemaker access) to the variable \"bucket_name\". Ex: sagemaker-ir-creation\n",
38-
"#### precision_type can be \"FP32\" or \"FP16\".\n",
81+
"`bucket_name` - S3 bucket name with \"sagemaker\" as part of the name.\n",
82+
"Example: sagemaker-ir-creation\n",
83+
"\n",
84+
"`tfhub_model_url` - TfHub model URL. See supported list in TFHub-SupportedModelList\n",
85+
"\n",
86+
"`input_shape` - Input shape of the selected model. See TFHub-SupportedModelList\n",
3987
"\n",
40-
"#### NOTE: If the bucket does not exist then it will be created."
88+
"`precision_type` - Use either FP32 or FP16\n",
89+
"\n",
90+
"#### NOTE: If the bucket does not exist then it will be created.\n",
91+
"\n",
92+
"#### Example:\n",
93+
"```\n",
94+
"bucket_name = \"sagemaker-ir-creation\"\n",
95+
"tfhub_model_url = \"https://tfhub.dev/google/efficientnet/b0/classification/1\"\n",
96+
"input_shape = \"1,224,224,3\"\n",
97+
"precision_type = \"FP32\"\n",
98+
"```"
4199
]
42100
},
43101
{
@@ -46,10 +104,10 @@
46104
"metadata": {},
47105
"outputs": [],
48106
"source": [
49-
"bucket_name = 'Your_bucket_name'\n",
50-
"tfhub_model_url = 'Your_model_URL'\n",
51-
"input_shape = 'Your_model_shape'\n",
52-
"precision_type = \"Your_precision\""
107+
"bucket_name = \"sagemaker-ir-creation\" # must have sagemaker as part of the name\n",
108+
"tfhub_model_url = \"Pick-supported-model-url\"\n",
109+
"input_shape = \"model-input-shape\"\n",
110+
"precision_type = \"FP32\" # FP16 also supported"
53111
]
54112
},
55113
{
@@ -104,6 +162,13 @@
104162
" print(f\"FAILED: Please find the error details below:\")\n",
105163
" print(err)"
106164
]
165+
},
166+
{
167+
"cell_type": "code",
168+
"execution_count": null,
169+
"metadata": {},
170+
"outputs": [],
171+
"source": []
107172
}
108173
],
109174
"metadata": {
@@ -122,7 +187,7 @@
122187
"name": "python",
123188
"nbconvert_exporter": "python",
124189
"pygments_lexer": "ipython3",
125-
"version": "3.6.10"
190+
"version": "3.6.13"
126191
}
127192
},
128193
"nbformat": 4,

0 commit comments

Comments
 (0)