You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Rename pipeline and pipeline template YAMLs to new convention
* Rename references to pipelines and pipeline templates
* Sort pipeline entries in code_description.md
Copy file name to clipboardExpand all lines: docs/canary_ab_deployment.md
+2-2
Original file line number
Diff line number
Diff line change
@@ -30,7 +30,7 @@ There are some extra variables that you need to setup in ***devopsforai-aml-vg**
30
30
31
31
#### 3. Configure a pipeline to build and deploy a scoring Image
32
32
33
-
Import and run the [azdo-abtest-pipeline.yml](./.pipelines/azdo-abtest-pipeline.yml) multistage deployment pipeline.
33
+
Import and run the [abtest.yml](./.pipelines/abtest.yml) multistage deployment pipeline.
34
34
35
35
The result of the pipeline will be a registered Docker image in the ACR repository attached to the AML Service:
36
36
@@ -46,7 +46,7 @@ model-green 1/1 1 1 19h
46
46
47
47
#### 4. Build a new Scoring Image
48
48
49
-
Change value of the ***SCORE_SCRIPT*** variable in the [azdo-abtest-pipeline.yml](./.pipelines/azdo-abtest-pipeline.yml) to point to ***scoring/scoreA.py*** and merge it to the master branch.
49
+
Change value of the ***SCORE_SCRIPT*** variable in the [abtest.yml](./.pipelines/abtest.yml) to point to ***scoring/scoreA.py*** and merge it to the master branch.
50
50
51
51
**Note:*****scoreA.py*** and ***scoreB.py*** files used in this tutorial are just mockups returning either "New Model A" or "New Model B" respectively. They are used to demonstrate the concept of testing two scoring images with different models or scoring code. In real life you would implement a scoring file similar to [score.py](./../code/scoring/score.py) (see [getting started](./getting_started.md)).
Copy file name to clipboardExpand all lines: docs/code_description.md
+7-10
Original file line number
Diff line number
Diff line change
@@ -43,15 +43,13 @@ The repository provides a template with folders structure suitable for maintaini
43
43
44
44
### Pipelines
45
45
46
-
-`.pipelines/azdo-base-pipeline.yml` : a pipeline template used by ci-build-train pipeline and pr-build-train pipelines. It contains steps performing linting, data and unit testing.
47
-
-`.pipelines/diabetes_regression-ci-build-train.yml` : a pipeline triggered when the code is merged into **master**. It performs linting, data integrity testing, unit testing, building and publishing an ML pipeline.
48
-
-`.pipelines/azdo-pr-build-train.yml` : a pipeline triggered when a **pull request** to the **master** branch is created. It performs linting, data integrity testing and unit testing only.
49
-
-`.pipelines/diabetes_regression-ci-image.yml` : a pipeline building a scoring image for the diabetes regression model.
50
-
-`.pipelines/diabetes_regression-template-get-model-version.yml` : a pipeline template used by the `.pipelines/diabetes_regression-ci-build-train.yml` pipeline. It finds out if a new model was registered and retrieves a version of the new model.
51
-
-`.pipelines/azdo-abtest-pipeline.yml` : a pipeline demonstrating [Canary deployment strategy](./docs/canary_ab_deployment.md).
52
-
-`.pipelines/azdo-helm-*.yml` : pipeline templates used by the `.pipelines/azdo-abtest-pipeline.yml` pipeline.
53
-
54
-
46
+
-`.pipelines/abtest.yml` : a pipeline demonstrating [Canary deployment strategy](./docs/canary_ab_deployment.md).
47
+
-`.pipelines/code-quality-template.yml` : a pipeline template used by the CI and PR pipelines. It contains steps performing linting, data and unit testing.
48
+
-`.pipelines/diabetes_regression-ci-image.yml` : a pipeline building a scoring image for the diabetes regression model.
49
+
-`.pipelines/diabetes_regression-ci.yml` : a pipeline triggered when the code is merged into **master**. It performs linting, data integrity testing, unit testing, building and publishing an ML pipeline.
50
+
-`.pipelines/diabetes_regression-get-model-version-template.yml` : a pipeline template used by the `.pipelines/diabetes_regression-ci.yml` pipeline. It finds out if a new model was registered and retrieves a version of the new model.
51
+
-`.pipelines/helm-*.yml` : pipeline templates used by the `.pipelines/abtest.yml` pipeline.
52
+
-`.pipelines/pr.yml` : a pipeline triggered when a **pull request** to the **master** branch is created. It performs linting, data integrity testing and unit testing only.
55
53
56
54
### ML Services
57
55
@@ -90,4 +88,3 @@ The repository provides a template with folders structure suitable for maintaini
90
88
-`diabetes_regression/scoring/score.py` : a scoring script which is about to be packed into a Docker Image along with a model while being deployed to QA/Prod environment.
91
89
-`diabetes_regression/scoring/inference_config.yml`, `deployment_config_aci.yml`, `deployment_config_aks.yml` : configuration files for the [AML Model Deploy](https://marketplace.visualstudio.com/items?itemName=ms-air-aiagility.private-vss-services-azureml&ssr=false#overview) pipeline task for ACI and AKS deployment targets.
92
90
-`diabetes_regression/scoring/scoreA.py`, `diabetes_regression/scoring/scoreB.py` : simplified scoring files for the [Canary deployment sample](./docs/canary_ab_deployment.md).
Copy file name to clipboardExpand all lines: docs/custom_container.md
+2-2
Original file line number
Diff line number
Diff line change
@@ -57,7 +57,7 @@ run from that image.
57
57
58
58
## Modify the model pipeline
59
59
60
-
Modify the model pipeline file [diabetes_regression-ci-build-train.yml](../.pipelines/diabetes_regression-ci-build-train.yml) by replacing this section:
60
+
Modify the model pipeline file [diabetes_regression-ci.yml](../.pipelines/diabetes_regression-ci.yml) by replacing this section:
61
61
62
62
```
63
63
resources:
@@ -94,6 +94,6 @@ A better approach would be to use a distinct name for your modified environment,
94
94
By changing the name of the image in your branch in both the container build pipeline
Copy file name to clipboardExpand all lines: docs/getting_started.md
+7-7
Original file line number
Diff line number
Diff line change
@@ -58,7 +58,7 @@ the BASE_NAME value should not exceed 10 characters and it should contain number
58
58
59
59
The **RESOURCE_GROUP** parameter is used as the name for the resource group that will hold the Azure resources for the solution. If providing an existing AML Workspace, set this value to the corresponding resource group name.
60
60
61
-
The **AZURE_RM_SVC_CONNECTION** parameter is used by the [Azure DevOps pipeline]((../environment_setup/iac-create-environment.yml)) that creates the Azure ML workspace and associated resources through Azure Resource Manager. The pipeline requires an **Azure Resource Manager**
61
+
The **AZURE_RM_SVC_CONNECTION** parameter is used by the [Azure DevOps pipeline]((../environment_setup/iac-create-environment-pipeline.yml)) that creates the Azure ML workspace and associated resources through Azure Resource Manager. The pipeline requires an **Azure Resource Manager**
@@ -88,7 +88,7 @@ For instructions on how to set up a local development environment, refer to the
88
88
89
89
### Azure DevOps configuration
90
90
91
-
For using Azure DevOps Pipelines all other variables are stored in the file `.pipelines/diabetes_regression-variables.yml`. Using the default values as a starting point, adjust the variables to suit your requirements.
91
+
For using Azure DevOps Pipelines all other variables are stored in the file `.pipelines/diabetes_regression-variables-template.yml`. Using the default values as a starting point, adjust the variables to suit your requirements.
92
92
93
93
**Note:** In `diabetes_regression` folder you can find `config.json` file that we would recommend to use in order to provide parameters for training, evaluation and scoring scripts. An example of a such parameter is a hyperparameter of a training algorithm: in our case it's the ridge regression [*alpha* hyperparameter](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html). We don't provide any special serializers for this config file. So, it's up to you which template to support there.
94
94
@@ -102,7 +102,7 @@ Up until now you should have:
102
102
103
103
The easiest way to create all required resources (Resource Group, ML Workspace,
104
104
Container Registry, Storage Account, etc.) is to leverage an
105
-
"Infrastructure as Code" [pipeline in this repository](../environment_setup/iac-create-environment.yml). This **IaC** pipeline takes care of setting up
105
+
"Infrastructure as Code" [pipeline in this repository](../environment_setup/iac-create-environment-pipeline.yml). This **IaC** pipeline takes care of setting up
106
106
all required resources based on these [ARM templates](../environment_setup/arm-templates/cloud-environment.json).
107
107
108
108
### Create a Build IaC Pipeline
@@ -111,7 +111,7 @@ In your Azure DevOps project, create a build pipeline from your forked repositor
Select the **Existing Azure Pipelines YAML file** option and set the path to [/environment_setup/iac-create-environment.yml](../environment_setup/iac-create-environment.yml):
114
+
Select the **Existing Azure Pipelines YAML file** option and set the path to [/environment_setup/iac-create-environment-pipeline.yml](../environment_setup/iac-create-environment-pipeline.yml):
(Optional) To remove the resources created for this project you can use the [/environment_setup/iac-remove-environment.yml](../environment_setup/iac-remove-environment.yml) definition or you can just delete the resource group in the [Azure Portal](https://portal.azure.com).
126
+
(Optional) To remove the resources created for this project you can use the [/environment_setup/iac-remove-environment-pipeline.yml](../environment_setup/iac-remove-environment-pipeline.yml) definition or you can just delete the resource group in the [Azure Portal](https://portal.azure.com).
127
127
128
128
**Note:** The training ML pipeline uses a [sample diabetes dataset](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_diabetes.html) as training data. To use your own data, you need to [create a Dataset](https://docs.microsoft.com/azure/machine-learning/how-to-create-register-datasets) in your workspace and specify its name in a DATASET_NAME variable in the ***devopsforai-aml-vg*** variable group. You will also need to modify the test cases in the **ml_service/util/smoke_test_scoring_service.py** script to match the schema of the training features in your dataset.
129
129
@@ -157,7 +157,7 @@ performs linting, unit testing and publishes a training pipeline.
157
157
### Set up the Pipeline
158
158
159
159
In your [Azure DevOps](https://dev.azure.com) project create and run a new build
160
-
pipeline referring to the [diabetes_regression-ci-build-train.yml](../.pipelines/diabetes_regression-ci-build-train.yml)
160
+
pipeline referring to the [diabetes_regression-ci.yml](../.pipelines/diabetes_regression-ci.yml)
161
161
pipeline definition in your forked repository:
162
162
163
163
@@ -199,7 +199,7 @@ Wait until the pipeline finishes and verify that there is a new model in the **M
199
199
200
200
201
201
202
-
To disable the automatic trigger of the training pipeline, change the `auto-trigger-training` variable as listed in the `.pipelines\diabetes_regression-ci-build-train.yml` pipeline to `false`. This can also be overridden at runtime execution of the pipeline.
202
+
To disable the automatic trigger of the training pipeline, change the `auto-trigger-training` variable as listed in the `.pipelines\diabetes_regression-ci.yml` pipeline to `false`. This can also be overridden at runtime execution of the pipeline.
203
203
204
204
To skip model training and registration, and deploy a model successfully registered by a previous build (for testing changes to the score file or inference configuration), add the variable `MODEL_BUILD_ID` when the pipeline is queued, and set the value to the id of the previous build.
0 commit comments