diff --git a/.openpublishing.redirection.json b/.openpublishing.redirection.json index 7c122aeeae7..274c0186111 100644 --- a/.openpublishing.redirection.json +++ b/.openpublishing.redirection.json @@ -4973,7 +4973,7 @@ }, { "source_path": "docs/test/manual-exploratory-testing/mtm/share-steps-between-test-cases.md", - "redirect_url": "/previous-versions/azure/devops/test/mtm/share-steps-between-test-cases", + "redirect_url": "/azure/devops/test/share-steps-between-test-cases", "redirect_document_id": false }, { @@ -5258,7 +5258,7 @@ }, { "source_path": "docs/manual-test/mtm/share-steps-between-test-cases.md", - "redirect_url": "/previous-versions/azure/devops/test/mtm/share-steps-between-test-cases", + "redirect_url": "/azure/devops/test/share-steps-between-test-cases", "redirect_document_id": false }, { @@ -17003,7 +17003,7 @@ }, { "source_path": "docs/test/mtm/share-steps-between-test-cases.md", - "redirect_url": "/previous-versions/azure/devops/test/mtm/share-steps-between-test-cases", + "redirect_url": "/azure/devops/test/share-steps-between-test-cases", "redirect_document_id": false }, { @@ -18239,6 +18239,11 @@ "source_path": "docs/test/new-test-plans-page.md", "redirect_url": "/azure/devops/test/navigate-test-plans", "redirect_document_id": false + }, + { + "source_path": "docs/test/preview/new-test-plan-page.md", + "redirect_url": "/azure/devops/test/navigate-test-plans", + "redirect_document_id": false } ] } diff --git a/docs/boards/boards/add-run-update-tests.md b/docs/boards/boards/add-run-update-tests.md index 71deabc4c83..77c6063032c 100644 --- a/docs/boards/boards/add-run-update-tests.md +++ b/docs/boards/boards/add-run-update-tests.md @@ -9,14 +9,14 @@ ms.author: kaelli author: KathrynEE ms.topic: tutorial monikerRange: '>= tfs-2017' -ms.date: 02/14/2019 +ms.date: 09/14/2021 --- # Add, run, and update inline tests [!INCLUDE [temp](../../includes/dev15-and-ts-version-header.md)] -Similar to [task checklists](add-task-checklists.md), you can quickly define inline tests, or a set of manual tests, for a backlog item from your Kanban board. Not only can you add tests, you can run them and update their status. If you're new to working with the Kanban board, see [Kanban basics](kanban-basics.md). +Similar to [task checklists](add-task-checklists.md), you can quickly define inline tests, or a set of manual tests cases, for a backlog item from your Kanban board. Not only can you add tests, you can run them and update their status. If you're new to working with the Kanban board, see [Kanban basics](kanban-basics.md). If you're new to testing, see [Exploratory and manual testing scenarios and capabilities](../../test/overview.md). In this article, you'll learn: > [!div class="checklist"] @@ -29,7 +29,25 @@ In this article, you'll learn: Tests you create from the Kanban board are automatically linked to the user story or backlog item. -[!INCLUDE [temp](../includes/prerequisites-kanban.md)] +## Prerequisites + +::: moniker range="azure-devops" + +* You must connect to a project. If you don't have a project yet, [create one](../get-started/sign-up-invite-teammates.md). +* You must be added to a [team or project](../../organizations/security/add-users-team-project.md). +* To add work items and exercise all board features, you must be granted [**Basic** access or higher](../../organizations/security/access-levels.md). +* To view or modify work items, your **View work items in this node** and **Edit work items in this node** permissions set to **Allow**. By default, the **Contributors** group has this permission set. To learn more, see [Set permissions and access for work tracking](../../organizations/security/set-permissions-access-work-tracking.md). +- To view or run tests, you must have **Basic** access or higher. Users with **Stakeholder** access can't view or run tests. +::: moniker-end + +::: moniker range="< azure-devops" +* You must connect to a project. If you don't have a project yet, [create one](../../organizations/projects/create-project.md). +* You must be added to a [team or project](../../organizations/security/add-users-team-project.md). +* To add work items and exercise all board features, you must be granted [**Basic** access or higher](../../organizations/security/access-levels.md). +* To view or modify work items, your **View work items in this node** and **Edit work items in this node** permissions set to **Allow**. By default, the **Contributors** group has this permission set. To learn more, see [Set permissions and access for work tracking](../../organizations/security/set-permissions-access-work-tracking.md). +- To view or run tests, you must have **Basic** access or higher. Users with **Stakeholder** access can't view or run tests. + +::: moniker-end ## Open your Kanban board from the web portal diff --git a/docs/boards/includes/common-work-item-fields.md b/docs/boards/includes/common-work-item-fields.md index 45095ecb296..7e7ca8c30ee 100644 --- a/docs/boards/includes/common-work-item-fields.md +++ b/docs/boards/includes/common-work-item-fields.md @@ -3,13 +3,13 @@ ms.topic: include --- -## Definitions for common WIT fields +## Definitions for common work tracking fields The following fields and tabs appear in most work items. Each tab is used to track specific information, such as :::image type="icon" source="/azure/devops/boards/backlogs/media/icon-history-tab-wi.png" border="false"::: history, :::image type="icon" source="/azure/devops/boards/backlogs/media/icon-links-tab-wi.png" border="false"::: links, or :::image type="icon" source="/azure/devops/boards/backlogs/media/icon-attachments-tab-wi.png" border="false"::: attachments. These three tabs provide a history of changes, view of linked work items, and ability to view and attach files. -The only required field for all WITs is **Title**. When the work item is saved, the system assigns it a unique **ID**. The form highlights required field in yellow. For information about other fields, see [Work item field index](../work-items/guidance/work-item-field.md). +The only required field for all work item types is **Title**. When the work item is saved, the system assigns it a unique **ID**. The form highlights required field in yellow. For information about other fields, see [Work item field index](../work-items/guidance/work-item-field.md). diff --git a/docs/organizations/security/includes/test.md b/docs/organizations/security/includes/test.md index 0f79a8459b9..9f3398f0a09 100644 --- a/docs/organizations/security/includes/test.md +++ b/docs/organizations/security/includes/test.md @@ -3,14 +3,17 @@ ms.technology: devops-agile ms.author: kaelli author: KathrynEE ms.topic: include -ms.date: 03/11/2021 +ms.date: 09/15/2021 --- :::row::: :::column span="2"::: - **Task or permission** + **Permission** + :::column-end::: + :::column span="1"::: + **Level** :::column-end::: :::column span="1"::: **Readers** @@ -24,22 +27,10 @@ ms.date: 03/11/2021 :::row-end::: :::row::: :::column span="2"::: - **Create test runs** (Project-level permission) - **Delete test runs** (Project-level permission) - :::column-end::: - :::column span="1"::: - + **View test runs** :::column-end::: :::column span="1"::: - ✔️ - :::column-end::: - :::column span="1"::: - ✔️ - :::column-end::: -:::row-end::: -:::row::: - :::column span="2"::: - Provide feedback using the Test & Feedback extension + Project-level :::column-end::: :::column span="1"::: ✔️ @@ -53,7 +44,11 @@ ms.date: 03/11/2021 :::row-end::: :::row::: :::column span="2"::: - Request feedback using the Test & Feedback extension + **Create test runs** + **Delete test runs** + :::column-end::: + :::column span="1"::: + Project-level :::column-end::: :::column span="1"::: @@ -65,12 +60,17 @@ ms.date: 03/11/2021 ✔️ :::column-end::: :::row-end::: + :::row::: :::column span="2"::: - **View test runs** (Project-level permission) + **Manage test configurations** + **Manage test environments** :::column-end::: :::column span="1"::: - ✔️ + Project-level + :::column-end::: + :::column span="1"::: + :::column-end::: :::column span="1"::: ✔️ @@ -81,14 +81,14 @@ ms.date: 03/11/2021 :::row-end::: :::row::: :::column span="2"::: - **Manage test plans** (Area Path permission) - **Manage test suites** (Area Path permission) - **Manage test configurations** (Project-level permission) - **Manage test environments** (Project-level permission) - (Includes add and delete operations) + **Create tag definition** + **Delete and restore work items** :::column-end::: :::column span="1"::: - + Project-level + :::column-end::: + :::column span="1"::: + :::column-end::: :::column span="1"::: ✔️ @@ -99,25 +99,30 @@ ms.date: 03/11/2021 :::row-end::: :::row::: :::column span="2"::: - Exploratory testing + **Permanently delete work items** + :::column-end::: + :::column span="1"::: + Project-level :::column-end::: :::column span="1"::: :::column-end::: :::column span="1"::: - ✔️ + :::column-end::: :::column span="1"::: ✔️ :::column-end::: :::row-end::: -::: moniker range=">= tfs-2017" :::row::: :::column span="2"::: - Apply a work item template to a test case + **View work items in this node** :::column-end::: :::column span="1"::: - + Area Path + :::column-end::: + :::column span="1"::: + ✔️ :::column-end::: :::column span="1"::: ✔️ @@ -126,14 +131,17 @@ ms.date: 03/11/2021 ✔️ :::column-end::: :::row-end::: -::: moniker-end -::: moniker range=">= tfs-2015" :::row::: :::column span="2"::: - **Delete test runs** + **Edit work items in this node** + **Manage test plans** + **Manage test suites** :::column-end::: :::column span="1"::: - + Area Path + :::column-end::: + :::column span="1"::: + :::column-end::: :::column span="1"::: ✔️ @@ -142,6 +150,9 @@ ms.date: 03/11/2021 ✔️ :::column-end::: :::row-end::: -::: moniker-end - + +::: moniker range=">= azure-devops-2019" +> [!NOTE] +> The **Change work item type** permission doesn't apply to test-specific work items. Even if you choose this feature from the work item form, changing the work item type is disallowed. +::: moniker-end \ No newline at end of file diff --git a/docs/organizations/security/media/test-permissions/project-level-permissions-2020-earlier.png b/docs/organizations/security/media/test-permissions/project-level-permissions-2020-earlier.png new file mode 100644 index 00000000000..e81eff57bcb Binary files /dev/null and b/docs/organizations/security/media/test-permissions/project-level-permissions-2020-earlier.png differ diff --git a/docs/organizations/security/permissions.md b/docs/organizations/security/permissions.md index dd9971189f4..1672051e282 100644 --- a/docs/organizations/security/permissions.md +++ b/docs/organizations/security/permissions.md @@ -24,217 +24,73 @@ For a quick reference to default assignments, see [Default permissions and acces To learn how to add users to a group or set a specific permission that you can manage through the web portal, see the following resources: -::: moniker range="azure-devops" - -

Field/tab

Usage

- - - - - - - -
-

Users and groups

- -

Wiki

- -		 -

DevOps permissions

- -		 -

Work tracking

- -

Reporting permissions

- -
- -::: moniker-end - -::: moniker range="azure-devops-2019 || azure-devops-2020" - - - - - - - - - - -
-

Users and groups

- -

Wiki

- -		 -

DevOps permissions

- -		 -

Work tracking

- -

Reporting permissions

- -
- - -::: moniker-end - - -::: moniker range="tfs-2018" - - - - - - - - - -
-

Users and groups

- -

Wiki

- -		 -

DevOps permissions

- -		 -

Work tracking

- -

Reporting permissions

- -
- -::: moniker-end - - - -::: moniker range="<= tfs-2017" - - - - - - - - - - -
-

Users and groups

- -		 -

DevOps permissions

- -		 -

Work tracking

- -

Reporting permissions

- -
- -::: moniker-end +--- +:::row::: + :::column span="1"::: + **Users and groups** + ::: moniker range="azure-devops" + -[Add users to an administrator role](set-project-collection-level-permissions.md) + -[Add users to an organization](../accounts/add-organization-users.md) + -[Add users to a project or a team](add-users-team-project.md) + -[Add a team admin](../settings/add-team-administrator.md) + ::: moniker-end + ::: moniker range="< azure-devops" + -[Add a server-level administrator](/azure/devops/server/admin/add-administrator) + -[Add users to an administrator role](set-project-collection-level-permissions.md) + -[Add users to a project or team](add-users-team-project.md) + -[Add a team admin](../settings/add-team-administrator.md) + ::: moniker-end + --- + **Wiki** + -[README & Wiki](../../project/wiki/manage-readme-wiki-permissions.md) + :::column-end::: + :::column span="1"::: + *DevOps ** + -[Git branch](../../repos/git/branch-permissions.md) + -[Git repositories](../../repos/git/set-git-repository-permissions.md) + -[TFVC](../../repos/tfvc/set-tfvc-repository-permissions.md) + -[Build and release pipelines](../../pipelines/policies/set-permissions.md) + -[Approvals and approvers](../../pipelines/release/approvals/index.md) + -[Task groups](../../pipelines/policies/set-permissions.md) + -[Variable groups](../../pipelines/policies/set-permissions.md) + -[Role-based resources](../../pipelines/policies/set-permissions.md) + --- + **Wiki** + :::column-end::: + :::column span="1"::: + **Work tracking** + ::: moniker range=">= tfs-2017" + -[Area and iteration paths](set-permissions-access-work-tracking.md) + -[Queries and folders](../../boards/queries/set-query-permissions.md) + -[Plan permissions](../../boards/plans/edit-delivery-plan-permissions.md) + -[Customize process](set-permissions-access-work-tracking.md#process-permissions) + ::: moniker-end + ::: moniker range="< tfs-2017" + -[Area and iteration paths](set-permissions-access-work-tracking.md) + -[Queries and folders](../../boards/queries/set-query-permissions.md) + -[Customize process](set-permissions-access-work-tracking.md#process-permissions) + ::: moniker-end + --- + **Reporting** + ::: moniker range="azure-devops" + -[Dashboard permissions](../../report/dashboards/dashboard-permissions.md#set-permissions) + -[Analytics](../../report/powerbi/analytics-security.md) + -[Analytics views](../../report/powerbi/analytics-security.md) + ::: moniker-end + ::: moniker range="> tfs-2018 < azure-devops" + -[Dashboard permissions](../../report/dashboards/dashboard-permissions.md#set-permissions) + -[Analytics](../../report/powerbi/analytics-security.md) + -[Analytics views](../../report/powerbi/analytics-security.md) + -[SQL Server Reports](../../report/admin/grant-permissions-to-reports.md) + ::: moniker-end + ::: moniker range="<= tfs-2018" + -[Dashboard permissions](../../report/dashboards/dashboard-permissions.md#set-permissions) + -[SQL Server Reports](../../report/admin/grant-permissions-to-reports.md) + ::: moniker-end + :::column-end::: +:::row-end::: +--- + @@ -696,12 +552,6 @@ Release Administrators (TFS 2017, Azure DevOps) For each project that you create, the system creates the followings project-level groups. These groups are assigned [project-level permissions](#team-project-level-permissions). -> The full name of each of these groups is **[{project name}]\\{group name}**. -> For example, the contributors group for a project called "My Project" is -> **[My Project]\\Contributors**. - - - ::: moniker range="azure-devops" @@ -731,6 +581,12 @@ For each project that you create, the system creates the followings project-leve ::: moniker-end + +> [!TIP] +> The full name of each of these groups is **[{project name}]\\{group name}**. +> For example, the contributors group for a project called "My Project" is +> **[My Project]\\Contributors**. + :::row::: :::column span="1"::: **Group name** @@ -1545,17 +1401,7 @@ You manage project-level permissions through the [web portal admin context](set- Edit project-level information - Can edit project level permissions for users and groups. -
-

- Edit project-level information includes the ability - to perform these tasks for the project: -

- -
+ Can edit [project description](../projects/project-vision-status.md) and [project services visibility](../settings/set-services.md). @@ -1579,7 +1425,7 @@ You manage project-level permissions through the [web portal admin context](set- View project-level information - Can view project level group membership and permissions. + Can view project-level information, including security information group membership and permissions.

Boards diff --git a/docs/organizations/security/set-permissions-access-test.md b/docs/organizations/security/set-permissions-access-test.md index 21e575c812f..bb4a46222eb 100644 --- a/docs/organizations/security/set-permissions-access-test.md +++ b/docs/organizations/security/set-permissions-access-test.md @@ -1,54 +1,86 @@ --- title: Set permissions and access for manual testing titleSuffix: Azure DevOps -description: How to grant or restrict access to manual test tasks for Azure DevOps & TFS +description: How to grant or restrict access to test plans, test suites, test cases, and other test-related features. ms.technology: devops-security ms.assetid: ms.author: kaelli author: KathrynEE ms.topic: conceptual monikerRange: '<= azure-devops' -ms.date: 01/27/2020 +ms.date: 09/13/2021 --- # Set permissions and access for testing [!INCLUDE [version-all](../../includes/version-all.md)] -You grant or restrict access to various manual test features by granting users or groups specific permissions for an object or project. +To exercise the full features of Azure Test Plans, you must be granted [Basic + Test Plans](https://marketplace.visualstudio.com/items?itemName=ms.vss-testmanager-web) access level or have one of the following subscriptions: -You set manual test permissions for area paths and at the project-level. You set permissions to manage test controllers at the organization or collection level. Test controllers are used in performing load tests. +- [Enterprise](https://visualstudio.microsoft.com/vs/enterprise/) +- [Test Professional](https://visualstudio.microsoft.com/vs/test-professional/) +- [MSDN Platforms](https://visualstudio.microsoft.com/msdn-platforms/) -::: moniker range=">= tfs-2017" +In addition, you can grant or restrict access to various manual test features by granting users or groups specific permissions for an object or project. Many test artifacts correspond to test-specific work item types. So, work-tracking permissions apply to test-specific work items, such as test plans, test suites, test cases and more. You set permissions for work items and manual testing features for area paths and at the project-level. You set permissions to manage test controllers at the organization or collection level. Test controllers are used in performing load tests. - **Object-level, Area path level** - - Manage test plans - - Manage test suites + - **Edit work items in this node**: Add or edit test-specific work items, such as test plans, test suites, test cases, shared steps, or shared parameters. + - **Manage test plans**: Modify test plan properties such as build and test settings. + - **Manage test suites**: Create and delete test suites, add, and remove test cases from test suites, change test configurations associated with test suites, and modify a test suite hierarchy (move a test suite). - **Project-level** - - Manage test configurations - - Manage test environments - - Create test runs - - Delete test runs + ::: moniker range=">= tfs-2017" + - **Manage test configurations**: Add or edit [test configurations and configuration variables](../../test/test-different-configurations.md). + - **Manage test environments**: Add or edit [test plan settings](../../test/run-automated-tests-from-test-hub.md). + - **Create test runs**: [Run manual tests](../../test/run-manual-tests.md) + - **Delete and restore work items**: [Delete test-specific work items](../../boards/backlogs/remove-delete-work-items.md) + - **Delete test runs**: [Delete test results](../../boards/backlogs/delete-test-artifacts.md) + - **Manage test configurations**: Add or edit [test configurations and configuration variables](../../test/test-different-configurations.md). + - **Manage test environments**: Add or edit [test plan settings](../../test/run-automated-tests-from-test-hub.md). + - **Move work items out of this project**: [Move work items from one project to another](../../boards/backlogs/move-change-type.md) + - **Permanently delete work items**: [Permanently delete test-specific work items](../../boards/backlogs/remove-delete-work-items.md) + ::: moniker-end + ::: moniker range="< tfs-2017" + - **Manage test configurations**: Add or edit [test configurations and configuration variables](../../test/test-different-configurations.md). + - **Manage test environments**: Add or edit [test plan settings](../../test/run-automated-tests-from-test-hub.md). + - **Create test runs**: [Run manual tests](../../test/run-manual-tests.md) + - **Delete and restore work items**: [Delete test-specific work items](../../boards/backlogs/remove-delete-work-items.md) + - **Manage test configurations**: Add or edit [test configurations and configuration variables](../../test/test-different-configurations.md). + - **Manage test environments**: Add or edit [test plan settings](../../test/run-automated-tests-from-test-hub.md). + - **Move work items out of this project**: [Move work items from one project to another](../../boards/backlogs/move-change-type.md) + - **Permanently delete work items**: [Permanently delete test-specific work items](../../boards/backlogs/remove-delete-work-items.md) + ::: moniker-end - **Organization or collection-level** - - Manage test controllers + - **Manage test controllers**: Permission associated with a deprecated feature for TFS 2018 and later versions. To learn more, see [Overview of test agents and test controllers for running load tests](/visualstudio/test/configure-test-agents-and-controllers-for-load-tests) and [Install test agents and test controllers](/visualstudio/test/lab-management/install-configure-test-agents). +## Prerequisites + +::: moniker range="azure-devops" + +- To manage access levels or organization-level permissions, you must be a member of the **Project Collection Administrators** security group, or have your **Edit instance-level information** set to **Allow**. +- To manage project or object-level test-related permissions, you must be a member of the **Project Administrators** security group. + +For more information, see the following articles: + +- [About access levels](access-levels.md) +- [Add organization users and manage access](../accounts/add-organization-users.md) +- [Change access levels for users or groups](change-access-levels.md) +- [Set permissions at the project- or collection-level](set-project-collection-level-permissions.md) ::: moniker-end -::: moniker range="< tfs-2017" +::: moniker range="< azure-devops" +- To manage access-levels, you must be a member of the **Azure DevOps Server Administrators** group. +- To manage project or object-level test-related permissions, you must be a member of the **Project Administrators** security group. +- To manage collection-level permissions or manage access levels, you must be a member of the **Project Collection Administrators** security group, or have your **Edit instance-level information** set to **Allow**. + -- **Object-level, Area path level** - - Manage test plans - - Manage test suites -- **Project-level** - - Manage test configurations - - Manage test environments - - Create test runs -- **Organization or collection-level** - - Manage test controllers +For more information, see the following articles: +- [About access levels](access-levels.md) +- [Add a server-level administrator](/azure/devops/server/admin/add-administrator) +- [Change access levels for users or groups](change-access-levels.md) +- [Set permissions at the project- or collection-level](set-project-collection-level-permissions.md) ::: moniker-end - ## Grant access to manual testing features @@ -113,7 +145,7 @@ As a project admin you can grant a user, team group, or other group you've creat > [!NOTE] > To enable the new user interface for the Project Permissions Settings Page, see [Enable preview features](../../project/navigation/preview-features.md). -In this example, we grant members assigned to the Test Admin group permissions to create and view test runs and manage test configurations and environments. +In this example, we grant members assigned to the Team Admin group permissions to create and view test runs and manage test configurations and environments. #### [Preview page](#tab/preview-page) @@ -122,7 +154,7 @@ In this example, we grant members assigned to the Test Admin group permissions t #### [Current page](#tab/current-page) > [!div class="mx-imgBorder"] -> ![Screenshot of Set project-level test permissions for a custom group, Team Admin.](media/test-permissions/set-project-level-test-permissions-new-ui.png) +> ![Screenshot of Set project-level test permissions for a custom group, Team Admin.](media/test-permissions/project-level-permissions-2020-earlier.png) ::: moniker-end diff --git a/docs/project/toc.yml b/docs/project/toc.yml index 9b3740e425d..e2afd8d65f1 100644 --- a/docs/project/toc.yml +++ b/docs/project/toc.yml @@ -46,6 +46,8 @@ href: navigation/toc.yml - name: Notifications href: ../notifications/toc.yml +- name: Feedback + href: ../feedback/toc.yml - name: REST API Reference items: - name: Projects diff --git a/docs/test/add-to-bugs-exploratory-testing.md b/docs/test/add-to-bugs-exploratory-testing.md index b86c69cc50f..6eca29d6602 100644 --- a/docs/test/add-to-bugs-exploratory-testing.md +++ b/docs/test/add-to-bugs-exploratory-testing.md @@ -6,8 +6,8 @@ ms.technology: devops-test ms.topic: conceptual ms.author: sdanie author: steved0x -ms.date: 12/07/2018 monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- # Add findings to existing bugs with exploratory testing @@ -19,7 +19,7 @@ searches for and displays existing bugs, based on the keywords in the title, as you file a new bug. You can choose to continue creating a new bug or add your findings to an existing bug. -[!INCLUDE [feature-availability](includes/feature-availability.md)] +[!INCLUDE [prerequisites-define](includes/prerequisites-stakeholder.md)] 1. As you type the title for a new bug, in the background the extension searches for similar bugs that might be related to the issue you've found @@ -71,5 +71,4 @@ After you file bugs, create tasks, or create test cases, all these show up in th * [Use the Test & Feedback extension in Standalone mode](standalone-mode-exploratory-testing.md) * [Exploratory testing with Microsoft Test Manager](/previous-versions/azure/devops/test/mtm/exploratory-testing-using-microsoft-test-manager) * [Overview of manual and exploratory testing](index.yml) - -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + \ No newline at end of file diff --git a/docs/test/associate-automated-test-with-test-case.md b/docs/test/associate-automated-test-with-test-case.md index 1b9c37c65db..9802732e37d 100644 --- a/docs/test/associate-automated-test-with-test-case.md +++ b/docs/test/associate-automated-test-with-test-case.md @@ -20,7 +20,7 @@ Consider using Visual Studio to associate automated tests with a test case when: to automate, but you still want to be able to run that test as part of a test plan. Tests can be run in the CI/CD pipeline by choosing the test plan or test suite in the settings of the [Visual Studio Test](../pipelines/tasks/test/vstest.md) task. - Automated tests can also be run from [!INCLUDE [test-hub-include](includes/test-hub-include.md)]. + Automated tests can also be run from the **Test Plans** web portal. If you are using XAML builds you can also [run these automated tests by using Microsoft Test Manager](/previous-versions/azure/devops/test/mtm/run-automated-tests-with-microsoft-test-manager). @@ -67,7 +67,7 @@ and [Use UI Automation To Test Your Code](/visualstudio/test/use-ui-automation-t 1. Open your solution in Visual Studio Enterprise or Professional 2017 or a later version. 2. If you don't know the identifier of the work item for the test case, - locate the test case in [!INCLUDE [test-hub-include-nolink](includes/test-hub-include-nolink.md)], or [query for the work item](../boards/queries/using-queries.md) in the **Work** hub. + locate the test case in **Azure Test Plans** , or [query for the work item](../boards/queries/using-queries.md) in the **Work** hub. 3. When you know the identifier of the work item for the test case: @@ -134,4 +134,3 @@ change the **Work Items | General** setting from the **Tools | Options** menu in * [Run automated tests from test plans](run-automated-tests-from-test-hub.md) * [Run automated tests with Microsoft Test Manager](/previous-versions/azure/devops/test/mtm/run-automated-tests-with-microsoft-test-manager) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] \ No newline at end of file diff --git a/docs/test/collect-diagnostic-data.md b/docs/test/collect-diagnostic-data.md index 94280466eca..b64502091a1 100644 --- a/docs/test/collect-diagnostic-data.md +++ b/docs/test/collect-diagnostic-data.md @@ -6,8 +6,8 @@ ms.technology: devops-test ms.topic: conceptual ms.author: sdanie author: steved0x -ms.date: 12/07/2018 -monikerRange: '>= tfs-2015' +monikerRange: '>= tfs-2017' +ms.date: 09/14/2021 --- # Collect diagnostic data while testing @@ -20,9 +20,10 @@ during the test. You can collect diagnostic data from web apps and from desktop apps, and view it in Azure Test Plans or Team Foundation Server. -[!INCLUDE [feature-availability](includes/feature-availability.md)] +[!INCLUDE [prerequisites-define](includes/prerequisites-stakeholder.md)] + ## Collect diagnostic data from web and desktop apps For web apps under test, you can use web-based Microsoft Test Runner. @@ -124,6 +125,7 @@ Capture screen recordings from your apps: ![Viewing the screen recording](media/collect-diagnostic-data/collect-diagnostic-data-14.png) + ### View the diagnostic data When you create a bug while capturing diagnostic data, all the data captured @@ -148,9 +150,9 @@ tests using Microsoft Test Manager. For more details, see > You must [configure the data collectors](/previous-versions/azure/devops/test/mtm/collect-more-diagnostic-data-in-manual-tests) and other run settings in Microsoft Test Manager and specify these as the default settings for the test plan. > For more details, see [Run manual tests with Microsoft Test Manager](/previous-versions/azure/devops/test/mtm/plan-manual-tests-with-microsoft-test-manager). -## See also +## Related articles * [Exploratory test and submit feedback directly from your browser](perform-exploratory-tests.md) * [Overview of manual and exploratory testing](index.yml) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/connected-mode-exploratory-testing.md b/docs/test/connected-mode-exploratory-testing.md index 44c8fdf6e27..ecdc8447f86 100644 --- a/docs/test/connected-mode-exploratory-testing.md +++ b/docs/test/connected-mode-exploratory-testing.md @@ -6,8 +6,8 @@ ms.technology: devops-test ms.topic: quickstart ms.author: sdanie author: steved0x -ms.date: 12/07/2018 monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- # Exploratory testing with the Test & Feedback extension in Connected mode @@ -15,7 +15,7 @@ monikerRange: '>= tfs-2015' [!INCLUDE [version-header](includes/version-header.md)] To use the Test & Feedback extension in **Connected** mode you must connect -to Azure DevOps or TFS 2015 and later. +to an Azure DevOps project. This automatically configures the extension based on your access level: * Users with **Basic** access can use the extension to perform exploratory @@ -29,10 +29,11 @@ This automatically configures the extension based on your access level: by the team by choosing the **Provide feedback** link in the email. [More details](provide-stakeholder-feedback.md#email). -[!INCLUDE [feature-availability](includes/feature-availability.md)] +[!INCLUDE [prerequisites-define](includes/prerequisites-stakeholder.md)] -## Connect to Azure DevOps or TFS + +## Connect to Azure DevOps 1. If you want to use Azure DevOps, and you haven't already done so, [sign up for a subscription](https://visualstudio.microsoft.com/products/visual-studio-team-services-vs) @@ -164,7 +165,7 @@ The extension lets you create test cases as you explore your application. ![View the exploratory testing session](media/connected-mode-exploratory-testing/create-bugs-06.png) - Alternatively, open the **Recent exploratory sessions** list directly in the **Runs** page of [!INCLUDE [test-hub-include](includes/test-hub-include.md)]. + Alternatively, open the **Recent exploratory sessions** list directly in the **Runs** page of the **Test Plans** web portal. ![View exploratory testing sessions on the server](media/connected-mode-exploratory-testing/create-bugs-07.png) diff --git a/docs/test/create-a-test-plan.md b/docs/test/create-a-test-plan.md index f233f4404c8..c85a9c1ffa6 100644 --- a/docs/test/create-a-test-plan.md +++ b/docs/test/create-a-test-plan.md @@ -6,8 +6,8 @@ ms.technology: devops-test ms.topic: quickstart ms.author: sdanie author: steved0x -ms.date: 11/15/2019 monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- # Create test plans and test suites @@ -18,90 +18,17 @@ Create test plans and test suites to track manual testing for sprints or milesto That way, you can see when the testing for a specific sprint or milestone is complete. For more information about manual testing, see [Planned manual testing](overview.md#planned-manual-testing) and [Automated and Manual Testing with Azure Test Plans](https://www.youtube.com/watch?v=LF0hmSysWCg). - +[!INCLUDE [prerequisites-define](includes/prerequisites-define.md)] -## What are test plans, test suites, and test cases? - -[!INCLUDE [test-hub-include](includes/test-hub-include.md)] provides three main types of test management artifacts: test plans, test suites, and test cases. -These elements are stored in your work repository as special types of work items. -You can export and share them with your team, and benefit from close integration for all of your DevOps tasks. - -* **Test plans** group test suites and individual test cases together. - Test plans include static test suites, requirement-based suites, and [query-based suites](reference-qa.md#query-based-suites). - -* **Test suites** group test cases into separate testing scenarios within a single test plan. - Grouping test cases makes it easier to see which scenarios are complete. - -* **Test cases** validate individual parts of your code or app deployment. - You can ensure your code works correctly, has no errors, and meets business and customer requirements. - You can add individual test cases to a test plan without creating a test suite, if you wish. - More than one test suite or test plan can refer to a test case. - You can effectively reuse test cases without needing to copy or clone them for each suite or plan. - See [Create manual test cases](create-test-cases.md). - -For example, you're building version 1.* of your product and you might create several test cases for that version. -Each of these test cases can be updated, and more added, at any time. -For each development cycle and release of your product, you create a test plan and import the existing test cases into that plan. -You can also, if you wish, divide the test cases into separate test suites within the plan to enable easier management and monitoring of these separate sets of test cases. - -After you create your test plan, you [assign test configurations](test-different-configurations.md) and [assign testers](create-test-cases.md#assign-testers) to cover the required test matrix. -These testers [run the tests](run-manual-tests.md) and gauge the quality of the product. -Testers continue testing until the product meets exit criteria. -For the next development cycle and release, you can create a new test plan and reuse the same test cases. -You repeat this development-test-release cycle by importing the same test cases into each new test plan. - -The great thing is that, because test plans refer to test cases, updates to a test case automatically reflect in all the test plans and test suites that use it. - -In the next version of the product, you can reuse the existing test cases. -However, a better option may be to [copy or clone the test cases](reference-qa.md#creating-manual-test-cases). A copy creates a new baseline. Changes to these new test cases don't affect your previous test plans. - -> [!TIP] -> For more information about the ways you can work with test plans, test suites, and test cases, see the [FAQs for manual testing](reference-qa.md#testplans). - -[!INCLUDE [feature-availability](includes/feature-availability.md)] - -## Prerequisites - -To manage test artifacts, you must have the correct permissions and access level. - -### Manage test artifacts permissions - -You must be a member of the Project Administrators group or have Manage Test Plans and Manage Test Suites [permissions](../organizations/security/set-permissions-access-test.md). - -### Access level - -::: moniker range="< azure-devops-2019" - -To create and edit a test artifact, you must have your [access level set to Advanced](../organizations/security/access-levels.md). -These artifacts include test plan, test suite, test case, test configuration, shared step, and shared parameter. - -::: moniker-end - -::: moniker range=">= azure-devops-2019" - -To create and edit a test artifact, you must have one of the following access levels: - -- [Basic + Test plans](../organizations/security/access-levels.md) -- [Visual Studio Enterprise](../organizations/security/access-levels.md) -- [Basic](../organizations/security/access-levels.md) edit test cases, but not create them or other types of test artifacts - -Test artifacts include test plan, test suite, test case, test configuration, shared step, and shared parameter. - ->[!NOTE] -> Stakeholders can't create or manage test cases. You must have at least Basic access. -See [Default manual testing permissions and access](manual-test-permissions.md). - -::: moniker-end +[!INCLUDE [note-new-ui](includes/note-new-ui.md)] ## Create a test plan -If you want to use Azure Test Plans, and haven't done so already, [sign up for Azure DevOps Services](https://visualstudio.microsoft.com/products/visual-studio-team-services-vs). +In general, you create test plans to test requirements. Prior to creating a test plan, therefore, you may want to [define your backlog of requirements](../boards/backlogs/create-your-backlog.md). -You need a project. For more information, see [create your project](../organizations/accounts/organization-management.md). Then [create your backlog](../boards/backlogs/create-your-backlog.md). - -1. In Azure DevOps Services or Azure DevOps Server, open your project and go to [!INCLUDE [test-hub-include](includes/test-hub-include.md)]. +1. From the web portal, open your project and go to **Test Plans**. If you already have a test plan, choose **Test Plans** to go to the page that lists all test plans. ![Opening the list of test plans page](media/create-a-test-plan/goto-test-plans-page.png) @@ -115,11 +42,9 @@ You need a project. For more information, see [create your project](../organizat ![Adding test plan details](media/create-a-test-plan/CreateATestPlan2.png) -> [!TIP] -> You can also create a test plan and a test suite automatically when you create a test from a User Story work item in Azure Boards or Azure DevOps Server. -> Open the shortcut menu for the work item and choose **Add test**. + ## Add a requirement-based test suite and select backlog items to test Now add test suites for the backlog items that need manual tests. These tests could be user stories, requirements, or other work items based your project. @@ -143,6 +68,7 @@ Now add test suites for the backlog items that need manual tests. These tests co ![Adding requirement-based suites for your backlog items](media/create-a-test-plan/AddRequirementSuitesToTestPlan3.png) + ## Find a test case in a test plan In **Test Plans** for your test plan, use the :::image type="icon" source="media/create-a-test-plan/filter-icon.png" border="false"::: filter icon (**1**) to show the search and filter lists (**2**) that help you find the tests you want to work with. @@ -150,12 +76,37 @@ Or filter the list of tests using **Outcome**, **Tester**, and **Configuration** ![Finding a test plan](media/create-a-test-plan/select-test-plan.png) + +## Share work items across your test experience +**** +For example, you're building version 1.* of your product and you might create several test cases for that version. +Each of these test cases can be updated, and more added, at any time. +For each development cycle and release of your product, you create a test plan and import the existing test cases into that plan. +You can also, if you wish, divide the test cases into separate test suites within the plan to enable easier management and monitoring of these separate sets of test cases. + +After you create your test plan, you [assign test configurations](test-different-configurations.md) and [assign testers](create-test-cases.md#assign-testers) to cover the required test matrix. +These testers [run the tests](run-manual-tests.md) and gauge the quality of the product. +Testers continue testing until the product meets exit criteria. +For the next development cycle and release, you can create a new test plan and reuse the same test cases. +You repeat this development-test-release cycle by importing the same test cases into each new test plan. + +The great thing is that, because test plans refer to test cases, updates to a test case automatically reflect in all the test plans and test suites that use it. + +In the next version of the product, you can reuse the existing test cases. +However, a better option may be to [copy or clone the test cases](reference-qa.md#creating-manual-test-cases). A copy creates a new baseline. Changes to these new test cases don't affect your previous test plans. + +> [!TIP] +> For more information about the ways you can work with test plans, test suites, and test cases, see the [FAQs for manual testing](reference-qa.md#testplans). + + + ## Next steps > [!div class="nextstepaction"] > [Create manual test cases](create-test-cases.md#test-cases) -## See also +## Related articles +* [Test objects and terms](test-objects-overview.md) * [FAQs for manual testing](reference-qa.md#testplans) * [Link test cases to work items](../boards/queries/link-work-items-support-traceability.md) diff --git a/docs/test/create-test-cases.md b/docs/test/create-test-cases.md index 523c16893b8..d0c5901de0f 100644 --- a/docs/test/create-test-cases.md +++ b/docs/test/create-test-cases.md @@ -1,15 +1,17 @@ --- -title: Create manual tests -description: Test tools - Create manual tests in Azure DevOps and TFS to make sure each of the deliverables meets your users needs +title: Create manual test cases +titleSuffix: Azure Test Plans +description: Create manual test cases to test your deliverables ms.assetid: C3C10A82-C7F2-4AB6-9CED-B43DAF722800 ms.technology: devops-test ms.topic: quickstart ms.author: sdanie author: steved0x -ms.date: 12/07/2018 monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- + # Create manual test cases [!INCLUDE [version-header](includes/version-header.md)] @@ -20,24 +22,27 @@ meet your users' needs. Organize your test cases by adding them to test plans and test suites. Then choose which testers you want to run the tests. -[What are test plans, test suites, and test cases?](create-a-test-plan.md#testplans) - -[!INCLUDE [feature-availability](includes/feature-availability.md)] +For an overview of test artifacts, see [Test objects and terms](test-objects-overview.md). > [!NOTE] -> Stakeholders cannot create or manage test cases. You must have at least Basic access. -> See [Default manual testing permissions and access](manual-test-permissions.md). +> Test iterations are designed to support data-driven scenarios, not workflow-driven scenarios. + From a best practice perspective, if you have two test scenarios where the workflows are + different, consider creating separate test cases. Also see [FAQs for manual testing](reference-qa.md#testcases). + +[!INCLUDE [prerequisites-define](includes/prerequisites-define.md)] + +[!INCLUDE [note-new-ui](includes/note-new-ui.md)] + ## Create test cases -1. If you haven't already, - [create a test plan and requirement-based test suites](create-a-test-plan.md). +1. If you haven't already, [create a test plan and requirement-based test suites](create-a-test-plan.md). 1. Select a requirement-based test suite. In the right-hand pane, choose **+** (New) and then choose **New test case**. - ![Selecting the test suite for a backlog item and adding a new test case](media/create-test-cases/CreateTest_1.png) + ![Select the test suite for a backlog item and adding a new test case](media/create-test-cases/CreateTest_1.png) > The [test suite](create-a-test-plan.md) shown here was created from a User Story work item in the team's backlog board. > When you add a test case to this kind of suite, the test case is automatically linked to the backlog item. @@ -55,11 +60,14 @@ the tests. [How do I add multiple test cases at the same time?](reference-qa.md#add-multiple-test-cases) -> Test iterations are designed to support data-driven scenarios, not workflow-driven scenarios. - From a best practice perspective, if you have two test scenarios where the workflows are - different, consider creating separate test cases. Also see the [FAQs for manual testing](reference-qa.md#testcases). + + +> [!TIP] +> You can create a test case that automatically links to a requirement—User Story ([Agile](../boards/work-items/guidance/agile-process.md)), Product Backlog Item ([Scrum](../boards/work-items/guidance/scrum-process.md)), Requirement ([CMMI](../boards/work-items/guidance/cmmi-process.md)), or Issue ([Basic](../boards/get-started/plan-track-work.md))—when you create a test from the Kanban board. To learn more, see [Add, run, and update inline tests](../boards/boards/add-run-update-tests.md). +> + ## Assign testers 1. You can reassign test cases so that another tester can @@ -84,12 +92,15 @@ the tests. Testers need just [Basic access](../organizations/security/access-levels.md) to run tests from Azure Test Plans. -## See also - -* [FAQs for manual testing](reference-qa.md#testcases) -* [Link test cases to work items](../boards/queries/link-work-items-support-traceability.md) - ## Next step > [!div class="nextstepaction"] > [Run manual tests](run-manual-tests.md) + + +## Related articles + +* [Test objects and terms](test-objects-overview.md) +* [FAQs for manual testing](reference-qa.md#testcases) +* [Link test cases to work items](../boards/queries/link-work-items-support-traceability.md) + diff --git a/docs/test/explore-workitems-exploratory-testing.md b/docs/test/explore-workitems-exploratory-testing.md index dc510058cf0..060ef2bcc7f 100644 --- a/docs/test/explore-workitems-exploratory-testing.md +++ b/docs/test/explore-workitems-exploratory-testing.md @@ -5,16 +5,16 @@ ms.assetid: AFD66FBF-5DEC-4457-8867-A47FFB8EF407 ms.technology: devops-test ms.topic: conceptual ms.author: sdanie -author: steved0x -ms.date: 12/07/2018 +author: steved0x monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- -# Explore work items with the Test & Feedback extension +# Explore work items with the Test & Feedback extension [!INCLUDE [version-header](includes/version-header.md)] -Use the Test & Feedback extension to explore existing work items and +Use the Test & Feedback extension to explore existing work items and associate them with a new or an in-progress exploratory session. After a work item is associated with a session, all new bugs, tasks and test cases created in the current session are automatically linked to that work item. @@ -29,9 +29,10 @@ You can explore: You can explore a work item from the [Kanban board](#kanban) or from the [extension](#extension). You can also explore [multiple work items in the same session](#multipleitems). -[!INCLUDE [feature-availability](includes/feature-availability.md)] +[!INCLUDE [prerequisites-define](includes/prerequisites-stakeholder.md)] + ## Explore work items from the Kanban board 1. In the Kanban board, open the shortcut menu of the work item @@ -107,4 +108,4 @@ After you file bugs, create tasks, or create test cases, all these show up in th * [Exploratory testing with Microsoft Test Manager](/previous-versions/azure/devops/test/mtm/exploratory-testing-using-microsoft-test-manager) * [Overview of manual and exploratory testing](index.yml) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/how-long-to-keep-test-results.md b/docs/test/how-long-to-keep-test-results.md index fa289b99672..73c243aeb0f 100644 --- a/docs/test/how-long-to-keep-test-results.md +++ b/docs/test/how-long-to-keep-test-results.md @@ -21,10 +21,14 @@ Delete automated test results when you delete your builds. You can keep manual test results while you're still reviewing them, for example, up to a year. -[!INCLUDE [feature-availability](includes/feature-availability.md)] - +## Prerequisites + +- To manage project-level test-related policies, you must be a member of the [**Project Administrators** security group](../organizations/security/set-project-collection-level-permissions.md). +- To manage build pipeline object-level test retention policies, you must have the **Edit build pipeline** permissions set to **Allow**. + -## Manual test results + +## Set retention period for test results To delete manual test results after a specific number of days, set the retention limit at the project level. @@ -36,13 +40,11 @@ your test results before you can analyze the data. 1. Go to your project and then open the [project settings page](../project/navigation/go-to-service-page.md#open-project-settings). -1. In the **Test | Retention** page, select a limit for how long you want to keep manual test data. +1. In the **Test>Retention** page, select a limit for how long you want to keep manual test data. ![Select test data retention limits](media/how-long-to-keep-test-results/team-project-test-data-retention-limits.png) -## Automated test results - -### Automated test results associated with builds +## Set retention policy for automated test results associated with builds By default, Azure DevOps keeps automated test results related to builds only as long as you keep those builds. To keep test results after you delete your builds, @@ -63,16 +65,15 @@ you can specify how long to keep automated test results based on the branch. set the branch filter to delete test results and builds in specific branches as required. You can keep test results in other branches, even though you delete the builds in these branches. - -### Automated test results not associated with builds or orphaned from deleted builds - + +## Set retention policy for automated test results not associated with builds To clean up automated test results that are left over from deleted builds or test results that aren't related to builds, for example, results published from external test systems, -set the retention limits at the project level as shown [here](#manual-test-results-limits). +set the retention limits at the project level as described in [Set retention period for test results](#manual-test-results-limits) earlier in this article. -## See also +## Related articles * [FAQs for manual testing](reference-qa.md#manageresults) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/includes/feature-availability.md b/docs/test/includes/feature-availability.md deleted file mode 100644 index 1a1bcc56c27..00000000000 --- a/docs/test/includes/feature-availability.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -ms.topic: include ---- - -****** -To use all the features described in this article, you must have one of the following subscriptions: - -- [Enterprise](https://visualstudio.microsoft.com/vs/enterprise/) -- [Test Professional](https://visualstudio.microsoft.com/vs/test-professional/) -- [MSDN Platforms](https://visualstudio.microsoft.com/msdn-platforms/) - -Or, configure [Basic + Test Plans](https://marketplace.visualstudio.com/items?itemName=ms.vss-testmanager-web) access level. - -See [Manual testing permissions and access](../manual-test-permissions.md). -****** diff --git a/docs/test/includes/help-and-support-footer.md b/docs/test/includes/help-and-support-footer.md index 07311b9b633..5fbc96459f8 100644 --- a/docs/test/includes/help-and-support-footer.md +++ b/docs/test/includes/help-and-support-footer.md @@ -4,5 +4,4 @@ ms.topic: include ## Help and support -Report any problems via the [Developer Community](https://developercommunity.visualstudio.com/) -and get advice on [Stack Overflow](https://stackoverflow.com/questions/tagged/azure-devops). +Report any problems via the [Developer Community](https://developercommunity.visualstudio.com/) and get advice on [Stack Overflow](https://stackoverflow.com/questions/tagged/azure-devops). \ No newline at end of file diff --git a/docs/test/includes/navigation-include.md b/docs/test/includes/navigation-include.md deleted file mode 100644 index 570dc04c36d..00000000000 --- a/docs/test/includes/navigation-include.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -ms.topic: include -ms.technology: devops-cicd -ms.manager: mijacobs -ms.author: sdanie -author: steved0x -ms.date: 02/19/2020 ---- - -For more details about navigating in Azure DevOps Services and TFS, see [Web portal navigation](../../project/navigation/index.md). diff --git a/docs/test/includes/note-new-ui.md b/docs/test/includes/note-new-ui.md new file mode 100644 index 00000000000..441db9b5ae1 --- /dev/null +++ b/docs/test/includes/note-new-ui.md @@ -0,0 +1,14 @@ +--- +ms.technology: devops-test +ms.author: kaelli +author: KathrynEE +ms.topic: include +ms.date: 07/13/2021 +--- + +::: moniker range=">= azure-devops-2020" + +> [!NOTE] +> If you're new to using Azure Test Plans, review [Navigate Test Plans](../navigate-test-plans.md) to understand how to use the user interface to access select functions. + +::: moniker-end \ No newline at end of file diff --git a/docs/test/includes/prerequisites-define.md b/docs/test/includes/prerequisites-define.md new file mode 100644 index 00000000000..26eb215263d --- /dev/null +++ b/docs/test/includes/prerequisites-define.md @@ -0,0 +1,35 @@ +--- +ms.technology: devops-test +ms.author: kaelli +author: KathrynEE +ms.topic: include +ms.date: 07/13/2021 +--- + + +## Prerequisites + +- You must connect to a project. If you don't have a project yet, [create one](../../user-guide/sign-up-invite-teammates.md). +- You must be added to a project. To get added, [Add users to a project or team](../../organizations/security/add-users-team-project.md). +- To view test-related work items, you must have **Basic** access or higher and permissions to view work items under the corresponding Area Path. +- To [add test plans and test suites](../create-a-test-plan.md), [manual test cases](../create-test-cases.md), [delete test artifacts](../../boards/backlogs/delete-test-artifacts.md), and [define test configurations](../test-different-configurations.md) you must have [Basic + Test Plans](https://marketplace.visualstudio.com/items?itemName=ms.vss-testmanager-web) access level or have one of the following subscriptions: + + - [Enterprise](https://visualstudio.microsoft.com/vs/enterprise/) + - [Test Professional](https://visualstudio.microsoft.com/vs/test-professional/) + - [MSDN Platforms](https://visualstudio.microsoft.com/msdn-platforms/) + +- To add or edit test-related artifacts, you must have the following permissions: + - To add or modify test plans, test suites, test cases, or other test-based work item types, you must have **Edit work items in this node** permission set to **Allow** under the corresponding **Area Path**. + - To modify test plan properties such as build and test settings, you must have **Manage test plans** permission set to **Allow** under the corresponding **Area Path**. + - To create and delete test suites, add, and remove test cases from test suites, change test configurations associated with test suites, and modify a test suite hierarchy (move a test suite), you must have **Manage test suites** permission set to **Allow** under the corresponding **Area Path**. +- Project-level permissions that you may need include: + - To view, create, or delete test runs, you must be granted the corresponding permissions: **View test runs**, **Create test runs**, and **Delete test runs**. + - To manage test configurations or test environments, you must be granted the corresponding permissions: **Manage test configurations** and **Manage test environments**. + - To add new tags to test-based work items, you must have the **Create tag definition** permission set to **Allow**. + - To move test-based work items out of a project, you must have the **Move work items out of this project** permission set to **Allow**. + - To delete or restore test-based work items, you must have the **Delete and restore work items** permission set to **Allow**. + - To permanently delete test-based work items, you must have the **Permanently delete work items** permission set to **Allow**. + +To learn more, see [Set permissions and access for testing](../../organizations/security/set-permissions-access-test.md). + + diff --git a/docs/test/includes/prerequisites-run.md b/docs/test/includes/prerequisites-run.md new file mode 100644 index 00000000000..a1378507d1f --- /dev/null +++ b/docs/test/includes/prerequisites-run.md @@ -0,0 +1,18 @@ +--- +ms.technology: devops-test +ms.author: kaelli +author: KathrynEE +ms.topic: include +ms.date: 07/13/2021 +--- + + +## Prerequisites + +- You must connect to a project. If you don't have a project yet, [create one](../../user-guide/sign-up-invite-teammates.md). +- You must be added to a project. To get added, [Add users to a project or team](../../organizations/security/add-users-team-project.md). +- To view or run manual or automated tests, you must have **Basic** access or higher. + +To learn more, see [Set permissions and access for testing](../../organizations/security/set-permissions-access-test.md). + + diff --git a/docs/test/includes/prerequisites-stakeholder.md b/docs/test/includes/prerequisites-stakeholder.md new file mode 100644 index 00000000000..ad5a8576274 --- /dev/null +++ b/docs/test/includes/prerequisites-stakeholder.md @@ -0,0 +1,20 @@ +--- +ms.technology: devops-test +ms.author: kaelli +author: KathrynEE +ms.topic: include +ms.date: 07/13/2021 +--- + + +## Prerequisites + +- You must connect to a project. If you don't have a project yet, [create one](../../user-guide/sign-up-invite-teammates.md). +- You must be added to a project. To get added, [Add users to a project or team](../../organizations/security/add-users-team-project.md). +- To request or provide feedback, you must have **Stakeholder** access or higher. +- To add or modify bugs or other work item types, you must have the **Edit work items in this node** permission set to **Allow** under the corresponding **Area Path**. +- To add new tags, you must have the **Create tag definition** permission set to **Allow**. + +To learn more, see [Set permissions and access for testing](../../organizations/security/set-permissions-access-test.md). + + diff --git a/docs/test/includes/prerequisites.md b/docs/test/includes/prerequisites.md index 0fa8f0edb1b..583c1b5aa13 100644 --- a/docs/test/includes/prerequisites.md +++ b/docs/test/includes/prerequisites.md @@ -3,17 +3,31 @@ ms.technology: devops-test ms.author: kaelli author: KathrynEE ms.topic: include -ms.date: 07/09/2020 +ms.date: 07/13/2021 --- ## Prerequisites -- To use Test Plans, you must have been granted [Basic + Test Plans](https://marketplace.visualstudio.com/items?itemName=ms.vss-testmanager-web) access level or have one of the following subscriptions: - +- You must connect to a project. If you don't have a project yet, [create one](../../user-guide/sign-up-invite-teammates.md). +- You must be added to a project. To get added, [Add users to a project or team](../../organizations/security/add-users-team-project.md). +- To view test-related work items or run test cases, you must have **Basic** access or higher. Users with **Stakeholder** access have no access to Test Plans. +- To exercise the full range of test-related features, you must have been granted [Basic + Test Plans](../../organizations/billing/buy-access-tfs-test-hub.md) access level or have one of the following subscriptions: - [Enterprise](https://visualstudio.microsoft.com/vs/enterprise/) - [Test Professional](https://visualstudio.microsoft.com/vs/test-professional/) - [MSDN Platforms](https://visualstudio.microsoft.com/msdn-platforms/) - -- To create, delete, or manage test plans, test suites, test environments, or other test artifacts, you must have the required permissions. Members of the Contributors group are granted these permissions by default. For specific permission settings, see [Manual testing permissions and access](../manual-test-permissions.md). - \ No newline at end of file +- To add or edit test-related artifacts, you must have the following permissions: + - To add or modify test plans, test suites, test cases, or other test-based work item types, you must have **Edit work items in this node** permission set to **Allow** under the corresponding **Area Path**. + - To modify test plan properties such as build and test settings, you must have **Manage test plans** permission set to **Allow** under the corresponding **Area Path**. + - To create and delete test suites; add, and remove test cases from test suites; change test configurations associated with test suites—you must have **Manage test suites** permission set to **Allow** under the corresponding **Area Path**. +- Project-level permissions that you may need include: + - To view, create, or delete test runs, you must be granted the corresponding permissions: **View test runs**, **Create test runs**, and **Delete test runs**. + - To manage test configurations or test environments, you must be granted the corresponding permissions: **Manage test configurations** and **Manage test environments**. + - To add new tags to test-based work items, you must have the **Create tag definition** permission set to **Allow**. + - To move test-based work items out of a project, you must have the **Move work items out of this project** permission set to **Allow**. + - To delete or restore test-based work items, you must have the **Delete and restore work items** permission set to **Allow**. + - To permanently delete test-based work items, you must have the **Permanently delete work items** permission set to **Allow**. + +To learn more, see [Set permissions and access for testing](../../organizations/security/set-permissions-access-test.md). + + diff --git a/docs/test/includes/test-hub-include-adsonly.md b/docs/test/includes/test-hub-include-adsonly.md deleted file mode 100644 index 8e46c22e305..00000000000 --- a/docs/test/includes/test-hub-include-adsonly.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -ms.topic: include ---- - -**Azure Test Plans** (see [Web portal navigation](../../project/navigation/index.md)) diff --git a/docs/test/includes/test-hub-include-nolink.md b/docs/test/includes/test-hub-include-nolink.md deleted file mode 100644 index e289c585d3e..00000000000 --- a/docs/test/includes/test-hub-include-nolink.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -ms.topic: include ---- - -**Azure Test Plans** or the **Test** hub in TFS diff --git a/docs/test/includes/test-hub-include.md b/docs/test/includes/test-hub-include.md deleted file mode 100644 index c626445c563..00000000000 --- a/docs/test/includes/test-hub-include.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -ms.topic: include ---- - -**Azure Test Plans** or the **Test** hub in Azure DevOps Server (see [Web portal navigation](../../project/navigation/index.md)) diff --git a/docs/test/includes/version-header.md b/docs/test/includes/version-header.md index b08aa71ee6d..fb7da51a91f 100644 --- a/docs/test/includes/version-header.md +++ b/docs/test/includes/version-header.md @@ -3,7 +3,7 @@ ms.technology: devops-test ms.topic: include ms.author: sdanie author: steved0x -ms.date: 09/03/2020 +ms.date: 09/14/2021 --- -**Azure Test Plans | Azure DevOps Server 2020 | Azure DevOps Server 2019 | TFS 2015-2018** +**Azure Test Plans | Azure DevOps Server 2020 | Azure DevOps Server 2019 | TFS 2018-2015** diff --git a/docs/test/includes/version-inc-vs-all.md b/docs/test/includes/version-inc-vs-all.md index bd1a15e6d65..64edef2f5f7 100644 --- a/docs/test/includes/version-inc-vs-all.md +++ b/docs/test/includes/version-inc-vs-all.md @@ -6,4 +6,4 @@ author: steved0x ms.date: 09/03/2020 --- -**Azure Test Plans | Azure DevOps Server 2020 | Azure DevOps Server 2019 | TFS 2015-2018 | Visual Studio 2015-2019** +**Azure Test Plans | Azure DevOps Server 2020 | Azure DevOps Server 2019 | TFS 2018 -2015 | Visual Studio 2019-2015** diff --git a/docs/test/insights-exploratory-testing.md b/docs/test/insights-exploratory-testing.md index bd641a49a34..1340a2fc561 100644 --- a/docs/test/insights-exploratory-testing.md +++ b/docs/test/insights-exploratory-testing.md @@ -17,7 +17,7 @@ monikerRange: '>= tfs-2015' View completed exploratory testing sessions and derive meaningful insights at team or individual level, and for a specific period. -[!INCLUDE [feature-availability](includes/feature-availability.md)] +[!INCLUDE [prerequisites-define](includes/prerequisites-run.md)] 1. Open the **Recent exploratory sessions** page. You can do this: @@ -26,7 +26,7 @@ at team or individual level, and for a specific period. ![Opening the insights page from the extension](media/insights-exploratory-testing/insights-exploratory-testing-01.png) - - From [!INCLUDE [test-hub-include](includes/test-hub-include.md)] by opening the **Runs** page and choosing **Recent exploratory sessions**. + - From the **Test Plans** web portal. by opening the **Runs** page and choosing **Recent exploratory sessions**. ![Opening the insights page](media/insights-exploratory-testing/insights-exploratory-testing-02.png) @@ -138,4 +138,4 @@ Use a query to explore the work items that users have not yet explored. * [Exploratory testing with Microsoft Test Manager](/previous-versions/azure/devops/test/mtm/exploratory-testing-using-microsoft-test-manager) * [Overview of manual and exploratory testing](index.yml) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/manual-test-permissions.md b/docs/test/manual-test-permissions.md index c6073f3bd87..5e4c3f27196 100644 --- a/docs/test/manual-test-permissions.md +++ b/docs/test/manual-test-permissions.md @@ -6,65 +6,66 @@ ms.technology: devops-test ms.topic: reference ms.author: sdanie author: steved0x -ms.date: 06/02/2019 monikerRange: '>= tfs-2015' +ms.date: 09/15/2021 --- -# Manual test permissions and access +# Manual test access and permissions [!INCLUDE [version-header](includes/version-header.md)] -Azure Test Plans uses an access level called [Basic + Test Plans](https://marketplace.visualstudio.com/items?itemName=ms.vss-testmanager-web), -which is available from Azure DevOps. -## License requirements - -**Visual Studio Enterprise, Visual Studio Test Professional, and MSDN Platforms subscribers** have access to use this feature. These users do not require -an additional license, and can use this feature on their organization or collection for no additional charge. -For more information, see [Compare Visual Studio subscriptions](https://www.visualstudio.com/vs/pricing). +Access to Azure Test Plans features are controlled by access level and permissions. To access the Test Plans web portal, you must have a **Basic** access level or greater. To define and manage test plans, test suites, and test cases, you need **Basic + Test Plans**, which is available from Azure DevOps. -**Non-subscribers** must purchase Basic + Test Plans to use this functionality. This can be for any specified number of users. +> [!NOTE] +> Users granted **Stakeholder** access have no access to features or functions supported through the Test Plans or Test web portal. However, they are able to provide feedback through the **Test & Feedback** extension. To learn more, see [Stakeholder access quick reference](../organizations/security/stakeholder-access.md). + -**Notes:** +## Access and licensing -* Manual testers do not need this extension and can - [execute tests as a Basic user](#license-requirements) - in an Azure DevOps organization or collection. +To exercise the full range of test-related features, you must have been granted [Basic + Test Plans](../organizations/billing/buy-access-tfs-test-hub.md) access level or have one of the following subscriptions: + - [Visual Studio Enterprise](https://visualstudio.microsoft.com/vs/enterprise/) + - [Visual Studio Test Professional](https://visualstudio.microsoft.com/vs/test-professional/) + - [MSDN Platform](https://visualstudio.microsoft.com/msdn-platforms/) -* This extension can't be used by Stakeholders. +Users who have one of the above subscriptions can exercise all Test Plans features for no additional charge. For more information, see [Compare Visual Studio subscriptions](https://www.visualstudio.com/vs/pricing). + +Non-subscribers must purchase **Basic + Test Plans** to use Test Plans. This can be for any specified number of users. -* Licenses for this extension also gives users rights to use [Microsoft Test Manager](/previous-versions/azure/devops/test/mtm/guidance-mtm-usage) (a deprecated on-premises client). +> [!IMPORTANT] +> - Manual testers can execute tests with **Basic** access. +> -Licenses for this extension also give users rights to use [Microsoft Test Manager](/previous-versions/azure/devops/test/mtm/guidance-mtm-usage) (a deprecated on-premises client). -This table describes the license requirements in more detail: +The following table summarizes the license requirements required to exercise select tasks. -| Task or Use Case | License requirement | +| Task | Access level| | --- | --- | -| Execute tests | Basic license | -| Use the out-of-the-box standard charts | Basic license | -| Create and manage test plans | Basic + Test Plans license | -| Create and manage test suites | Basic + Test Plans license | -| Author test cases using a grid-like view and edit in the Test Runner | Basic + Test Plans license | -| Assign test cases to suites, move test cases, and order test cases | Basic + Test Plans license | -| Prepare for execution such as assigning configurations or testers | Basic + Test Plans license | -| Prepare User Acceptance Testing | Basic + Test Plans license | +| Provide feedback | Stakeholder | +| Execute tests, Mark test outcomes | Basic | +| Chart tasks, View reports | Basic | +| Create and manage test plans and test suites | Basic + Test Plans | +| Author test cases using a grid-like view and edit in the Test Runner | Basic + Test Plans | +| Assign test cases to suites, move test cases, and order test cases | Basic + Test Plans | +| Prepare for execution such as assigning configurations or testers | Basic + Test Plans | +| Prepare User Acceptance Testing | Basic + Test Plans | -## Access by user role + -You can access most manual testing features when you are added as a team member or a member -of the Contributors group for a project. The most common built-in groups include Readers, -Contributors, and Project Administrators. For a simplified view of all default permissions -assigned to built-in groups, see [Default permissions and access](../organizations/security/permissions-access.md). +## Permissions -Stakeholders have limited access to manual testing features. -To learn more, see [About access levels](../organizations/security/access-levels.md). +In addition to having the necessary access level, you also need the necessary permissions to exercise select tasks. Because manual testing is managed through [test-specific work item types](test-objects-overview.md), they are subject to some of the same permissions that manage work items. -[!INCLUDE [test](../organizations/security/includes/test.md)] +The following table provides the default permissions assigned to the built-in security groups: **Readers**, **Contributors**, and **Project Administrators**. Permissions are assigned for Area Paths and at the project-level. To learn how, see [Set permissions and access for testing](../organizations/security/set-permissions-access-test.md). -Permissions can be applied on area paths and at the project-level. To learn how, see [Set permissions and access for testing](../organizations/security/set-permissions-access-test.md). +[!INCLUDE [test](../organizations/security/includes/test.md)] -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] +For a simplified view of all default permissions assigned to built-in groups, see [Default permissions and access](../organizations/security/permissions-access.md). + ## Related articles -- [Set permissions and access for manual testing](../organizations/security/set-permissions-access-test.md) \ No newline at end of file +- [Test objects and terms](test-objects-overview.md) +- [Set permissions and access for manual testing](../organizations/security/set-permissions-access-test.md) +- [Default permissions and access](../organizations/security/permissions-access.md) +- [Security groups, service accounts, and permissions in Azure DevOps](../organizations/security/permissions.md) \ No newline at end of file diff --git a/docs/test/preview/media/4.png b/docs/test/media/navigate/all-page-favorite-test-plan.png similarity index 100% rename from docs/test/preview/media/4.png rename to docs/test/media/navigate/all-page-favorite-test-plan.png diff --git a/docs/test/preview/media/3.png b/docs/test/media/navigate/filter-options.png similarity index 100% rename from docs/test/preview/media/3.png rename to docs/test/media/navigate/filter-options.png diff --git a/docs/test/preview/media/2.png b/docs/test/media/navigate/mine-page-more-options-menu.png similarity index 100% rename from docs/test/preview/media/2.png rename to docs/test/media/navigate/mine-page-more-options-menu.png diff --git a/docs/test/preview/media/5.png b/docs/test/media/navigate/test-plan-favorite.png similarity index 100% rename from docs/test/preview/media/5.png rename to docs/test/media/navigate/test-plan-favorite.png diff --git a/docs/test/media/shared-steps/create-shared-result.png b/docs/test/media/shared-steps/create-shared-result.png new file mode 100644 index 00000000000..bbe70906e1c Binary files /dev/null and b/docs/test/media/shared-steps/create-shared-result.png differ diff --git a/docs/test/media/shared-steps/create-shared-steps.png b/docs/test/media/shared-steps/create-shared-steps.png new file mode 100644 index 00000000000..5968a95e118 Binary files /dev/null and b/docs/test/media/shared-steps/create-shared-steps.png differ diff --git a/docs/test/media/shared-steps/run-shared-steps.png b/docs/test/media/shared-steps/run-shared-steps.png new file mode 100644 index 00000000000..23ff387087a Binary files /dev/null and b/docs/test/media/shared-steps/run-shared-steps.png differ diff --git a/docs/test/media/shared-steps/shared-step-query.png b/docs/test/media/shared-steps/shared-step-query.png new file mode 100644 index 00000000000..ef5227ee392 Binary files /dev/null and b/docs/test/media/shared-steps/shared-step-query.png differ diff --git a/docs/test/media/shared-steps/use-shared-steps.png b/docs/test/media/shared-steps/use-shared-steps.png new file mode 100644 index 00000000000..e629fd245b2 Binary files /dev/null and b/docs/test/media/shared-steps/use-shared-steps.png differ diff --git a/docs/test/navigate-test-plans.md b/docs/test/navigate-test-plans.md index 976bcef5114..fee478c0539 100644 --- a/docs/test/navigate-test-plans.md +++ b/docs/test/navigate-test-plans.md @@ -5,8 +5,8 @@ ms.technology: devops-test ms.topic: conceptual ms.author: kaelli author: KathrynEE -ms.date: 09/09/2021 monikerRange: '>=azure-devops-2020' +ms.date: 09/14/2021 --- @@ -16,9 +16,11 @@ monikerRange: '>=azure-devops-2020' Learn how to navigate Test Plans to support your test goals and operations. Use the following sections of the Test Plans page to accomplish the corresponding tasks. +- **Mine**: View list of test plans that you created or are of interest to you. +- **All**: View a list of all test plans defined for the project. - **Test plan header**: Locate, favorite, edit, copy or clone a test plan. - **Test suites tree**: Add, manage, export, or order test suites. Leverage this to also assign configurations and perform user acceptance testing. -- **Define tab**: Collate, add, and manage test cases in a test suite of choice via this tab. +- **Define tab**: Collate, add, and manage test cases in a test suite of choice via this tab. This tab doesn't appear for users with **Basic** access. - **Execute tab**: Assign and execute tests via this tab or locate a test result to drill into. - **Chart tab**: Track test execution and status via charts, which can also be copied to dashboards. @@ -33,9 +35,34 @@ To learn about specific test artifacts, see the following articles: [!INCLUDE [prerequisites](includes/prerequisites.md)] > [!NOTE] -> Most operations under the **Define** tab are only available to users with [Basic + Test Plans](https://marketplace.visualstudio.com/items?itemName=ms.vss-testmanager-web) access level or equivalent. Most operations under the **Execute** tab can be exercised by users granted a **Basic** access level. +> Most operations under the **Define** tab are only available to users with [Basic + Test Plans](https://marketplace.visualstudio.com/items?itemName=ms.vss-testmanager-web) access level or equivalent. Most operations under the **Execute** tab can be exercised by users granted a **Basic** access level. +## The "Mine" page + +The **Mine** page shows a list of test plans that are of interest to you. +This includes plans for teams you are a member of, and test plans you have marked as favorites. +You can expand and hide the plans for each team to make it easier to locate or review specific team plans. Use the shortcut menu to edit or delete the selected plan. + +:::image type="content" source="media/navigate/mine-page-more-options-menu.png" alt-text="Mine page, menu options to edit or delete a test plan."::: + +Show the filter controls to help you find plans if you have a large number of plans in the list. +Filter the plans by name, team, state, or iteration. + +:::image type="content" source="media/navigate/filter-options.png" alt-text="Filter a test plans list."::: + +## The "All" page + +The **All** page shows a list of all test plans. +Filter test plans, and edit and delete plans using the shortcut menu, in the same way as in the **Mine** page. Use this page to add any test plan to your favorites list. + +:::image type="content" source="media/navigate/all-page-favorite-test-plan.png" alt-text="The All list of test plans, favorite a test plan."::: + +You can also add a plan to your favorites list while you are viewing it. + +:::image type="content" source="media/navigate/test-plan-favorite.png" alt-text="Add a plan to your favorites list."::: + + ## Test plan header diff --git a/docs/test/overview.md b/docs/test/overview.md index 641e83506f5..7f502bdf383 100644 --- a/docs/test/overview.md +++ b/docs/test/overview.md @@ -6,8 +6,8 @@ ms.technology: devops-test ms.topic: overview ms.author: sdanie author: steved0x -ms.date: 09/26/2019 monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- # Exploratory and manual testing scenarios and capabilities @@ -38,8 +38,6 @@ and gathering feedback from stakeholders. ![Holistic approach to manual testing, types of manual testing and personas involved](media/manual-testing/schematic-01.png)     **Holistic approach to manual testing, types of manual testing, and personas involved** -[!INCLUDE [feature-availability](includes/feature-availability.md)] - ## Planned manual testing @@ -102,7 +100,7 @@ to use these features. See more at [Add, run, and update inline tests](../boards ## Manual testing in Test Manager -[!INCLUDE [test-hub-include](includes/test-hub-include.md)] provides +the **Test Plans** web portal. provides a rich test management solution for teams that need advanced manual testing capabilities. It includes all the capabilities required for the testing lifecycle - including test planning, authoring, @@ -130,7 +128,7 @@ across test suites. See more at [Create test cases](create-test-cases.md). ### Testing applications -[!INCLUDE [test-hub-include](includes/test-hub-include.md)] provides +the **Test Plans** web portal. provides test runners to run tests for your web and desktop applications. Mark test steps and test outcomes as pass or fail, and collect diagnostic data such as system information, image action logs, screen recordings, and screen captures as you test. Bugs filed during the tests automatically include all the captured diagnostic data to help your developers reproduce the issues. See more at [Run tests for web apps](run-manual-tests.md#run-web) and [Run tests for desktop apps](run-manual-tests.md#run-desktop). @@ -204,7 +202,7 @@ Using the extension is a simple, three step process: bugs and tasks, and managing feedback requests in one place. As users perform exploratory testing, you can -[get insights from the sessions](insights-exploratory-testing.md) in [!INCLUDE [test-hub-include-nolink](includes/test-hub-include-nolink.md)] of +[get insights from the sessions](insights-exploratory-testing.md) in **Azure Test Plans** of Azure DevOps or TFS. View completed exploratory sessions and derive meaningful insights across all the sessions. Get end-to-end traceability such as a breakdown of the work items created, the work items explored and not explored, session owners, @@ -228,7 +226,7 @@ and [Provide stakeholder feedback](provide-stakeholder-feedback.md). ## Key benefits -* **Test on any platform**. With [!INCLUDE [test-hub-include](includes/test-hub-include.md)], you can use your browser to access all the manual testing capabilities. It enables you to [create](create-test-cases.md) and [run manual tests](run-manual-tests.md) through an easy-to-use, web-based interface that can be accessed from all major browsers on any platform. +* **Test on any platform**. With the **Test Plans** web portal, you can use your browser to access all the manual testing capabilities. It enables you to [create](create-test-cases.md) and [run manual tests](run-manual-tests.md) through an easy-to-use, web-based interface that can be accessed from all major browsers on any platform. * **Rich Diagnostic data collection**. Using the web-based Test Runner and Test Runner client you can [collect rich diagnostic data](collect-diagnostic-data.md) during your tests. This includes screenshots, an image action log, screen recordings, code coverage, IntelliTrace traces, and test impact data for your apps under test. This data is automatically included in all the bugs you create during test, making it easy for developers to reproduce the issues. @@ -246,4 +244,4 @@ and [Provide stakeholder feedback](provide-stakeholder-feedback.md). > See also [Continuous testing](../pipelines/index.yml), [Unit testing](/visualstudio/test/developer-testing-scenarios). -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/perform-exploratory-tests.md b/docs/test/perform-exploratory-tests.md index 1b917d37cb0..364532dcb78 100644 --- a/docs/test/perform-exploratory-tests.md +++ b/docs/test/perform-exploratory-tests.md @@ -6,8 +6,8 @@ ms.technology: devops-test ms.topic: quickstart ms.author: sdanie author: steved0x -ms.date: 06/02/2019 monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- # Install the Test & Feedback extension @@ -20,7 +20,7 @@ marketing teams, early adopters, and other stakeholders can use the extension to submit bugs or provide feedback and contribute to the quality of your product. -[!INCLUDE [feature-availability](includes/feature-availability.md)] +[!INCLUDE [prerequisites-define](includes/prerequisites-run.md)] @@ -96,7 +96,7 @@ update it automatically. If you have problems connecting to Azure DevOps or TFS, you may find the topic [TF31002: Unable to connect](../organizations/projects/tf31002-unable-connect-tfs.md) useful. -## See also +## Related articles * [FAQs for manual testing](reference-qa.md#tandfext) diff --git a/docs/test/preview/media/1.png b/docs/test/preview/media/1.png deleted file mode 100644 index 50201472988..00000000000 Binary files a/docs/test/preview/media/1.png and /dev/null differ diff --git a/docs/test/preview/new-test-plan-page.md b/docs/test/preview/new-test-plan-page.md deleted file mode 100644 index d82de559e88..00000000000 --- a/docs/test/preview/new-test-plan-page.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: New test plans view -description: A preview of a new user experience for test plans on Azure DevOps -ms.assetid: 4F99CDBA-8264-4BFA-8AB1-491403B6402C -ms.technology: devops-test -ms.topic: conceptual -ms.author: sdanie -author: steved0x -ms.date: 12/07/2018 -monikerRange: '> tfs-2018' ---- - -# New test plans view - -[!INCLUDE [version-header-devops-services](../includes/version-header-devops-services.md)] - -A new and fully redesigned user experience is available for test plans in Azure DevOps. -To use this page, you simply need to switch it on using one of the on-screen prompts, -or from the [user profile **Previews** panel](../../project/navigation/preview-features.md#enable-features-for-your-use). - -[!INCLUDE [feature-availability](../includes/feature-availability.md)] - -In [!INCLUDE [test-hub-include](../includes/test-hub-include.md)], choose the **Test plans** link to access the new test plans view. - -![Opening the test plans view](media/1.png) - -## The "Mine" page - -The **Mine** page shows a list of test plans that are of interest to you. -This includes plans for teams you are a member of, and test plans you have marked as favorites. -You can expand and hide the plans for each team to make it easier to locate or review specific team plans. -Use the shortcut menu to edit or delete the selected plan. - -![Editing or deleting a test plan](media/2.png) - -Show the filter controls to help you find plans if you have a large number of plans in the list. -Filter the plans by name, team, state, or iteration. - -![Filtering the lits of test plans](media/3.png) - - -## The "All" page - -The **All** page shows a list of all test plans. -Filter test plans, and edit and delete plans using the shortcut menu, in the same way as in the **Mine** page. -Use this page to add any test plan to your favorites list. - -![The All list of test plans](media/4.png) - -You can also add a plan to your favorites list while you are viewing it. - -![Adding a plan to your favorites list](media/5.png) - - -[!INCLUDE [help-and-support-footer](../includes/help-and-support-footer.md)] diff --git a/docs/test/provide-stakeholder-feedback.md b/docs/test/provide-stakeholder-feedback.md index 17eaa7c7a08..e4272ecac8f 100644 --- a/docs/test/provide-stakeholder-feedback.md +++ b/docs/test/provide-stakeholder-feedback.md @@ -1,4 +1,4 @@ ---- +--- title: Give feedback with Test & Feedback extension description: Provide feedback in Azure DevOps or TFS using the Exploratory Testing browser extension ms.assetid: 41CCD562-C071-4C33-A178-71DDAE83912E @@ -25,12 +25,14 @@ Before you start, ensure you have installed the [Test & Feedback extension](perform-exploratory-tests.md). This is required in order to respond to feedback requests. ->Any user with a Stakeholder access can use the +> [!NOTE] +> Any user with a Stakeholder access can use the Test & Feedback extension in **Stakeholder** mode. This mode is designed to allow the widest possible range of users to assist test teams by providing feedback. + ## Provide feedback directly from a feedback request email 1. Open the feedback request email and choose the @@ -148,7 +150,7 @@ to assist test teams by providing feedback. 1. Capture and submit your feedback as [shown above](#capture-feedback). -## See also +## Related articles * [Request stakeholder feedback using the Test & Feedback extension](request-stakeholder-feedback.md#request) * [Voluntary stakeholder feedback using the Test & Feedback extension](voluntary-stakeholder-feedback.md#voluntary) @@ -156,4 +158,4 @@ to assist test teams by providing feedback. * [Exploratory test and submit feedback directly from your browser](perform-exploratory-tests.md) * [Overview of manual and exploratory testing](index.yml) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/reference-qa.md b/docs/test/reference-qa.md index 22b79bfcf0e..d050441cb94 100644 --- a/docs/test/reference-qa.md +++ b/docs/test/reference-qa.md @@ -204,18 +204,18 @@ position. After reordering the tests, you can sort them by the **A:** Yes, you can tag test cases in a suite with any tag that you want. For example, tag all the tests related to login so that you can rerun these tests if a bug is fixed for the login page. -Then you can filter on that tag from [!INCLUDE [test-hub-include](includes/test-hub-include.md)]. +Then you can filter on that tag from the **Test Plans** web portal. You can add and edit tags when you edit a test case, or bulk edit tags in the grid view. You can also create suites based on queries when you use tags. -![In [!INCLUDE [test-hub-include](includes/test-hub-include.md)], on the Test Plans page, choose or add tags from the test case pane](media/create-test-cases/TestHubTags.png) +![In the **Test Plans** web portal, on the Test Plans page, choose or add tags from the test case pane](media/create-test-cases/TestHubTags.png) ### Q: Can I share test steps between test cases? **A:** Yes, choose the steps that you want to share. Find out more about -[sharing test steps](/previous-versions/azure/devops/test/mtm/share-steps-between-test-cases). +[sharing test steps](share-steps-between-test-cases.md). ![Create shared test steps](media/create-test-cases/CreateSharedSteps.png) @@ -271,6 +271,7 @@ test case is removed from the test suite. Also, it's no longer linked to the backlog item for that test suite. + ### Q: Why do I see the wrong test suite and tests when I click 'View Tests' from the notification email about tests that are assigned to me? **A:** This might happen if you were prompted to enter sign-in @@ -281,11 +282,13 @@ again to see the correct test suite and tests. ***** + ## Running manual tests [Go to related topic >](run-manual-tests.md) + ### Q: How do I rerun a test? **A:** Just select any test and choose **Run**. @@ -475,6 +478,7 @@ especially with continuous integration builds. Test results are often deleted before you can analyze them. + ### Q: How do I keep a build indefinitely? **A**: Like this: @@ -484,9 +488,10 @@ Test results are often deleted before you can analyze them. ***** -## Sharing steps between test cases -[Go to related topic >](/previous-versions/azure/devops/test/mtm/share-steps-between-test-cases) +## Share steps between test cases + +[Go to related topic >](share-steps-between-test-cases.md) ### Q: Can I share steps between test plans and projects?** @@ -500,11 +505,13 @@ You don't have to provide values in the shared steps definition. However, you ca ***** + ## Test & Feedback extension [Go to related topic >](perform-exploratory-tests.md) + ### Q: Which web browsers does the extension support? **A:** The Test & Feedback extension is currently available for @@ -538,6 +545,7 @@ For more details, see Visual Studio Marketplace, Azure DevOps tab. + ### Q: How do I play the video recordings I created with the extension? **A:** The video recordings created by the Test & Feedback extension can be @@ -552,6 +560,7 @@ mode with all the functionality except session insights and the request and provide feedback flow, which are supported only for TFS 2017. + ### Q: Can I edit an existing bug instead of creating a new bug when using the Test & Feedback extension? **A:** Yes, the extension automatically shows bugs that may be related to the one you are creating @@ -568,6 +577,7 @@ For more details, see [Add findings to existing bugs with exploratory testing](a ## FAQ on New Test Plans Page ### Q: How can I assign configurations to my test cases? + **A:** You can assign configurations in two ways: a. To assign configuration to whole suite, go to test suite tree, select the test suite and open the in-context menu bar to assign configurations. @@ -577,14 +587,17 @@ b. To assign configuration to test case(s), navigate to test plan > test suite a ### Q: How can I see the configurations assigned to my test cases? + **A:** Once you assign the configurations to the test cases in Define tab, the corresponding test points are created in the Execute tab. Navigate to Execute tab and see the configurations assigned to all the test points. ![Screenshot showing configurations assigned to the test points.](media/FAQ2.png) ### Q: How can I edit the test case? + **A:** If you have Basic Access Level, you can edit the test case by navigating to test plan > test suite > Execute > Test Point and clicking on In-context menu bar. If you have Basic + Test Plans Access Level, in addition to editing in Execute tab, you can also navigate to Define tab and double click on test case to edit it. ![Screenshot that shows editing the test case.](media/FAQ3.png) ### Q: How can I see the outcome of each test case? + **A:** The executable unit of a test case is called the test point. Hence, outcome is linked to each test point and not the test case. You can view it by navigating to test plan > test suite > Execute and see outcome. ![Screenshot that shows viewing the outcome of the test case.](media/FAQ4.png) @@ -593,12 +606,14 @@ b. To assign configuration to test case(s), navigate to test plan > test suite a ![Screenshot that shows assigning testers to test cases.](media/FAQ5.png) ### Q: How can I view the execution history of the test case? + **A:** You can view the execution history by navigating to Test Plan > Test Suite > Execute and right click on In-Context Menu bar. Initially, it will show the history of all the test points for that test case in that test suite. You can clear filters to view the execution history of that test case across all the test suites. ### Q: Can I select more than 1 work item while creating a requirement based suite? + **A:** No, the requirement based suite is linked to one work item. ***** -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/repeat-test-with-different-data.md b/docs/test/repeat-test-with-different-data.md index d378a92ca52..26420e8370d 100644 --- a/docs/test/repeat-test-with-different-data.md +++ b/docs/test/repeat-test-with-different-data.md @@ -6,8 +6,8 @@ ms.technology: devops-test ms.topic: conceptual ms.author: sdanie author: steved0x -ms.date: 06/02/2019 monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- # Repeat a test with different data @@ -27,7 +27,8 @@ use the web portal with TFS 2015 and later or Azure DevOps. That way you can run multiple test cases with the same data. -[!INCLUDE [feature-availability](includes/feature-availability.md)] + +[!INCLUDE [prerequisites-define](includes/prerequisites-define.md)] ## Add parameters to a test case @@ -136,7 +137,7 @@ values, and then play it back for the other sets. Record and playback doesn't work with all applications. For details, see [Supported Configurations and Platforms for Coded UI Tests and Action Recordings](/visualstudio/test/supported-configurations-and-platforms-for-coded-ui-tests-and-action-recordings). -## See also +## Related articles * [FAQs for manual testing](reference-qa.md#repeatdifferent) * [Overview of manual and exploratory testing](index.yml) @@ -144,4 +145,4 @@ Record and playback doesn't work with all applications. For details, see * [Collect diagnostic data](collect-diagnostic-data.md) * [Manage test results](how-long-to-keep-test-results.md) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/request-stakeholder-feedback.md b/docs/test/request-stakeholder-feedback.md index 8ad9bc2b922..64828f6c53e 100644 --- a/docs/test/request-stakeholder-feedback.md +++ b/docs/test/request-stakeholder-feedback.md @@ -11,10 +11,13 @@ monikerRange: '>= tfs-2017' --- # Request stakeholder feedback using the Test & Feedback extension - + +[!INCLUDE [version-header](includes/version-header.md)] + [!INCLUDE [feedback-header-text](includes/feedback-header-text.md)] + ## Request feedback from stakeholders Request feedback from stakeholders directly from an Azure DevOps or TFS work item. @@ -39,7 +42,7 @@ Request feedback from stakeholders directly from an Azure DevOps or TFS work ite > Just add their names in the feedback request form so that a **Request feedback** email is sent to them. > Also see [Can users with Basic access respond to feedback requests?](provide-stakeholder-feedback.md#non-stakeholder-feedback). -## See also +## Related articles * [Provide stakeholder feedback using the Test & Feedback extension](provide-stakeholder-feedback.md#provide) * [Voluntary stakeholder feedback using the Test & Feedback extension](voluntary-stakeholder-feedback.md#voluntary) @@ -47,5 +50,5 @@ Request feedback from stakeholders directly from an Azure DevOps or TFS work ite * [Exploratory test and submit feedback directly from your browser](perform-exploratory-tests.md) * [Overview of manual and exploratory testing](index.yml) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/run-automated-tests-from-test-hub.md b/docs/test/run-automated-tests-from-test-hub.md index 938cd8224ee..a3f08d95c8d 100644 --- a/docs/test/run-automated-tests-from-test-hub.md +++ b/docs/test/run-automated-tests-from-test-hub.md @@ -6,54 +6,44 @@ ms.technology: devops-test ms.topic: conceptual ms.author: sdanie author: steved0x -ms.date: 06/17/2021 monikerRange: '>= tfs-2017' +ms.date: 09/14/2021 --- # Run automated tests from test plans [!INCLUDE [version-header-tfs17](includes/version-header-tfs17.md)] -Automate test cases in your test plans and run them directly from [!INCLUDE [test-hub-include-nolink](includes/test-hub-include-nolink.md)]: +Automate test cases in your test plans and run them directly from **Azure Test Plans** . Automated tests provide you with the following benefits: -* Provides a user-friendly process for testers who may not be well +* A user-friendly process for testers who may not be well versed with running tests in Build or Release workflows. - -* Gives you the flexibility to run selected tests on demand, +* The flexibility to run selected tests on demand, rather than scheduled testing in Build or Release workflows where all tests meeting the filter criteria are run. - -* Useful when you want to rerun a few tests that failed due +* The ability to rerun a few tests that failed due to test infrastructure issues, or you have a new build that includes fixes for failed tests. -You will need: -* A [test plan](create-a-test-plan.md) +[!INCLUDE [prerequisites-define](includes/prerequisites-run.md)] + +In addition, you'll need: + +- A [test plan](create-a-test-plan.md) containing your automated tests, which you have associated with automated test methods using [Visual Studio 2017](associate-automated-test-with-test-case.md), or [Visual Studio 2015 or earlier](/previous-versions/visualstudio/visual-studio-2013/dd380741(v=vs.120)). - -* A [Team Build pipeline](../pipelines/apps/aspnet/build-aspnet-4.md) +- A [build pipeline](../pipelines/apps/aspnet/build-aspnet-4.md) that generates builds containing the test binaries. - -* The app to test. You can deploy the app as part of the +- An app to test. You can deploy the app as part of the [build and release workflow](../pipelines/get-started/what-is-azure-pipelines.md) and also use it for on-demand testing. - -You must also be a Project Contributor, or have the following permissions: - -* Create releases -* Manage releases -* Edit release environment -* Manage deployment - -For more information, see [Set permissions for release pipelines](../pipelines/policies/set-permissions.md) and -[Release permissions](../pipelines/policies/permissions.md#set-release-permissions). +- Permissions to create and manage releases, edit a release environment, and manage deployment. For more information, see [Set permissions for release pipelines](../pipelines/policies/set-permissions.md) and [Release permissions](../pipelines/policies/permissions.md#set-release-permissions). ## Set up your environment -1. In the **Test plans** page of [!INCLUDE [test-hub-include](includes/test-hub-include.md)], choose your test plan, +1. In the **Test Plans** page, choose your test plan, open the shortcut menu, and choose **Test plan settings**. ![Choosing Test plan settings](media/run-automated-tests-from-test-hub/run-auto-tests-from-hub-101.png) @@ -66,7 +56,7 @@ For more information, see [Set permissions for release pipelines](../pipelines/p 3. You will need a release pipeline that was created from the **Run automated tests from Test Manager** template to run tests from test plans - in [!INCLUDE [test-hub-include-nolink](includes/test-hub-include-nolink.md)]. If you have an existing release pipeline that was created + in **Azure Test Plans** . If you have an existing release pipeline that was created using this template, select it and then select the existing stage in the release pipeline where the tests will be executed. Otherwise, choose the **Create new** link in the @@ -162,7 +152,7 @@ For more information, see [Set permissions for release pipelines](../pipelines/p ## Run the automated tests -1. In [!INCLUDE [test-hub-include](includes/test-hub-include.md)], open the test plan and select a test suite that contains the +1. In the **Test Plans** web portal, open the test plan and select a test suite that contains the automated tests. 2. Select the test(s) you want to run, open the **Run** menu, @@ -189,7 +179,7 @@ For more information, see [Set permissions for release pipelines](../pipelines/p console logs, and attachments. 5. After test execution is complete, the **Runs** page of the - [!INCLUDE [test-hub-include-nolink](includes/test-hub-include-nolink.md)] shows the test results. The **Run summary** page + **Azure Test Plans** shows the test results. The **Run summary** page shows an overview of the run. ![Viewing the test run summary](media/run-automated-tests-from-test-hub/run-auto-tests-from-hub-110.png) @@ -406,4 +396,3 @@ must be specified in the [Visual Studio Test task](../pipelines/tasks/test/vstes * [Associate automated test results with requirements](../pipelines/test/requirements-traceability.md) * [Continuous testing scenarios and capabilities](index.yml) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] \ No newline at end of file diff --git a/docs/test/run-manual-tests.md b/docs/test/run-manual-tests.md index 121d9728c71..63f6ec1c4b2 100644 --- a/docs/test/run-manual-tests.md +++ b/docs/test/run-manual-tests.md @@ -6,8 +6,8 @@ ms.technology: devops-test ms.topic: quickstart ms.author: sdanie author: steved0x -ms.date: 12/07/2018 monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- # Run manual tests @@ -20,14 +20,11 @@ use Test Runner to create a bug. Test steps, screenshots, and comments are automatically included in the bug. You can use the web runner for web apps, or the desktop runner for desktop app data collection. -> You just need [Basic access](https://visualstudio.microsoft.com/team-services/compare-features/) -to run tests that have been assigned to you with Azure DevOps. -Learn more about the [access](https://visualstudio.microsoft.com/pricing/visual-studio-online-pricing-vs) -that you need for more advanced testing features. - -[!INCLUDE [feature-availability](includes/feature-availability.md)] + +[!INCLUDE [prerequisites-define](includes/prerequisites-run.md)] + ## Run tests for web apps 1. If you haven't already, [create your manual tests](create-test-cases.md#test-cases). @@ -86,6 +83,7 @@ that you need for more advanced testing features. [Can I run tests offline and then import the results?](reference-qa.md#runoffline) + ## Run tests for desktop apps If you want to collect more diagnostic data for your desktop application, run your tests using Test Runner client: @@ -105,7 +103,7 @@ If you want to collect more diagnostic data for your desktop application, run yo [Can I run tests offline and then import the results?](reference-qa.md#runoffline) -## See also +## Related articles * [FAQs for manual testing](reference-qa.md#runtests) diff --git a/docs/test/share-steps-between-test-cases.md b/docs/test/share-steps-between-test-cases.md new file mode 100644 index 00000000000..9345a5d86da --- /dev/null +++ b/docs/test/share-steps-between-test-cases.md @@ -0,0 +1,55 @@ +--- +title: Share steps between test cases +titleSuffix: Azure Test Plans +description: Share steps between all your test cases when you want to test web applications +ms.technology: devops-test +ms.topic: how-to +ms.author: sdanie +author: steved0x +monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 +--- + +# Share steps between test cases + + +[!INCLUDE [version-header](includes/version-header.md)] + + +When you [plan manual tests](create-a-test-plan.md) there are some sequences of steps, such as logging in, that occur in many test cases. To avoid having to enter these sequences again and again, create *shared steps*. + +[!INCLUDE [prerequisites-define](includes/prerequisites-define.md)] + +[!INCLUDE [note-new-ui](includes/note-new-ui.md)] + +## Create shared steps + +While you're editing a test case, select a sequence of steps that you want to share: + +![Create shared steps](media/shared-steps/create-shared-steps.png) + +The steps you selected are replaced with a link to the new shared steps work item: + +![Resulting test case with a shared step.](media/shared-steps/create-shared-result.png) + +## Use shared steps + +Now you can use the shared steps in another test case: + +![Use shared steps in test cases.](media/shared-steps/use-shared-steps.png) + +A query opens. Run it to find the steps you want to insert: + +![Run the query to find shared steps](media/shared-steps/shared-step-query.png) + +## When you run a test with shared steps + +When you [run a test](run-manual-tests.md), you can either mark the whole shared sequence as passed or failed, or mark each step separately: + +![Shared steps in Test Runner.](media/shared-steps/run-shared-steps.png) + +## Related articles + + +* [Test objects and terms](test-objects-overview.md) +- [FAQs for manual testing](reference-qa.md#sharesteps) diff --git a/docs/test/test-different-configurations.md b/docs/test/test-different-configurations.md index b5be188f201..8729414eaa9 100644 --- a/docs/test/test-different-configurations.md +++ b/docs/test/test-different-configurations.md @@ -6,8 +6,8 @@ ms.technology: devops-test ms.topic: conceptual ms.author: sdanie author: steved0x -ms.date: 12/07/2018 monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- # Test different configurations @@ -20,12 +20,12 @@ operating systems, web browsers, and other variations. You will want to run at least some of your tests in stages that have those different configurations. -Use your test plan to decide which tests you want to run +Use your test plans to decide which tests you want to run on which configurations. You have to make sure that when you run your tests that you have set up your stages for the configurations that you need. -You might draw up a schematic matrix of the combinations that you want to test: +You might draw a schematic matrix of the combinations that you want to test: ![Test matrix of browser and operating system configurations](media/shared/testing-configurations-schematic.png) @@ -35,14 +35,12 @@ Then you can: * [Assign the configurations to test plans and test suites](#assign-configs) * [Run tests with each of the configurations](#run-configs) * [Track your test results for each configuration](#track-configs) - -> [!NOTE] -> This feature is available only in Azure DevOps. -> In addition, Stakeholders and Basic users cannot create or manage configurations. - -[!INCLUDE [feature-availability](includes/feature-availability.md)] + + +[!INCLUDE [prerequisites-define](includes/prerequisites-define.md)] + ## Create configurations and variables A test configuration is a combination of configuration variable @@ -53,7 +51,7 @@ might be "Windows 8 + 32-bit CPU" or "Windows 10 + 64-bit CPU." You must create the configuration variables first. Then combine multiple variable values to create a configuration. -1. Open the **Configurations** page in [!INCLUDE [test-hub-include](includes/test-hub-include.md)], choose +1. Open the **Configurations** page in the **Test Plans** web portal, choose the ![new](media/plus-and-arrow-icon.png) icon, and select **New configuration variable**. @@ -154,9 +152,9 @@ or test suite apply to all tests or suites within it. > If you have a test case that appears in several test plans and test suites, you can set the different configurations for each of these. The same test case can have different configuration settings in different test suites and test plans. -## See also +## Related articles * [Overview of manual and exploratory testing](index.yml) * [Exploratory test and submit feedback directly from your browser](perform-exploratory-tests.md) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/test-objects-overview.md b/docs/test/test-objects-overview.md new file mode 100644 index 00000000000..104076ce357 --- /dev/null +++ b/docs/test/test-objects-overview.md @@ -0,0 +1,425 @@ +--- +title: Test objects and terms +titleSuffix: Azure Test Plans +description: Understand the different test objects and terms that support manual or automated testing. +ms.technology: devops-test +ms.topic: overview +ms.author: kaelli +author: KathrynEE +monikerRange: '>= tfs-2015' +ms.date: 09/15/2021 +--- + + +# Test objects and terms + +Read this article to gain an understanding of the objects and terms used in manual and exploratory testing. + + + + +## Test-specific work item types + +To support manual and automated testing, you add and group three main types of test-specific work item types: **Test Plans**, **Test Suites**, and **Test Cases**. To support sharing of various test steps and test parameters, you define **Shared Steps** and **Shared Parameters**. These objects are stored in the work tracking data store as specific types of work items. + + +![Test management work item types](../boards/work-items/guidance/media/ALM_PT_WITS_TestExperience.png) + +The following table describes the work item types used to support the Azure DevOps test experience. Test-specific work items link together using the link types shown in the previous image. + + +:::row::: + :::column span="1"::: + **Work item type** + :::column-end::: + :::column span="3"::: + **Description** + :::column-end::: +:::row-end::: +--- +:::row::: + :::column span="1"::: + **Test plans** + :::column-end::: + :::column span="3"::: + Are used to group test suites and individual test cases. To define a test plan, see [Create test plans and test suites](create-a-test-plan.md). + :::column-end::: +:::row-end::: +:::row::: + :::column span="1"::: + **Test suite** + :::column-end::: + :::column span="3"::: + Group test cases into separate testing scenarios within a single test plan. Grouping test cases makes it easier to see which scenarios are complete. When creating a test suite, you can specify one of three types: + - **Static test suites**: Used to group test cases under a single test suite. + - [**Requirement-based suites**](create-a-test-plan.md#backlog): Select one or more requirements from a query which are then linked to the test suite. + - [**Query-based suites**](reference-qa.md#query-based-suites): Select one or more test cases which are then linked to the test suite. + > [!TIP] + > The [**Test Suite Type**](../boards/queries/build-test-integration.md) read-only field indicates the type of suite selected. To add test suites, see [Create test plans and test suites](create-a-test-plan.md). + :::column-end::: +:::row-end::: +:::row::: + :::column span="1"::: + **Test cases** + :::column-end::: + :::column span="3"::: + Define the steps used to test code or an app for deployment. Define test cases to ensure your code works correctly, has no errors, and meets business and customer requirements. You can add individual test cases to a test plan without creating a test suite. More than one test suite or test plan can refer to a test case. You can effectively reuse test cases without needing to copy or clone them for each suite or plan. There are two types of test cases: + - [**Manual**](create-test-cases.md): Test cases that define different steps that you run using Test Runner or other supported client. + - [**Automated**](run-automated-tests-from-test-hub.md): Test cases that are designed to run within an Azure Pipeline or + > [!TIP] + > You can create a test case that automatically links to a requirement—User Story ([Agile](../boards/work-items/guidance/agile-process.md)), Product Backlog Item ([Scrum](../boards/work-items/guidance/scrum-process.md)), Requirement ([CMMI](../boards/work-items/guidance/cmmi-process.md)), or Issue ([Basic](../boards/get-started/plan-track-work.md))—when you create a test from the Kanban board. To learn more, see [Add, run, and update inline tests](../boards/boards/add-run-update-tests.md). + :::column-end::: +:::row-end::: +:::row::: + :::column span="1"::: + **Shared steps** + :::column-end::: + :::column span="3"::: + Use to share steps between multiple test cases. For example, log-in and verify steps for signing into an application are steps that can be shared across a number of test cases. To learn how, see [Share steps between test cases](share-steps-between-test-cases.md). + :::column-end::: +:::row-end::: +:::row::: + :::column span="1"::: + **Shared parameters** + :::column-end::: + :::column span="3"::: + Use to specify different parameters for executing test a test step within a test case. To learn how, see [Repeat a test with different data](repeat-test-with-different-data.md). + :::column-end::: +:::row-end::: +--- + +## Common fields to all test-specific work item types + +The following fields and tabs appear in most work items. Each tab is used to track specific information, such as :::image type="icon" source="../boards/backlogs/media/icon-history-tab-wi.png" border="false"::: history, :::image type="icon" source="../boards/backlogs/media/icon-links-tab-wi.png" border="false"::: links, or :::image type="icon" source="../boards/backlogs/media/icon-attachments-tab-wi.png" border="false"::: attachments. These three tabs provide a history of changes, view of linked work items, and ability to view and attach files. + +The only required field for all work item types is **Title**. When the work item is saved, the system assigns it a unique **ID**. The form highlights required field in yellow. For information about test-related fields, see [Query based on build and test integration fields](../boards/queries/build-test-integration.md). For all other fields, see [Work item field index](../boards/work-items/guidance/work-item-field.md). + + + +:::row::: + :::column span=""::: + **Field** + :::column-end::: + :::column span="3"::: + **Usage** + :::column-end::: +:::row-end::: +--- +:::row::: + :::column span=""::: + [Title](../boards/queries/titles-ids-descriptions.md) + :::column-end::: + :::column span="3"::: + Enter a description of 255 characters or less. You can always modify the title later. + :::column-end::: +:::row-end::: +:::row::: + :::column span=""::: + [Assigned To](../boards/queries/query-by-workflow-changes.md) + :::column-end::: + :::column span="3"::: + Assign the work item to the team member responsible for performing the work. Depending on the context you are working in, the drop-down menu lists only team members or contributors to the project. + > [!NOTE] + > You can only assign work to a single user. If you need to assign work to more than one user, add a work item for each user and distinguish the work to be done by title and description. The Assigned To field only accepts user accounts that have been [added to a project or team](../organizations/security/add-users-team-project.md). + :::column-end::: +:::row-end::: +:::row::: + :::column span=""::: + [State](../boards/queries/query-by-workflow-changes.md) + :::column-end::: + :::column span="3"::: + When the work item is created, the State defaults to the first state in the workflow. As work progresses, update it to reflect the current status. + :::column-end::: +:::row-end::: +:::row::: + :::column span=""::: + [Reason](../boards/queries/query-by-workflow-changes.md) + :::column-end::: + :::column span="3"::: + Use the default first. Update it when you change state as need. Each State is associated with a default reason. + :::column-end::: +:::row-end::: +:::row::: + :::column span=""::: + [Area (Path)](../boards/queries/query-by-area-iteration-path.md) + :::column-end::: + :::column span="3"::: + Choose the area path associated with the product or team, or leave blank until assigned during a planning meeting. To change the dropdown list of areas, see [Define area paths and assign to a team](../organizations/settings/set-area-paths.md). + :::column-end::: +:::row-end::: +:::row::: + :::column span=""::: + [Iteration (Path)](../boards/queries/query-by-workflow-changes.md) + :::column-end::: + :::column span="3"::: + Choose the sprint or iteration in which the work is to be completed, or leave it blank and assign it later during a planning meeting. To change the drop-down list of iterations, see [Define iteration paths and configure team iterations](../organizations/settings/set-iteration-paths-sprints.md). + :::column-end::: +:::row-end::: +:::row::: + :::column span=""::: + [Description](../boards/queries/titles-ids-descriptions.md) + :::column-end::: + :::column span="3"::: + Provide enough detail to create shared understanding of scope and support estimation efforts. Focus on the user, what they want to accomplish, and why. Don't describe how to develop the product. Do provide sufficient details so that your team can write tasks and test cases to implement the item. + :::column-end::: +:::row-end::: +--- + + +## Common controls to all test-specific work item types + +Several controls appear in several test-specific work items as described in the following table. If these controls are not of interest, you can hide them from the work item form layout as described in [Add and manage fields (Inheritance process)](../organizations/settings/work/customize-process-field.md#hide-a-field-or-custom-control). + + +:::row::: + :::column span=""::: + **Control** + :::column-end::: + :::column span="3"::: + **Description** + :::column-end::: +:::row-end::: +--- +:::row::: + :::column span=""::: + **Deployment** + :::column-end::: + :::column span="3"::: + Provides insight into whether a feature or user story has been deployed and to what stage. You gain visual insight into the status of a work item as it is deployed to different release environments as well as quick navigation to each release stage and run. + This control is available from **Test Plans**, **Test Suites**, and **Test Cases**. + :::column-end::: +:::row-end::: +:::row::: + :::column span=""::: + **Development** + :::column-end::: + :::column span="3"::: + Records all Git development processes that support completion of the work item. It is typically used to [drive Git development from a requirement](../boards/backlogs/connect-work-items-to-git-dev-ops.md). This control supports traceability, providing visibility into all the branches, commits, pull requests, and builds related to the work item. + This control is available from **Test Plans**, **Test Suites**, and **Test Cases**. + :::column-end::: +:::row-end::: +:::row::: + :::column span=""::: + **Related Work** + :::column-end::: + :::column span="3"::: + Control used in **Test Plans**, **Test Suites**, and **Test Cases** to show or link to other work items such as requirements and bugs, usually through the **Related** link type. + :::column-end::: +:::row-end::: +:::row::: + :::column span=""::: + **Test Cases** + :::column-end::: + :::column span="3"::: + Control used in **Shared Steps** and **Shared Parameters** work items to indicate or link to **Test Cases**. + :::column-end::: +:::row-end::: +--- + +## Customize test-specific work item types + +For the Inherited process, you can customize test plans, test suites, and test cases. For the On-premises XML process, you can customize all test-specific work item types. To learn more, see [Customize work tracking objects to support your team's processes](../reference/customize-work.md). + +## Permissions required to modify work items + +There are a number of permissions that control select features for viewing, modifying, or deleting work items. These include those listed in the following table. + +::: moniker range=">= azure-devops-2019" +> [!NOTE] +> The **Change work item type** permission doesn't apply to test-specific work items. Even if you choose this feature from the work item form, changing the work item type is disallowed. +::: moniker-end + + +:::row::: + :::column span="2"::: + **Permission** + :::column-end::: + :::column span="1"::: + **Level** + :::column-end::: + :::column span="2"::: + **Task** + :::column-end::: +:::row-end::: +:::row::: + :::column span="2"::: + **View test runs** + **Create test runs** + **Delete test runs** + :::column-end::: + :::column span="1"::: + Project-level + :::column-end::: + :::column span="2"::: + To view, create, or delete test runs, you must be granted the corresponding permission. + :::column-end::: +:::row-end::: +:::row::: + :::column span="2"::: + **Manage test configurations** + **Manage test environments** + :::column-end::: + :::column span="1"::: + Project-level + :::column-end::: + :::column span="2"::: + Manage test configurations or test environments, you must be granted the corresponding permission. + :::column-end::: +:::row-end::: +:::row::: + :::column span="2"::: + **Create tag definition** + :::column-end::: + :::column span="1"::: + Project-level + :::column-end::: + :::column span="2"::: + Add new tags to test-based work items. + :::column-end::: +:::row-end::: +::: moniker range=">= tfs-2015" +:::row::: + :::column span="2"::: + **Delete and restore work items** + :::column-end::: + :::column span="1"::: + Project-level + :::column-end::: + :::column span="2"::: + Delete test-specific work items and restore them from the Recycle bin. + :::column-end::: +:::row-end::: +::: moniker-end +::: moniker range=">= tfs-2015" +:::row::: + :::column span="2"::: + **Permanently delete work items** + :::column-end::: + :::column span="1"::: + Project-level + :::column-end::: + :::column span="2"::: + Permanently delete test-specific work items from the data store. + :::column-end::: +:::row-end::: +::: moniker-end +:::row::: + :::column span="2"::: + **View work items in this node** + **Edit work items in this node** + :::column-end::: + :::column span="1"::: + Area Path + :::column-end::: + :::column span="2"::: + View or add or modify test plans, test suites, test cases, or other test-based work item types requires the corresponding permission. + :::column-end::: +:::row-end::: +:::row::: + :::column span="2"::: + **Manage test plans** + :::column-end::: + :::column span="1"::: + Area Path + :::column-end::: + :::column span="2"::: + Modify test plan properties such as test run and test outcome settings. + :::column-end::: +:::row-end::: +:::row::: + :::column span="2"::: + **Manage test plans** + :::column-end::: + :::column span="1"::: + Area Path + :::column-end::: + :::column span="2"::: + Create and delete test suites; add, and remove test cases from test suites; change test configurations associated with test suites; and modify a test suite hierarchy (move a test suite). + :::column-end::: +:::row-end::: + + +To learn more about setting these permissions, see [Set permissions and access for testing](../organizations/security/set-permissions-access-test.md) and [Set permissions at the project- or collection-level](../organizations/security/set-project-collection-level-permissions.md). + +## Export, import, and bulk update of test-specific work items + +As with other work items, you can bulk edit test-specific work items. To learn more, see the following articles: + +::: moniker range=">= azure-devops-2020" +- [Bulk modify work items](../boards/backlogs/bulk-modify-work-items.md). +- [Navigate Test Plans, Test suites context menu options](navigate-test-plans.md#test-suites-context-menu-options) +::: moniker-end + +::: moniker range="< azure-devops-2020" +- [Bulk modify work items](../boards/backlogs/bulk-modify-work-items.md). +- [FAQs for manual testing, Can I export the test plan to share or review offline](reference-qa.md#q-can-i-export-the-test-plan-to-share-or-review-offline) +::: moniker-end + +## Test terms + +The following table describes several terms used in manual and exploratory testing. + + +:::row::: + :::column span="1"::: + **Term** + :::column-end::: + :::column span="3"::: + **Definition** + :::column-end::: +:::row-end::: +--- +:::row::: + :::column span="1"::: + **Test points** + :::column-end::: + :::column span="3"::: + Test cases by themselves are not executable. When you add a test case to a test suite then test point(s) are generated. A test point is a unique combination of test case, test suite, configuration, and tester. For example, if you have a test case named Test login functionality and you add two configurations for the *Edge* and *Chrome* browsers, you have two test points. You can execute or run each of these test points. On execution, test results are generated. Through the test results view, or execution history, you can see all executions of a test point. The latest execution for the test point is what you see in the Execute tab. + :::column-end::: +:::row-end::: +:::row::: + :::column span="1"::: + **Test run settings** + :::column-end::: + :::column span="3"::: + Dialog used to associate test plans with a build or release pipelines. + :::column-end::: +:::row-end::: +:::row::: + :::column span="1"::: + **Test outcome settings** + :::column-end::: + :::column span="3"::: + Dialog used to choose how test outcomes in multiple suites under the same test plans should be configured. + :::column-end::: +:::row-end::: +:::row::: + :::column span="1"::: + **Outcome** + :::column-end::: + :::column span="3"::: + Outcome of a test point as marked by the tester upon executing the test. Valid options are: + - **Active** (Unspecified) + - **Pass Test** + - **Fail Test** + - **Block Test** + - **Not Applicable** + --- + To learn more, see [Repeat a test with different data](repeat-test-with-different-data.md). Note that pipeline test outcomes differ as described in [About pipeline tests](../pipelines/test/test-glossary.md). + :::column-end::: +:::row-end::: +:::row::: + :::column span="1"::: + **Traceability** + :::column-end::: + :::column span="3"::: + Ability to trace test results with the requirements and bugs that they are linked to. + :::column-end::: +:::row-end::: + + +## Related articles + +- [Exploratory and manual testing scenarios and capabilities](overview.md) +- [Navigate Test Plans](navigate-test-plans.md) +- [About pipeline tests](../pipelines/test/test-glossary.md) \ No newline at end of file diff --git a/docs/test/toc.yml b/docs/test/toc.yml index 4f43fe34bbb..18622a5eb94 100644 --- a/docs/test/toc.yml +++ b/docs/test/toc.yml @@ -10,50 +10,63 @@ - name: Navigate Test Plans href: navigate-test-plans.md displayName: test suites tree, define, execute, chart - - name: Create a test plan + - name: Test objects and terms + href: test-objects-overview.md + displayName: test plan, test case, test suites, test runs, test results, test points + - name: Create test plans and test suites href: create-a-test-plan.md displayName: create, plan - name: Create test cases href: create-test-cases.md displayName: create, case + - name: Share steps between test cases + href: share-steps-between-test-cases.md - name: Run manual tests href: run-manual-tests.md displayName: manual test, run, test runs - - name: Set permissions and access - href: ../organizations/security/set-permissions-access-test.md?toc=/azure/devops/test/toc.json&bc=/azure/devops/test/breadcrumb/toc.json - displayName: access, permissions -- name: Manual testing + - name: Default permissions (Security) + href: manual-test-permissions.md +- name: Manual tests items: - - name: Test from the Kanban board - href: ../boards/boards/add-run-update-tests.md?toc=/azure/devops/test/toc.json&bc=/azure/devops/test/breadcrumb/toc.json - displayName: kanban, tesr, board - - name: Track test status - href: track-test-status.md - displayName: track, status - - name: Perform user acceptance testing - href: user-acceptance-testing.md - displayName: user acceptance, testing - name: Test different configurations href: test-different-configurations.md displayName: configuration, testing - name: Repeat a test with different data href: repeat-test-with-different-data.md - displayName: data, testing + displayName: data, testing, shared steps, shared parameters + - name: Test from the Kanban board + href: ../boards/boards/add-run-update-tests.md?toc=/azure/devops/test/toc.json&bc=/azure/devops/test/breadcrumb/toc.json + displayName: inline testing + - name: Perform user acceptance testing + href: user-acceptance-testing.md + displayName: user acceptance, testing - name: Collect diagnostic data href: collect-diagnostic-data.md displayName: diagnostic, data, testing - - name: Manage test results - href: how-long-to-keep-test-results.md - displayName: results, logs - - name: Associate tests with test cases + - name: Delete test artifacts + href: ../boards/backlogs/delete-test-artifacts.md?toc=/azure/devops/test/toc.json&bc=/azure/devops/test/breadcrumb/toc.json.md + displayName: delete, artifacts + - name: Set permissions and access + href: ../organizations/security/set-permissions-access-test.md?toc=/azure/devops/test/toc.json&bc=/azure/devops/test/breadcrumb/toc.json + displayName: access, permissions +- name: Automated tests + items: + - name: Associate automated tests with test cases href: associate-automated-test-with-test-case.md displayName: test cases, automated, associate - name: Run automated tests from test plans href: run-automated-tests-from-test-hub.md displayName: automated, test plans, run - - name: Delete test artifacts - href: ../boards/backlogs/delete-test-artifacts.md?toc=/azure/devops/test/toc.json&bc=/azure/devops/test/breadcrumb/toc.json.md - displayName: delete, artifacts +- name: Track progress and results + items: + - name: Track test status + href: track-test-status.md + displayName: track, status + - name: Requirements traceability + href: ../pipelines/test/requirements-traceability.md?toc=/azure/devops/test/toc.json&bc=/azure/devops/test/breadcrumb/toc.json.md + - name: Set test retention policies + href: how-long-to-keep-test-results.md + displayName: results, logs - name: "Test & Feedback extension" items: - name: Install the extension @@ -88,8 +101,6 @@ displayName: stakeholder, feedback, track - name: Reference items: - - name: Default permissions (Security) - href: manual-test-permissions.md - name: Manual testing FAQs href: reference-qa.md - name: TCM Reporting FAQs @@ -98,14 +109,24 @@ href: ../organizations/projects/tf31002-unable-connect-tfs.md?toc=/azure/devops/test/toc.json&bc=/azure/devops/test/breadcrumb/toc.json - name: REST API Reference items: - - name: Test Management + - name: Test href: /rest/api/azure/devops/test - name: Test Plans - href: /rest/api/azure/devops/testplan/test%20%20plans/list + href: /rest/api/azure/devops/testplan/test-plans + - name: Test Suites + href: /rest/api/azure/devops/testplan/test-suites + - name: Test Variables + href: /rest/api/azure/devops/testplan/variables + - name: Test Results + href: /rest/api/azure/devops/testresults/result-meta-data - name: Resources items: - name: Azure Pipelines Test href: ../pipelines/test/test-glossary.md + - name: Azure Boards, About work items + href: ../boards/work-items/about-work-items.md + - name: Marketplace extensions + href: https://marketplace.visualstudio.com/search?term=test&target=AzureDevOps&category=All%20categories&sortBy=Relevance - name: Azure DevOps Blog, Test href: https://devblogs.microsoft.com/devops/ - name: Plug-ins for Visual Studio diff --git a/docs/test/track-stakeholder-feedback.md b/docs/test/track-stakeholder-feedback.md index d120c1731f4..466172c1aac 100644 --- a/docs/test/track-stakeholder-feedback.md +++ b/docs/test/track-stakeholder-feedback.md @@ -17,19 +17,27 @@ monikerRange: '>= tfs-2017' [!INCLUDE [feedback-header-text](includes/feedback-header-text.md)] + ## Track feedback requests -1. Select your project and open the **Queries** page of the **Agile** section in Azure DevOps or the **Work** hub in TFS. - [Web portal navigation](../project/navigation/index.md) +1. Select your project and open **Boards>Queries** or **Work>Queries**. To learn how, see [Web portal navigation](../project/navigation/index.md). 1. In the list of shared queries, select **Feedback**. This query displays a list of all the feedback responses received. ![Viewing the feedback responses](media/track-stakeholder-feedback/track-stakeholder-feedback-31.png) + Or, create a feedback query with the parameters, as shown. + + ![Editor view for flat-list feedback query](../project/feedback/media/ALM_GF_FeedbackQueryEditor.png) + +1. You should see a list of all active feedback responses for your team project. + + ![Results view of feedback responses](../project/feedback/media/ALM_GF_FeedbackQueryResults.png) + 1. Open the response work item to see the details of the feedback. -## See also +## Related articles * [Request stakeholder feedback using the Test & Feedback extension](request-stakeholder-feedback.md#request) * [Provide stakeholder feedback using the Test & Feedback extension](provide-stakeholder-feedback.md#provide) @@ -37,4 +45,4 @@ monikerRange: '>= tfs-2017' * [Exploratory test and submit feedback directly from your browser](perform-exploratory-tests.md) * [Overview of manual and exploratory testing](index.yml) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/track-test-status.md b/docs/test/track-test-status.md index 3f9e00552eb..beca6107910 100644 --- a/docs/test/track-test-status.md +++ b/docs/test/track-test-status.md @@ -6,8 +6,8 @@ ms.technology: devops-test ms.topic: conceptual ms.author: sdanie author: steved0x -ms.date: 09/04/2019 monikerRange: '>= tfs-2015' +ms.date: 09/14/2021 --- # Track test status @@ -15,8 +15,7 @@ monikerRange: '>= tfs-2015' [!INCLUDE [version-header](includes/version-header.md)] View the status of your planned testing using an out-of-the-box Progress Report and lightweight charts. - -[!INCLUDE [feature-availability](includes/feature-availability.md)] + :::moniker range=">= azure-devops-2020" @@ -30,11 +29,11 @@ To track the progress of more than one test plan or test suite, use the Progress 5. Get a breakdown by test plan and suite to understand where execution is going slow or has many failed or blocked tests. 6. View the progress each suite has made within a test plan and identify the areas that need attention. -### Configuring the report +### Configure the report By navigating to Test Plans > Progress Report, you can start using the report. It requires no setup and is applicable for all Azure DevOps Services organizations. When you view the report, it shows you the status of the test plan you had accessed last. However, using the filter bar, you can select one or more test plans in the project. -### Understanding the report +### Understand the report The progress report consists of three sections: 1. **Summary**: This section provides you with a consolidated view for the selected test plans. To understand what test points are, see [Navigate Test Plans](navigate-test-plans.md). @@ -51,23 +50,22 @@ An example of an unhealthy outcome trend is as follows because as time is progre ![bad trend](media/progress-report/badtrend.png) -### Filtering +### Filter + By using the filter bar you can filter this report by Test Suites, Configuration, Tester, Test Case priority, and Test Case Assigned to. As example: you can filter by configuration 'Chrome' and then 'Edge' to understand where the execution and pass % are higher. Note: Tester is the person to whom the test point is assigned for execution whereas Assigned to is the person who is responsible for the reusable test case. +> [!NOTE] > The test plan is considered Level 1. Its child suites are considered level 2. Their child suites in turn are considered level 3. The Test Suites filter shows only the Level 3 test suites inside the selected test plans. To select a Level 2 test suite select all the Level 3 test suites underneath it. This assumes the Level 2 test suites itself does not have any test points. ### Behavior + When using this progress report, it is good to know the following points: 1. The report lets you view data for one or more Test Plans in a single project. To view data across projects, use the [OData APIs](../report/extend-analytics/data-model-analytics-service.md). This report is also rendered using the same OData APIs. 2. Whenever you visit the report, it will show you the data for the test plan you had accessed last. Currently we do not store your last set of filters and also do not let you store an instance of this report. 3. All the data shown in the report is as per the current suite hierarchy in the selected test plans. Currently, we are not storing the hierarchy history. 4. Data processing for this report is occurring approximately every 15 mins. Do not use this report for real-time analysis or reporting. Expect ~15 mins of duration between a test execution and the same to show up in this report. 5. Data for test plans migrated from TFS/Server will not show up in this report. You can leverage this report for test plans created in Azure DevOps Services on or after September 1, 2019. - -### Provide feedback - -To report any issues you might be having with the Progress Report, go to https://developercommunity.visualstudio.com/content/problem/post.html?space=21. If relevant to the issue you report, include screenshots. - + :::moniker-end ## Charts @@ -105,6 +103,7 @@ for your team. If you already know how to add a chart, jump to the for the test plan or test suite that you selected. + #### Test results examples **What's the test status for a specific test suite?** @@ -194,6 +193,7 @@ create for test cases. page for the test plan or test suite that you selected. + #### Test case examples **How can I track burn down for test case creation?** @@ -239,15 +239,12 @@ You can [configure the dashboard widget](../report/dashboards/widget-catalog.md) to show a range of chart types. You must be a team administrator to configure the dashboard widget, but team members with Stakeholder access can view the charts on the dashboard. + -Learn more about -[dashboards](../report/dashboards/dashboards.md). -Or learn more about -[team administration](../organizations/settings/manage-teams.md). +## Related articles +- [Control how long to keep test results](how-long-to-keep-test-results.md) +- [FAQs for manual testing](reference-qa.md#trackstatus) +- [Widget catalog](../report/dashboards/widget-catalog.md) +- [Team administration](../organizations/settings/manage-teams.md) -## See also -* [FAQs for manual testing](reference-qa.md#trackstatus) -## Next step -> [!div class="nextstepaction"] -> [Control how long to keep test results](how-long-to-keep-test-results.md) diff --git a/docs/test/user-acceptance-testing.md b/docs/test/user-acceptance-testing.md index ddf9b1afaed..212e6ca857c 100644 --- a/docs/test/user-acceptance-testing.md +++ b/docs/test/user-acceptance-testing.md @@ -19,21 +19,17 @@ enable test teams to more easily verify value based on business requirements, and the high quality software demanded by customers. This type of testing is often referred to as -_user acceptance testing_ and is available as a -feature in Azure DevOps and Team -Foundation Server (TFS). +_user acceptance testing_. Typically you create a Test Suite using a formal requirement work item type. However, today's agile teams often prefer to work from User Stories or Product Backlog items as their requirements. -[!INCLUDE [feature-availability](includes/feature-availability.md)] +[!INCLUDE [prerequisites-define](includes/prerequisites-stakeholder.md)] -## Before you start - -You must have already created work items and -a test plan. If not, follow the steps in: +In addition, you must have already created work items and +a test plan. If not, follow the steps provided in the following articles: * [Create your backlog](../boards/backlogs/create-your-backlog.md) * [Create a test plan](create-a-test-plan.md) @@ -48,8 +44,7 @@ to not just assign one tester to a test case (see multiple testers an entire set of tests. This can also be accomplished by selecting the -suite and choosing **Assign testers to run all -tests**. This option also enables the sending of +suite and choosing **Assign testers to run all tests**. This option also enables the sending of emails to the testers assigned to the tests. ![User acceptance testing email notification](media/user-acceptance-testing/uat10.png) @@ -60,6 +55,7 @@ owners who created the original business requirements. + ## Search for and assign testers In scenarios where you have large development teams @@ -93,4 +89,4 @@ the dashboard charts, verify that the columns in your data have been added to the Tests view. For details see [this blog post](https://devblogs.microsoft.com/devops/visual-studio-team-services-manual-testing-tips-charts-iterations-and-runs/). -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] + diff --git a/docs/test/voluntary-stakeholder-feedback.md b/docs/test/voluntary-stakeholder-feedback.md index 9c5135a3881..707b42659d0 100644 --- a/docs/test/voluntary-stakeholder-feedback.md +++ b/docs/test/voluntary-stakeholder-feedback.md @@ -75,7 +75,7 @@ voluntarily, even if you haven't received a specific ![Ending your feedback session](media/shared/provide-stakeholder-feedback-12.png) -## See also +## Related articles * [Request stakeholder feedback using the Test & Feedback extension](request-stakeholder-feedback.md#request) * [Provide stakeholder feedback using the Test & Feedback extension](provide-stakeholder-feedback.md#provide) @@ -83,4 +83,4 @@ voluntarily, even if you haven't received a specific * [Exploratory test and submit feedback directly from your browser](perform-exploratory-tests.md) * [Overview of manual and exploratory testing](index.yml) -[!INCLUDE [help-and-support-footer](includes/help-and-support-footer.md)] +