Skip to content

[Doc Feature][In-Meeting/In-call Experience for Copilot Agents][4405900] #12832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions msteams-platform/Teams-AI-library-tutorial.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ items:

> [!NOTE]
>
> * Custom engine agents are supported only in personal chats, group chats, and meetings.
> * Custom engine agent support for Microsoft 365 Copilot Chat is available only in [public developer preview](https://learn.microsoft.com/microsoftteams/platform/resources/dev-preview/developer-preview-intro?tabs=new-teams-client).
> * Custom engine agent isn't supported in Python.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ ms.owner: ginobuzz

# Build extensible conversation for meeting chat

You can make conversations extensible in Microsoft Teams meetings. Bots, message extensions, cards, and dialogs (referred as task modules in TeamsJS v1.x) can be combined to deliver an intuitive experience.
You can make conversations extensible in Microsoft Teams meetings. Bots, Copilot agents, message extensions, cards and dialogs (referred as task modules in TeamsJS v1.x) can be combined to deliver an intuitive experience.

## Bots

Expand All @@ -27,6 +27,10 @@ The [Bot Framework](https://dev.botframework.com/) is a rich SDK used to create
| Teams conversation bot | This sample app shows how to use different bot conversation events available in bot framework v4. | [View](https://github.com/OfficeDev/Microsoft-Teams-Samples/tree/main/samples/bot-conversation/csharp) | [View](https://github.com/OfficeDev/Microsoft-Teams-Samples/tree/main/samples/bot-conversation/nodejs) | [View](https://github.com/OfficeDev/Microsoft-Teams-Samples/tree/main/samples/bot-conversation/python) |[View](https://github.com/OfficeDev/Microsoft-Teams-Samples/tree/main/samples/bot-conversation/java) |[View](https://github.com/OfficeDev/Microsoft-Teams-Samples/tree/main/samples/bot-conversation/csharp/demo-manifest/bot-conversation.zip)
|Bot samples | Set of bot framework v4 samples | [View](https://github.com/OfficeDev/Microsoft-Teams-Samples#bots-samples-using-the-v4-sdk) | [View](https://github.com/OfficeDev/Microsoft-Teams-Samples#bots-samples-using-the-v4-sdk) | [View](https://github.com/OfficeDev/Microsoft-Teams-Samples#bots-samples-using-the-v4-sdk) |[View](https://github.com/OfficeDev/Microsoft-Teams-Samples#bots-samples-using-the-v4-sdk) |

## Copilot agents

Agents are scoped versions of Microsoft 365 Copilot that act as AI assistants to automate and run business processes. Agents enable you to bring custom knowledge, skills, and process automation into Microsoft 365 Copilot for your organizational needs. You can optimize agents for specific tasks, such as summarize meetings, list action items, and take actions such as updating records. Declarative agents, custom engine agents, plugin promoted declarative agents and all other store copilot agents are supported in Teams meetings. For more information, see [your extensibility options for Microsoft 365 Copilot](/microsoft-365-copilot/extensibility/decision-guide).

## Message extensions

Message extensions allow the users to interact with your web service through buttons and forms in the Teams client. Users can search or initiate actions in an external system from the compose message area, the command box, or directly from a message. You can send back the results of that interaction to the Teams client in the form of a richly formatted card. Implementing message extensions for meeting chats is no different than regular chats. To implement message extension, start with [message extensions](/microsoftteams/platform/messaging-extensions/what-are-messaging-extensions?tabs=dotnet).
Expand All @@ -39,12 +43,12 @@ Cards provide users with various visual, audio, and selectable messages and help

The following table provides the user types and lists the features that each user can access in meetings:

| User type | Bots | Message extensions | Adaptive Cards | Dialogs |
| User type | Bots and Copilot agents | Message extensions | Adaptive Cards | Dialogs |
| :-- | :-- | :-- | :-- | :-- |
| In-tenant | Can view messages, interact with the content, and invoke the bot. | Available | Available | Available |
| Guest, part of the tenant Microsoft Entra ID | Can view messages, interact with the content, and invoke the bot. | Not available | Interactions in the meeting chat are allowed. | Interactions in the meeting chat from Adaptive Card are allowed. |
| Federated, for more information, see [non-standard users](/microsoftteams/non-standard-users). |Can view messages, interact with the content, and invoke the bot. | Not available | Interactions in the meeting chat are allowed. | Interactions in the meeting chat from Adaptive Card are allowed. |
| Anonymous | Can view and interact with the messages sent by the bot. Can't invoke the bot. | Not available | Interactions in the meeting chat are allowed. | Interactions in the meeting chat from Adaptive Card are allowed. |
| In-tenant | Can view messages, interact with the content, and invoke the bot and agent. | Available | Available | Available |
| Guest, part of the tenant Microsoft Entra ID | Can view messages, interact with the content, and invoke the bot and agent. | Not available | Interactions in the meeting chat are allowed. | Interactions in the meeting chat from Adaptive Card are allowed. |
| Federated, for more information, see [non-standard users](/microsoftteams/non-standard-users). |Can view messages, interact with the content, and invoke the bot and agent. | Not available | Interactions in the meeting chat are allowed. | Interactions in the meeting chat from Adaptive Card are allowed. |
| Anonymous | Can view and interact with the messages sent by the bot. Can't invoke the bot and agent. | Not available | Interactions in the meeting chat are allowed. | Interactions in the meeting chat from Adaptive Card are allowed. |

> [!Note]
>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,13 +69,12 @@ Meetings enable collaboration, partnership, informed communication, and shared f
:::row-end:::

<br>
You can create scenes for meetings, provide notifications to users, populate in-meeting dialogs, and more with meeting app extensibility.

Custom apps built for your org (LOB apps) for meetings and calls are available in [Government Community Cloud (GCC), GCC High, Department of Defense (DoD)](~/concepts/cloud-overview.md), and [Teams operated by 21Vianet](~/concepts/sovereign-cloud.md) environments.

Third-party apps built for meetings and calls are available in GCC but aren't available for GCC High, DoD, and Teams operated by 21Vianet environments.

You can use Copilot agents in meetings, create scenes, provide notifications to users, populate in-meeting dialogs, and more with meeting app extensibility.

> [!IMPORTANT]
>
> * Custom apps built for your org (LOB apps) for meetings and calls are available in [Government Community Cloud (GCC), GCC High, Department of Defense (DoD)](~/concepts/cloud-overview.md), and [Teams operated by 21Vianet](~/concepts/sovereign-cloud.md) environments.
> * Third-party apps built for meetings and calls are available in GCC but aren't available for GCC High, DoD, and Teams operated by 21Vianet environments.

You must be familiar with the concepts in this article to create custom meeting experiences with apps in Microsoft Teams.

Expand All @@ -89,9 +88,10 @@ Teams supports access to apps during meeting for the following meeting types:
* [**Group calls**](https://support.microsoft.com/office/start-a-call-from-a-chat-in-teams-f5138c9d-df4c-43d8-9cf6-53400c1a7798): Calls initiated in a group chat.
* [**Instant meetings**](https://support.microsoft.com/office/start-an-instant-meeting-in-teams-ff95e53f-8231-4739-87fa-00b9723f4ef5): Meetings initiated through **Meet now** button in Teams calendar.
* [**Webinar**](https://support.microsoft.com/office/get-started-with-teams-webinars-42f3f874-22dc-4289-b53f-bbc1a69013e3): Webinar initiated through **Webinar** button under **New Meeting** dropdown.
* [**Calling**](/microsoftteams/cloud-voice-landing-page#public-switched-telephone-network-connectivity-options): Extend Teams Phone to initiate calls outside the organization through PSTN connectivity.
* [**Calling**](/microsoftteams/cloud-voice-landing-page#public-switched-telephone-network-connectivity-options): Extend Teams Phone to initiate calls outside the organization through (Public Switched Telephone Network) PSTN connectivity. Copilot agents aren't supported in PSTN calls.

Learn more about [Teams meetings, expiration, and policies](/microsoftteams/meeting-expiration) and [meetings, webinars, and live events](/microsoftteams/quick-start-meetings-live-events).

> [!NOTE]
>
> Apps aren't supported in the following:
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -790,7 +790,9 @@ If you already have a bot on Teams, you can upgrade your existing bots to custom

> [!NOTE]
>
> Custom engine agent isn't supported in Python.
> * Custom engine agents are supported only in personal chats, group chats, and meetings.
> * Custom engine agent support for Microsoft 365 Copilot Chat is available only in [public developer preview](../../../resources/dev-preview/developer-preview-intro.md).
> * Custom engine agent isn't supported in Python.

Here is the required to-do list to upgrade your bot to custom engine agent:

Expand Down
8 changes: 7 additions & 1 deletion msteams-platform/sbs-Teams-AI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,20 @@ items:
- durationInMinutes: 1
content: |
You can create a custom engine agent for Microsoft Teams to gain precise control over light operations. This showcases the effectiveness of the language model in accurately understanding and executing user intent, aligning it with app actions.

> [!NOTE]
>
> * Custom engine agents are supported only in personal chats, group chats, and meetings.
> * Custom engine agent support for Microsoft 365 Copilot Chat is available only in [public developer preview](https://learn.microsoft.com/microsoftteams/platform/resources/dev-preview/developer-preview-intro?tabs=new-teams-client).
> * Custom engine agent isn't supported in Python.

In this tutorial, you'll learn:

- How to set up a new project with Microsoft 365 Agents Toolkit (previously known as Teams Toolkit).
- How to import Teams AI library capabilities.
- How to deploy your custom engine agent in Teams and control over light operations.

You can see the following output:
You can see the following output:

:::image type="content" source="assets/images/bots/ai-concepts-lightbot.png" alt-text="Screenshot shows an example of the chat window in the local machine.":::

Expand Down