diff --git a/files/en-us/web/api/createmonitor/index.md b/files/en-us/web/api/createmonitor/index.md index 92e2a2d11cc420e..1374233fe33d4f1 100644 --- a/files/en-us/web/api/createmonitor/index.md +++ b/files/en-us/web/api/createmonitor/index.md @@ -9,12 +9,13 @@ browser-compat: api.CreateMonitor {{APIRef("Summarizer API")}}{{SeeCompatTable}}{{securecontext_header}} -The **`CreateMonitor`** interface provides information on the progress of an AI model download or some fine-tuning data for the model. +The **`CreateMonitor`** interface provides information on the progress of an AI model download, for example a language pack or some fine-tuning data. It can be used via: -- {{domxref("Summarizer.create_static", "Summarizer.create()")}} - {{domxref("LanguageDetector.create_static", "LanguageDetector.create()")}} +- {{domxref("LanguageModel.create_static", "LanguageModel.create()")}} +- {{domxref("Summarizer.create_static", "Summarizer.create()")}} - {{domxref("Translator.create_static", "Translator.create()")}} {{InheritanceDiagram}} @@ -56,5 +57,7 @@ const summary = await summarizer.summarize(myText); ## See also +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) - [Using the Summarizer API](/en-US/docs/Web/API/Summarizer_API/Using) +- [Using the Translator and Language Detector APIs](/en-US/docs/Web/API/Translator_and_Language_Detector_APIs/Using) - [Web AI demos](https://chrome.dev/web-ai-demos/) on chrome.dev. diff --git a/files/en-us/web/api/languagemodel/append/index.md b/files/en-us/web/api/languagemodel/append/index.md new file mode 100644 index 000000000000000..85670ae4d453414 --- /dev/null +++ b/files/en-us/web/api/languagemodel/append/index.md @@ -0,0 +1,161 @@ +--- +title: "LanguageModel: append() method" +short-title: append() +slug: Web/API/LanguageModel/append +page-type: web-api-instance-method +browser-compat: api.LanguageModel.append +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`append()`** method of the {{domxref("LanguageModel")}} interface adds content to the session's context window without generating a model response. It returns a {{jsxref("Promise")}} that resolves when the content has been successfully loaded into context. Use this method to preload a context before asking the model a question. + +A context may be a document, conversation, history, or background information. You can call the `append()` method at any point during the session's lifetime. + +## Syntax + +```js-nolint +append(input) +append(input, options) +``` + +### Parameters + +- `input` + - : The content to append to the context window. This is either: + - A string — Shorthand for a single textual message. + - An array of objects, each representing a single message in a conversation with a language model. + Objects may have the following properties: + - `role` + - : A string indicating the point of view the message is phrased from. Must be one of: + - `"system"` + - : A system-level instruction that guides the model's overall behavior. This must be the first instruction passed to the model. + - `"user"` + - : A message from the user, which the API should respond to. + - `"assistant"` + - : An input that provides context for the AI assistant, such as its persona or the format of its responses. Such messages mainly serve to provide context/history, and further shape how the model responds. + - `content` + - : A string representing a textual prompt, or an array of objects. Each object includes the following properties: + - `type` + - : An enumerated value representing the type of content. This can be one of: + - `audio` + - : Audio content. + - `image` + - : Image content. + - `text` + - : Textual content. + - `"tool-call"` + - : A tool invocation issued by the model. + - `"tool-response"` + - : The result of a tool invocation. + - `value` + - : The content of the message. If the `type` is `text`, this is always a string. If the `type` is `audio` or `image`, the `value` can be one of several different object types; see [What data types are accepted?](/en-US/docs/Web/API/Prompt_API/Multimodal#what_data_types_are_accepted). + - `prefix` {{optional_inline}} + - : A boolean, defaulting to `false`. When `true`, the message is treated as a prefix for the model's next generated response rather than a complete turn. +- `options` {{optional_inline}} + - : An object representing the options that can be passed. Properties include: + - `signal` + - : An {{domxref("AbortSignal")}} to cancel the append operation. + +### Return value + +A {{jsxref("Promise")}} that resolves with `undefined` when the content has been prefilled into the context window, or rejects with one of the following exception values on failure. + +### Exceptions + +- `AbortError` {{domxref("DOMException")}} + - : Thrown if the operation was cancelled via the `signal` option. +- `NotAllowedError` {{domxref("DOMException")}} + - : Thrown if usage of the method is blocked by a {{httpheader("Permissions-Policy/language-model", "language-model")}} {{httpheader("Permissions-Policy")}}. +- `NotSupportedError` {{domxref("DOMException")}} + - : Thrown if: + - A message's `role` is `"assistant"` and its `type` is anything other than `"text"`. + - A message's `type` is `text` and its `value` is not a string. + - The input or output text is in a language the user agent doesn't support for prompting. + - A message's `type` is `"image"` or `"audio"` but the type was not listed in `expectedInputs`, or the `value` is not an [accepted data type](/en-US/docs/Web/API/Prompt_API/Multimodal#what_data_types_are_accepted). +- `OperationError` {{domxref("DOMException")}} + - : Thrown if prefilling fails for any other reason not listed in the other exception types. +- `QuotaExceededError` {{domxref("DOMException")}} + - : Thrown if appending `input` would cause the session's context usage to exceed the model's {{domxref("LanguageModel.contextWindow")}}. +- `SyntaxError` {{domxref("DOMException")}} + - : Thrown if: + - No messages are included in the messages array. + - A message's `prefix` property is set to `true` and: + - The message's `role` is not `assistant`. + - The message is not the last item in the messages array. +- `TypeError` {{domxref("DOMException")}} + - : Thrown if a message's `role` is `system` but it was not the first message passed to the context. + +## Examples + +### Append context before prompting + +This example shows how to append to a context for the user role before calling `prompt()`. +Note that we can just specify text input (`documentText`) in this case, because `user` is the default role. + +```js +const documentText = "This is my important essay..."; +const session = await LanguageModel.create(); + +// Preload the document text into context +await session.append(documentText); + +// Now ask questions about the document +const summary = await session.prompt( + "Summarize the key points of this document.", +); +console.log(summary); +``` + +### Appending context with an abort signal + +An abort signal lets you cancel an append operation. The example below passes an {{domxref("AbortSignal")}} to the `signal` member and calls its `abort()` method after 3 seconds. + +```js +const controller = new AbortController(); +setTimeout(() => controller.abort(), 3000); + +try { + await session.append( + "Here is some background context for future questions.", + { + signal: controller.signal, + }, + ); + console.log("Context appended successfully."); +} catch (err) { + if (err.name === "AbortError") { + console.log("Append was aborted."); + } +} +``` + +### Checking context usage after appending + +The code below shows how to log the number of tokens used after appending a large amount of context. + +```js +const largeDocument = "This is my large body of text..."; +const session = await LanguageModel.create(); +await session.append(largeDocument); + +console.log( + `Context used: ${session.contextUsage} / ${session.contextWindow} tokens`, +); +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- {{domxref("LanguageModel.prompt()")}} +- {{domxref("LanguageModel.measureContextUsage()")}} +- [Prompt API](/en-US/docs/Web/API/Prompt_API) +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) +- [Adding context with initial and ongoing prompt inputs](/en-US/docs/Web/API/Prompt_API/Adding_context) diff --git a/files/en-us/web/api/languagemodel/availability_static/index.md b/files/en-us/web/api/languagemodel/availability_static/index.md new file mode 100644 index 000000000000000..01ee35b059f85be --- /dev/null +++ b/files/en-us/web/api/languagemodel/availability_static/index.md @@ -0,0 +1,173 @@ +--- +title: "LanguageModel: availability() static method" +short-title: availability() +slug: Web/API/LanguageModel/availability_static +page-type: web-api-static-method +browser-compat: api.LanguageModel.availability_static +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`availability()`** static method of the {{domxref("LanguageModel")}} interface returns a status identifier indicating whether the browser's language model supports a given set of configuration options, without creating a session or triggering a download. + +Use `availability()` before calling {{domxref("LanguageModel.create_static", "LanguageModel.create()")}} to determine whether the desired configuration is supported. This avoids initiating a session only to have it fail, and lets you provide a meaningful fallback to users when the configuration is not supported. + +## Syntax + +```js-nolint +LanguageModel.availability() +LanguageModel.availability(options) +``` + +### Parameters + +- `options` {{optional_inline}} + - : An object that represents the base set of options used when checking language model support. + Properties include: + - `expectedInputs` {{optional_inline}} + - : An array of objects representing the required input modalities and languages. + Each object can include the following properties: + - `type` + - : An enumerated value indicating the content type. Must be one of: + - `"text"` + - : Plain text content. + - `"image"` + - : Image content. + - `"audio"` + - : Audio content. + - `"tool-call"` + - : A tool invocation issued by the model. + - `"tool-response"` + - : The result of a tool invocation. + - `languages` {{optional_inline}} + - : An array of strings containing [BCP 47](https://www.rfc-editor.org/rfc/rfc5646) language tags (for example, `"en"`, `"fr"`, `"ja"`) representing languages that the session is expected to handle. The user agent uses this list to determine whether the model supports the specified languages. + - `expectedOutputs` + - : An array of objects representing the required output modalities and languages. + Each object can include the following properties: + - `type` + - : An enumerated value indicating the content type. Must be one of: + - `"text"` + - : Textual content. + - `"image"` + - : Image content. + - `"audio"` + - : Audio content. + - `"tool-call"` + - : A tool invocation issued by the model. + - `"tool-response"` + - : The result of a tool invocation. + - `languages` {{optional_inline}} + - : An array of strings containing [BCP 47](https://www.rfc-editor.org/rfc/rfc5646) language tags (for example, `"en"`, `"fr"`, `"ja"`) that the session is expected to output. + - `tools` + - : An array of objects representing tools available to the AI. + Each object can include the following properties: + - `name` + - : A string giving the tool a unique name the model uses to refer to it when issuing a tool call. + - `description` + - : A string describing what the tool does. + The model uses this description to decide when and whether to invoke the tool. + - `inputSchema` + - : An object containing a [JSON Schema](https://json-schema.org/) that describes the tool's input parameters. + The model uses this schema to construct the arguments it passes to the tool's `execute` function. + - `execute` + - : A callback function that the user agent invokes when the model calls this tool. + It can receive any arguments provided by the model as appropriate and returns a {{jsxref("Promise")}} that resolves with a {{jsxref("String")}} representing the tool's result. + +### Return value + +A {{jsxref("Promise")}} that resolves with one of the values listed below. + +- `"available"` + - : The model is ready to use with the given options. +- `"downloadable"` + - : The model can support the given options but needs to download additional data to do so. The download has not yet started. +- `"downloading"` + - : The model can support the given options with an additional data download. The download is currently in progress. +- `"unavailable"` + - : The model cannot support the given options, or the user agent cannot determine availability, for example, due to a [transient activation](/en-US/docs/Glossary/Transient_activation) error. In that case, the caller should retry or fall back to an alternative implementation. + +### Exceptions + +- `InvalidStateError` {{domxref("DOMException")}} + - : Thrown if the calling document is not fully active. +- `NotAllowedError` {{domxref("DOMException")}} + - : Thrown if usage of the method is blocked by a {{httpheader("Permissions-Policy/language-model", "language-model")}} {{httpheader("Permissions-Policy")}}. + +## Examples + +### Requesting input support + +This example shows how to determine whether text and image inputs are supported by the model. + +```js +const status = await LanguageModel.availability({ + expectedInputs: [{ type: "text" }, { type: "image" }], +}); +``` + +### Checking availability for a specific language + +This example tests whether the model supports English before asking it to translate Japanese text to English. + +```js +const status = await LanguageModel.availability({ + expectedInputs: [{ type: "text", languages: ["ja"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], +}); + +if (status === "available") { + const session = await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["ja"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + }); + + const translation = await session.prompt([ + { + role: "user", + content: "Translate the following text into English", + }, + { + role: "user", + content: "桜はきれいです", + }, + ]); + + console.log(translation); +} +``` + +### Checking availability for multimodal input + +[Multimodal input](/en-US/docs/Web/API/Prompt_API/Multimodal) describes sessions that can use more than one type of input, such as text and images. +Since the availability of input types varies by language model, your code should check the availability of desired modes before creating a session. +An example is shown here. + +```js +const availability = await LanguageModel.availability({ + expectedInputs: [{ type: "text" }, { type: "image" }], + expectedOutputs: [{ type: "text", languages: ["en"] }], +}); + +if (availability === "unavailable") { + console.warn("This configuration is not supported."); +} else { + const session = await LanguageModel.create({ + expectedInputs: [{ type: "text" }, { type: "image" }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + }); +} +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- {{domxref("LanguageModel.create_static", "LanguageModel.create()")}} +- [Prompt API](/en-US/docs/Web/API/Prompt_API) +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) diff --git a/files/en-us/web/api/languagemodel/clone/index.md b/files/en-us/web/api/languagemodel/clone/index.md new file mode 100644 index 000000000000000..a9e2847e890adfd --- /dev/null +++ b/files/en-us/web/api/languagemodel/clone/index.md @@ -0,0 +1,125 @@ +--- +title: "LanguageModel: clone() method" +short-title: clone() +slug: Web/API/LanguageModel/clone +page-type: web-api-instance-method +browser-compat: api.LanguageModel.clone +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`clone()`** method of the {{domxref("LanguageModel")}} interface creates a copy of the `LanguageModel` it is called on, including its full context window state. The cloned session can be used independently without affecting the original. + +The original and the clone share the same context history up to the point of cloning, enabling you to explore multiple response paths or test variations without starting from scratch. + +For example, you might build a shared context using {{domxref("LanguageModel.append()", "append()")}} or early {{domxref("LanguageModel.prompt()", "prompt()")}} `prompt()` calls, clone the session, and then send different follow-up prompts to each clone in parallel. + +## Syntax + +```js-nolint +clone() +clone(options) +``` + +### Parameters + +- `options` {{optional_inline}} + - : An object representing the options that can be passed. If this argument is absent, the `options` from the original session, such as its abort signal, are used. + Properties include: + - `signal` + - : An {{domxref("AbortSignal")}} to cancel the clone operation. + +### Return value + +A {{jsxref("Promise")}} that resolves with a cloned {{domxref("LanguageModel")}} instance. + +### Exceptions + +- `AbortError` {{domxref("DOMException")}} + - : Thrown if the operation was cancelled via the `signal` option. +- `NotAllowedError` {{domxref("DOMException")}} + - : Thrown if usage of the method is blocked by a {{httpheader("Permissions-Policy/language-model", "language-model")}} {{httpheader("Permissions-Policy")}}. +- `OperationError` {{domxref("DOMException")}} + - : Thrown if cloning fails for any other reason not listed in the other exception types. + +## Examples + +### Exploring multiple response paths + +The following example shows how to explore different response paths. First, it creates a single session with the start of a story. Then it clones the original session twice before prompting for different endings. This approach preserves the original session in case more exploration is wanted. + +```js +const session = await LanguageModel.create({ + initialPrompts: [ + { role: "system", content: "You are a creative writing assistant." }, + ], +}); + +await session.append( + "The story begins in a small coastal town during a storm.", +); + +const [clone1, clone2] = await Promise.all([session.clone(), session.clone()]); + +const [ending1, ending2] = await Promise.all([ + clone1.prompt("Write a happy ending."), + clone2.prompt("Write a mysterious ending."), +]); + +console.log("Happy ending:", ending1); +console.log("Mysterious ending:", ending2); +``` + +### Cloning to retry after a context overflow + +This example uses a checkpoint and rollback pattern to save the state of a session before attempting to append a large amount of data. Cloning the session before calling `append()` allows the app to restore the state if the context window is exceeded. + +```js +const veryLargeDocument = "This is my very long story..."; +let session = await LanguageModel.create(); +const checkpoint = await session.clone(); + +try { + await session.append(veryLargeDocument); +} catch (err) { + if (err.name === "QuotaExceededError") { + console.warn("Document too large."); + session = checkpoint; + } +} +``` + +### Cloning a session with an abort signal + +The following example creates a timeout to abort the clone operation if it takes more than three seconds. + +```js +const controller = new AbortController(); +setTimeout(() => controller.abort(), 3000); + +try { + const clonedSession = await session.clone({ + signal: controller.signal, + }); + console.log("Session cloned successfully."); +} catch (err) { + if (err.name === "AbortError") { + console.log("Clone operation was aborted."); + } +} +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- {{domxref("LanguageModel.append()")}} +- [Prompt API](/en-US/docs/Web/API/Prompt_API) +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) +- [Adding context with initial and ongoing prompt inputs](/en-US/docs/Web/API/Prompt_API/Adding_context) diff --git a/files/en-us/web/api/languagemodel/contextoverflow_event/index.md b/files/en-us/web/api/languagemodel/contextoverflow_event/index.md new file mode 100644 index 000000000000000..a05092cf21821d1 --- /dev/null +++ b/files/en-us/web/api/languagemodel/contextoverflow_event/index.md @@ -0,0 +1,92 @@ +--- +title: "LanguageModel: contextoverflow event" +short-title: contextoverflow +slug: Web/API/LanguageModel/contextoverflow_event +page-type: web-api-event +browser-compat: api.LanguageModel.contextoverflow_event +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`contextoverflow`** event fires on a {{domxref("LanguageModel")}} instance when a call to {{domxref("LanguageModel.prompt()", "prompt()")}}, {{domxref("LanguageModel.promptStreaming()", "promptStreaming()")}}, or {{domxref("LanguageModel.append()", "append()")}} causes the session's {{domxref("LanguageModel.contextUsage", "contextUsage")}} to exceed the {{domxref("LanguageModel.contextWindow", "contextWindow")}}. + +## Syntax + +Use the event name in methods like {{domxref("EventTarget.addEventListener", "addEventListener()")}}, or set an event handler property. + +```js-nolint +addEventListener("contextoverflow", (event) => {}) + +oncontextoverflow = (event) => {} +``` + +## Event type + +A generic {{domxref("Event")}}. + +## Examples + +### Reacting to a context overflow + +The code below shows two methods of creating an event listener for the `contextoverflow` event. + +```js +const session = await LanguageModel.create(); + +session.addEventListener("contextoverflow", () => { + console.warn("Context overflow detected."); +}); +``` + +Alternatively: + +```js +const session = await LanguageModel.create(); + +session.oncontextoverflow = () => { + console.warn( + "The session's context window is full. " + + "Consider cloning the session or starting a new one.", + ); +}; +``` + +### Resetting the session on overflow + +The following example creates a new session when the `contextoverflow` event is triggered. + +```js +let session = await LanguageModel.create({ + initialPrompts: [{ role: "system", content: "You are a helpful assistant." }], +}); + +session.addEventListener("contextoverflow", async () => { + console.log("Context full — creating a fresh session."); + session.destroy(); + session = await LanguageModel.create({ + initialPrompts: [ + { role: "system", content: "You are a helpful assistant." }, + ], + }); +}); + +async function chat(userMessage) { + const response = await session.prompt(userMessage); + return response; +} +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- {{domxref("LanguageModel.contextUsage")}} +- {{domxref("LanguageModel.contextWindow")}} +- {{domxref("LanguageModel.measureContextUsage()")}} +- [Prompt API](/en-US/docs/Web/API/Prompt_API) diff --git a/files/en-us/web/api/languagemodel/contextusage/index.md b/files/en-us/web/api/languagemodel/contextusage/index.md new file mode 100644 index 000000000000000..5f2c70396945dc1 --- /dev/null +++ b/files/en-us/web/api/languagemodel/contextusage/index.md @@ -0,0 +1,77 @@ +--- +title: "LanguageModel: contextUsage property" +short-title: contextUsage +slug: Web/API/LanguageModel/contextUsage +page-type: web-api-instance-property +browser-compat: api.LanguageModel.contextUsage +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`contextUsage`** read-only property of the {{domxref("LanguageModel")}} interface returns the number of context window tokens currently consumed by the session that calls it, including initial prompts and all subsequent turns. + +This value increases every time you call {{domxref("LanguageModel.prompt()", "prompt()")}}, {{domxref("LanguageModel.promptStreaming()", "promptStreaming()")}}, or {{domxref("LanguageModel.append()", "append()")}}. + +Compare `contextUsage` with {{domxref("LanguageModel.contextWindow")}} to determine how many tokens remain. When `contextUsage` would exceed `contextWindow`, subsequent method calls throw a `QuotaExceededError` and the {{domxref("LanguageModel.contextoverflow_event", "contextoverflow")}} event fires. + +To estimate how many tokens a new prompt would use before sending it, call {{domxref("LanguageModel.measureContextUsage()")}}. + +## Value + +A number representing the current context window usage in tokens. + +## Examples + +### Monitoring context usage during a conversation + +This example writes context usage to the console after a session prompt completes. + +```js +const session = await LanguageModel.create(); + +await session.prompt("Tell me about the history of the internet."); + +console.log( + `Context used: ${session.contextUsage} / ${session.contextWindow} tokens`, +); +``` + +### Warning when the context is nearly full + +The following example uses a function to verify that context is available before calling {{domxref("LanguageModel.prompt()")}}. It first calculates the remaining context and passes that value to `measureContextUsage()`. If `needed` is less than or equal to `remaining`, it returns `true` and the session continues. + +```js +const promptText = "Let me ask you an interesting question..."; + +async function contextAvailable(promptText) { + const remaining = session.contextWindow - session.contextUsage; + const needed = await session.measureContextUsage(promptText); + + return needed <= remaining; +} + +const session = await LanguageModel.create(); + +if (await contextAvailable(promptText)) { + const response = await session.prompt(promptText); + console.log(response); +} else { + console.warn("Prompt skipped: Not enough context window remaining."); +} +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- {{domxref("LanguageModel.contextWindow")}} +- {{domxref("LanguageModel.measureContextUsage()")}} +- {{domxref("LanguageModel.oncontextoverflow")}} +- [Prompt API](/en-US/docs/Web/API/Prompt_API) +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) diff --git a/files/en-us/web/api/languagemodel/contextwindow/index.md b/files/en-us/web/api/languagemodel/contextwindow/index.md new file mode 100644 index 000000000000000..da3c9e25e3347f7 --- /dev/null +++ b/files/en-us/web/api/languagemodel/contextwindow/index.md @@ -0,0 +1,63 @@ +--- +title: "LanguageModel: contextWindow property" +short-title: contextWindow +slug: Web/API/LanguageModel/contextWindow +page-type: web-api-instance-property +browser-compat: api.LanguageModel.contextWindow +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`contextWindow`** read-only property of the {{domxref("LanguageModel")}} interface returns the total number of context window tokens available for this session. It is set when the session is created and does not change during the session's lifetime. + +Compare `contextWindow` against {{domxref("LanguageModel.contextUsage", "contextUsage")}} to determine how many tokens remain. Use {{domxref("LanguageModel.measureContextUsage()", "measureContextUsage()")}} to estimate how many tokens a new prompt would consume before sending it. + +The value is implementation-specific and varies depending on the model, device capabilities, and the session's configuration. A value of `Infinity` indicates that the user agent does not impose a hard limit. + +## Value + +A number representing the session's context window capacity in tokens. This value may be `Infinity` if the user agent does not impose a specific limit beyond available memory or JavaScript string constraints. + +## Examples + +### Warning when the context is nearly full + +The following example uses a function to verify that context is available before calling {{domxref("LanguageModel.prompt()")}}. It first calculates the remaining context and passes that value to `measureContextUsage()`. If `needed` is less than or equal to `remaining`, it returns `true` and the session continues. + +```js +const promptText = "Let me ask you an interesting question..."; + +async function contextAvailable(promptText) { + if (session.contextWindow === Infinity) { + return true; + } + const remaining = session.contextWindow - session.contextUsage; + const needed = await session.measureContextUsage(promptText); + + return needed <= remaining; +} + +const session = await LanguageModel.create(); + +if (await contextAvailable(promptText)) { + const response = await session.prompt(promptText); + console.log(response); +} else { + console.warn("Prompt skipped: Not enough context window remaining."); +} +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- {{domxref("LanguageModel.contextUsage")}} +- {{domxref("LanguageModel.measureContextUsage()")}} +- [Prompt API](/en-US/docs/Web/API/Prompt_API) +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) diff --git a/files/en-us/web/api/languagemodel/create_static/index.md b/files/en-us/web/api/languagemodel/create_static/index.md new file mode 100644 index 000000000000000..20acf776f79a2df --- /dev/null +++ b/files/en-us/web/api/languagemodel/create_static/index.md @@ -0,0 +1,309 @@ +--- +title: "LanguageModel: create() static method" +short-title: create() +slug: Web/API/LanguageModel/create_static +page-type: web-api-static-method +browser-compat: api.LanguageModel.create_static +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`create()`** static method of the {{domxref("LanguageModel")}} interface constructs a new {{domxref("LanguageModel")}} instance, automatically downloading the corresponding model data if it is not already available. + +## Syntax + +```js-nolint +LanguageModel.create() +LanguageModel.create(options) +``` + +### Parameters + +- `options` {{optional_inline}} + - : An object representing the options for creating a {{domxref("LanguageModel")}} session. Properties include: + - `expectedInputs` + - : An array of objects representing the required input modalities and languages. + Each object can include the following properties: + - `type` + - : An enumerated value indicating the content type. Must be one of: + - `"text"` + - : Plain text content. + - `"image"` + - : Image content. + - `"audio"` + - : Audio content. + - `"tool-call"` + - : A tool invocation issued by the model. + - `"tool-response"` + - : The result of a tool invocation. + - `languages` {{optional_inline}} + - : An array of strings containing [BCP 47](https://www.rfc-editor.org/rfc/rfc5646) language tags (for example, `"en"`, `"fr"`, `"ja"`) that the session is expected to handle for this content type. The user agent uses this list to determine whether the model supports the specified languages and to select appropriate model components or fine-tunings. + - `expectedOutputs` + - : An array of objects representing the required output modalities and languages. + Each object can include the following properties: + - `type` + - : An enumerated value indicating the content type. Must be one of: + - `"text"` + - : Plain text content. + - `"image"` + - : Image content. + - `"audio"` + - : Audio content. + - `"tool-call"` + - : A tool invocation issued by the model. + - `"tool-response"` + - : The result of a tool invocation. + - `languages` {{optional_inline}} + - : An array of strings containing [BCP 47](https://www.rfc-editor.org/rfc/rfc5646) language tags (for example, `"en"`, `"fr"`, `"ja"`) that the session is expected to handle for this content type. The user agent uses this list to determine whether the model supports the specified languages and to select appropriate model components or fine-tunings. + - `initialPrompts` + - : An array of objects representing messages passed during the creation of a language model session. This allows the model to "remember" instructions or previous dialogue without resending them with every new query. Each object can include the following properties: + - `role` + - : A string indicating the point of view the message is phrased from. Must be one of: + - `"system"` + - : A system-level instruction that guides the model's overall behavior. This must be the first instruction passed to the model. + - `"user"` + - : A message from the user, which the API should respond to. + - `"assistant"` + - : An input that provides context for the AI assistant, such as its persona or the format of its responses. Such messages mainly serve to provide context/history, and further shape how the model responds. + - `content` + - : A string representing a textual prompt, or an array of objects. Each object includes the following properties: + - `type` + - : An enumerated value representing the type of content. This can be one of: + - `audio` + - : Audio content. + - `image` + - : Image content. + - `text` + - : Textual content. + - `"tool-call"` + - : A tool invocation issued by the model. + - `"tool-response"` + - : The result of a tool invocation. + - `value` + - : The content of the message. If the `type` is `text`, this is always a string. If the `type` is `audio` or `image`, the `value` can be one of several different object types; see [What data types are accepted?](/en-US/docs/Web/API/Prompt_API/Multimodal#what_data_types_are_accepted). + - `prefix` {{optional_inline}} + - : A boolean, defaulting to `false`. When `true`, the message is treated as a prefix for the model's next generated response rather than a complete turn. + - `monitor` + - : A reference to a {{domxref("CreateMonitor")}} callback function to receive download progress events. + - `signal` + - : An {{domxref("AbortSignal")}} to cancel session creation. + - `tools` + - : An array of objects representing tools available to the AI. + Each object can include the following properties: + - `name` + - : A string giving the tool a unique name the model uses to refer to it when issuing a tool call. + - `description` + - : A string describing what the tool does. The model uses this description to decide if and when to invoke the tool. + - `inputSchema` + - : A [JSON Schema](https://json-schema.org/) that describes the tool's input parameters. The model uses this schema to construct the arguments it passes to the tool's `execute` function. + - `execute` + - : A callback function that the user agent invokes when the model calls this tool. Its arguments are specific to the model being used. It must return a {{jsxref("Promise")}} that resolves with a {{jsxref("String")}} representing the tool's result. + +### Return value + +A {{jsxref("Promise")}} that resolves with a new {{domxref("LanguageModel")}} instance. + +### Exceptions + +- `AbortError` {{domxref("DOMException")}} + - : Thrown if the operation was aborted via the `signal` option. +- `InvalidStateError` {{domxref("DOMException")}} + - : Thrown if the calling document is not fully active. +- `NotAllowedError` {{domxref("DOMException")}} + - : Thrown if usage of the method is blocked by a {{httpheader("Permissions-Policy/language-model", "language-model")}} {{httpheader("Permissions-Policy")}}. +- `NotSupportedError` {{domxref("DOMException")}} + - : Thrown if: + - A message's `role` is `"assistant"` and its `type` is anything other than `"text"`. + - A message's `type` is `text` and its `value` is not a string. + - The input or output text is in a language the user agent doesn't support for prompting. + - A message's `type` is `"image"` or `"audio"` but the type was not listed in `expectedInputs`, or the `value` is not an [accepted data type](/en-US/docs/Web/API/Prompt_API/Multimodal#what_data_types_are_accepted). +- `OperationError` {{domxref("DOMException")}} + - : Thrown if creation fails for any other reason not listed in the other exception types. +- `QuotaExceededError` {{domxref("DOMException")}} + - : Thrown if the content provided in `initialPrompts` exceeds the model's {{domxref("LanguageModel.contextWindow")}}. +- `SyntaxError` {{domxref("DOMException")}} + - : Thrown if: + - No messages are included in the messages array. + - A message's `prefix` property is set to `true` and: + - The message's `role` is not `assistant`. + - The message is not the last item in the messages array. +- `TypeError` {{domxref("DOMException")}} + - : Thrown if: + - A message's `role` is `system` but it was not the first message passed to the context. + +## Description + +The `create()` method constructs a new language model session, automatically downloading the model if it is not already available. +You can monitor progress of a model download using the [`monitor`](#monitor) option. + +Before calling `create()`, use {{domxref("LanguageModel.availability_static", "LanguageModel.availability()")}} to check whether the desired configuration is supported. + +Once a session is created, use its instance methods — {{domxref("LanguageModel.prompt()")}}, {{domxref("LanguageModel.promptStreaming()")}}, {{domxref("LanguageModel.append()")}}, and others — to interact with the model. + +## Security + +[Transient user activation](/en-US/docs/Web/Security/Defenses/User_activation) is required. The user has to interact with the page or a UI element for this feature to work. + +## Examples + +### Creating a basic session + +This example creates a default session and then prompts it for the result of summing `2` and `2`. +Note that text is supported by default, so the downloaded model should be suitable for this case. + +```js +const session = await LanguageModel.create(); +const answer = await session.prompt("What is 2 + 2?"); +console.log(answer); +``` + +See also [Using the Prompt API > Creating a `LanguageModel` session](/en-US/docs/Web/API/Prompt_API/Using#creating_a_languagemodel_session). + +### Creating a session with a system prompt + +The following example provides the AI with instructions on the persona to adopt before generating an answer. + +```js +const session = await LanguageModel.create({ + initialPrompts: [ + { + role: "system", + content: "You are a concise assistant. Respond in one sentence.", + }, + ], +}); + +const response = await session.prompt("What is photosynthesis?"); +console.log(response); +``` + +See also [Adding context with initial and ongoing prompt inputs > Providing initial prompts during session creation](/en-US/docs/Web/API/Prompt_API/Adding_context#providing_initial_prompts_during_session_creation). + +### Monitoring download progress + +This code shows how you can monitor the download progress of a model. +Note that if the model is unavailable or already available, the event will never fire. + +```js +const session = await LanguageModel.create({ + monitor(monitor) { + monitor.addEventListener("downloadprogress", ({ loaded, total }) => { + console.log(`Model download: ${Math.round((loaded / total) * 100)}%`); + }); + }, +}); +``` + +See also [Using the Prompt API > Monitoring download progress](/en-US/docs/Web/API/Prompt_API/Using#monitoring_download_progress). + +### Providing few-shot prompts + +The following example shows how to use a [few-shot prompt](/en-US/docs/Web/API/Prompt_API/Adding_context#few-shot_prompts) to ask the API for a specific task (French translation) to be delivered in a specific format, before providing some examples to help it learn the correct output format. + +```js +const session = await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en", "fr"] }], + initialPrompts: [ + { + role: "system", + content: + "Translate the user's input to French. Use the output format 'English input: French output'", + }, + { role: "user", content: "Hello" }, + { role: "assistant", content: "Hello: Bonjour" }, + { role: "user", content: "Goodbye" }, + { role: "assistant", content: "Goodbye: Au revoir" }, + { role: "user", content: "The train is late" }, + { + role: "assistant", + content: "The train is late: Le train est en retard", + }, + { role: "user", content: "My shoes are pink" }, + { + role: "assistant", + content: "My shoes are pink: Mes chaussures sont roses", + }, + ], +}); + +const result = await session.prompt("Window"); +console.log(result); // "Window: Fenêtre" +``` + +See also [Adding context with initial and ongoing prompt inputs > Few-shot prompts](/en-US/docs/Web/API/Prompt_API/Adding_context#few-shot_prompts). + +### Defining a tool with a callback + +This example creates a session with a hypothetical "get weather" tool. When the model decides to call the tool, the user agent invokes `execute` with the arguments the model provides. + +```js +async function getWeatherData(location) { + const response = await fetch( + `https://api.example.com/weather?city=${location}`, + ); + const data = await response.json(); + return `${data.temp}°C, ${data.description}`; +} + +const session = await LanguageModel.create({ + tools: [ + { + name: "getWeather", + description: "Returns the current weather for a given city.", + inputSchema: { + type: "object", + properties: { + location: { type: "string", description: "The city name." }, + }, + required: ["location"], + }, + execute: async (...args) => { + const location = args[0]; + return await getWeatherData(location); + }, + }, + ], +}); + +const response = await session.prompt("What's the weather like in Tokyo?"); +console.log(response); +``` + +### Cancelling a session + +The following example enables a user to cancel a prompt. It does this by first creating an {{domxref("AbortController")}} and assigning its `abort()` method to a cancel button's click handler. Next, it calls `create()` and passes `AbortController.signal` as the `signal` property. + +```js +const controller = new AbortController(); + +const cancelButton = document.getElementById("cancel-button"); +cancelButton.addEventListener("click", () => controller.abort()); + +const session = await LanguageModel.create({ + signal: controller.signal, + initialPrompts: [ + { + role: "system", + content: "You are a helpful assistant.", + }, + ], +}); +``` + +See also [Using the Prompt API > Cancelling operations and destroying instances](/en-US/docs/Web/API/Prompt_API/Using#cancelling_operations_and_destroying_instances). + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- {{domxref("LanguageModel.availability_static", "LanguageModel.availability()")}} +- [Prompt API](/en-US/docs/Web/API/Prompt_API) +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) +- [Adding context with initial and ongoing prompt inputs](/en-US/docs/Web/API/Prompt_API/Adding_context) diff --git a/files/en-us/web/api/languagemodel/destroy/index.md b/files/en-us/web/api/languagemodel/destroy/index.md new file mode 100644 index 000000000000000..8c4965221d959f7 --- /dev/null +++ b/files/en-us/web/api/languagemodel/destroy/index.md @@ -0,0 +1,62 @@ +--- +title: "LanguageModel: destroy() method" +short-title: destroy() +slug: Web/API/LanguageModel/destroy +page-type: web-api-instance-method +status: + - experimental +browser-compat: api.LanguageModel.destroy +--- + +{{APIRef("Prompt API")}}{{SeeCompatTable}}{{securecontext_header}} + +The **`destroy()`** method of the {{domxref("LanguageModel")}} interface releases the resources assigned to the `LanguageModel` instance it is called on and stops any further activity on it. Any ongoing and subsequent method calls made on the `LanguageModel` will reject with an `AbortError`. + +It makes sense to destroy `LanguageModel` objects if they are no longer being used, as they tie up significant resources in their handling. + +## Syntax + +```js-nolint +destroy() +``` + +### Parameters + +None. + +### Return value + +None ({{jsxref("undefined")}}). + +### Exceptions + +- `NotAllowedError` {{domxref("DOMException")}} + - : Thrown if usage of the method is blocked by a {{httpheader("Permissions-Policy/language-model", "language-model")}} {{httpheader("Permissions-Policy")}}. + +## Examples + +### Basic `destroy()` usage + +```js +const session = await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], +}); + +// ... + +session.destroy(); +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- [Prompt API](/en-US/docs/Web/API/Prompt_API) +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) diff --git a/files/en-us/web/api/languagemodel/index.md b/files/en-us/web/api/languagemodel/index.md new file mode 100644 index 000000000000000..f43b4094b9f9ba9 --- /dev/null +++ b/files/en-us/web/api/languagemodel/index.md @@ -0,0 +1,95 @@ +--- +title: LanguageModel +slug: Web/API/LanguageModel +page-type: web-api-interface +browser-compat: api.LanguageModel +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`LanguageModel`** interface of the [Prompt API](/en-US/docs/Web/API/Prompt_API) represents a session with a browser-provided language model. It exposes static methods for creating sessions and checking availability, as well as instance methods for prompting the model, appending context, and managing the context window. + +`LanguageModel` instances cannot be constructed directly. Instead, use the static {{domxref("LanguageModel.create_static", "LanguageModel.create()")}} method. + +{{InheritanceDiagram}} + +## Static methods + +- {{domxref("LanguageModel.availability_static", "LanguageModel.availability()")}} + - : Returns a {{jsxref("Promise")}} that resolves with an enumerated value indicating whether the language model is available for the given options. +- {{domxref("LanguageModel.create_static", "LanguageModel.create()")}} + - : Returns a {{jsxref("Promise")}} that resolves with a new `LanguageModel` session, downloading the model data if necessary. + +## Instance methods + +- {{domxref("LanguageModel.append()")}} + - : Returns a {{jsxref("Promise")}} that resolves when the given input has been added to the session's context window, without generating a response. +- {{domxref("LanguageModel.clone()")}} + - : Returns a {{jsxref("Promise")}} that resolves with a new `LanguageModel` session that is a copy of the session it is called on, including all context. +- {{domxref("LanguageModel.destroy()")}} + - : Releases the resources assigned to the `LanguageModel` instance it is called on and stops any further activity on it. +- {{domxref("LanguageModel.measureContextUsage()")}} + - : Returns a {{jsxref("Promise")}} that resolves with the number of context window tokens that the given input would consume if it were used in an operation such as `prompt()` or `append()`. +- {{domxref("LanguageModel.prompt()")}} + - : Returns a {{jsxref("Promise")}} that resolves with the model's complete response to the given input. +- {{domxref("LanguageModel.promptStreaming()")}} + - : Returns a {{domxref("ReadableStream")}} that streams the model's response to the given input as it is generated. + +## Instance properties + +- {{domxref("LanguageModel.contextUsage")}} {{ReadOnlyInline}} + - : Returns the number of context window tokens currently consumed by this session. +- {{domxref("LanguageModel.contextWindow")}} {{ReadOnlyInline}} + - : Returns the total context window size available for this session, in tokens. + +## Events + +- {{domxref("LanguageModel.contextoverflow_event", "contextoverflow")}} + - : Fired when a `prompt()`, `promptStreaming()`, or `append()` call exceeds the context window size. + +## Examples + +### Creating a session and prompting the model + +This example first calls {{domxref("LanguageModel.create_static", "create()")}} to get a new session. It specifies that the language model adopt a `"system"` role and defines how it should behave. Note that the example uses `await` because `create()` returns a {{jsxref("Promise")}}. This may take some time to resolve if the model needs to be downloaded. + +After creating the session, the example calls {{domxref("LanguageModel.prompt()", "prompt()")}} to ask a specific question. + +```js +const session = await LanguageModel.create({ + initialPrompts: [ + { + role: "system", + content: "You are a helpful assistant.", + }, + ], +}); + +const response = await session.prompt("What is the capital of France?"); +console.log(response); // "The capital of France is Paris." +``` + +### Streaming a response + +This example calls {{domxref("LanguageModel.promptStreaming()", "promptStreaming()")}} to get an instance of {{domxref("ReadableStream")}} and writes it to the console in chunks. + +```js +const session = await LanguageModel.create(); +const readableStream = session.promptStreaming("Tell me a short story."); + +for await (const chunk of readableStream) { + console.log(chunk); +} +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- [Prompt API](/en-US/docs/Web/API/Prompt_API) diff --git a/files/en-us/web/api/languagemodel/measurecontextusage/index.md b/files/en-us/web/api/languagemodel/measurecontextusage/index.md new file mode 100644 index 000000000000000..caf5e32a21dcc04 --- /dev/null +++ b/files/en-us/web/api/languagemodel/measurecontextusage/index.md @@ -0,0 +1,133 @@ +--- +title: "LanguageModel: measureContextUsage() method" +short-title: measureContextUsage() +slug: Web/API/LanguageModel/measureContextUsage +page-type: web-api-instance-method +browser-compat: api.LanguageModel.measureContextUsage +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`measureContextUsage()`** method of the {{domxref("LanguageModel")}} interface estimates how many context window tokens the given input would consume without sending it to the model or modifying the session's state. + +This allows you to check how much of the context window a given input requires before deciding whether to send it. The result can be compared against {{domxref("LanguageModel.contextWindow")}} and {{domxref("LanguageModel.contextUsage")}} to determine whether the input can fit into the context window limit. + +This is particularly useful for long-context applications such as document summarization, where you may need to split or truncate content to stay within the context window limit. + +## Syntax + +```js-nolint +measureContextUsage(input) +measureContextUsage(input, options) +``` + +### Parameters + +- `input` + - : The content to append to the context window. This is either: + - A string — Shorthand for a single textual message. + - An array of objects, each representing a single message in a conversation with a language model. + Objects may have the following properties: + - `role` + - : A string indicating the point of view the message is phrased from. Must be one of: + - `"system"` + - : A system-level instruction that guides the model's overall behavior. This must be the first instruction passed to the model. + - `"user"` + - : A message from the user, which the API should respond to. + - `"assistant"` + - : An input that provides context for the AI assistant, such as its persona or the format of its responses. Such messages mainly serve to provide context/history, and further shape how the model responds. + - `content` + - : A string representing a textual prompt, or an array of objects. Each object includes the following properties: + - `type` + - : An enumerated value representing the type of content. This can be one of: + - `audio` + - : Audio content. + - `image` + - : Image content. + - `text` + - : Textual content. + - `"tool-call"` + - : A tool invocation issued by the model. + - `"tool-response"` + - : The result of a tool invocation. + - `value` + - : The content of the message. If the `type` is `text`, this is always a string. If the `type` is `audio` or `image`, the `value` can be one of several different object types; see [What data types are accepted?](/en-US/docs/Web/API/Prompt_API/Multimodal#what_data_types_are_accepted). + - `prefix` {{optional_inline}} + - : A boolean, defaulting to `false`. When `true`, the message is treated as a prefix for the model's next generated response rather than a complete turn. +- `options` {{optional_inline}} + - : Options for measuring context usage. Properties include: + - `responseConstraint` + - : An object following the structure defined by [JSON Schema](https://json-schema.org/) defining the precise format the model's output should be delivered in. When provided and `omitResponseConstraintInput` is `false`, any implementation-defined constraint-description message is included in the measurement. + - `omitResponseConstraintInput` + - : A boolean; when `true`, the automatic constraint-description message is excluded from the measurement. + - `signal` + - : An {{domxref("AbortSignal")}} to cancel the operation. + +### Return value + +A {{jsxref("Promise")}} that resolves with a {{jsxref("Number")}} representing the number of context window tokens the input would consume. + +### Exceptions + +- `AbortError` {{domxref("DOMException")}} + - : Thrown if the operation was cancelled via the `signal` option. +- `NotAllowedError` {{domxref("DOMException")}} + - : Thrown if usage of the method is blocked by a {{httpheader("Permissions-Policy/language-model", "language-model")}} {{httpheader("Permissions-Policy")}}. +- `NotSupportedError` {{domxref("DOMException")}} + - : Thrown if: + - A message's `role` is `"assistant"` and its `type` is anything other than `"text"`. + - A message's `type` is `text` and its `value` is not a string. + - The input or output text is in a language the user agent doesn't support for prompting. + - A message's `type` is `"image"` or `"audio"` but the type was not listed in `expectedInputs`, or the `value` is not an [accepted data type](/en-US/docs/Web/API/Prompt_API/Multimodal#what_data_types_are_accepted). +- `SyntaxError` {{domxref("DOMException")}} + - : Thrown if: + - No messages are included in the messages array. + - A message's `prefix` property is set to `true` and: + - The message's `role` is not `assistant`. + - The message is not the last item in the messages array. +- `TypeError` + - : Thrown if: + - `omitResponseConstraintInput` is `true` but `responseConstraint` is not provided. + - A message's `role` is `system` but it was not the first message passed to the context. + +## Examples + +### Warning when the context is nearly full + +The following example uses a function to verify that context is available before calling {{domxref("LanguageModel.prompt()")}}. It first calculates the remaining context and passes that value to `measureContextUsage()`. If `needed` is less than or equal to `remaining`, it returns `true` and the session continues. + +```js +const promptText = "Let me ask you an interesting question..."; + +async function contextAvailable(promptText) { + const remaining = session.contextWindow - session.contextUsage; + const needed = await session.measureContextUsage(promptText); + + return needed <= remaining; +} + +const session = await LanguageModel.create(); + +if (await contextAvailable(promptText)) { + const response = await session.prompt(promptText); + console.log(response); +} else { + console.warn("Prompt skipped: Not enough context window remaining."); +} +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- {{domxref("LanguageModel.contextUsage")}} +- {{domxref("LanguageModel.contextWindow")}} +- {{domxref("LanguageModel.append()")}} +- [Prompt API](/en-US/docs/Web/API/Prompt_API) +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) diff --git a/files/en-us/web/api/languagemodel/prompt/index.md b/files/en-us/web/api/languagemodel/prompt/index.md new file mode 100644 index 000000000000000..16b6295dee9afc5 --- /dev/null +++ b/files/en-us/web/api/languagemodel/prompt/index.md @@ -0,0 +1,193 @@ +--- +title: "LanguageModel: prompt() method" +short-title: prompt() +slug: Web/API/LanguageModel/prompt +page-type: web-api-instance-method +browser-compat: api.LanguageModel.prompt +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`prompt()`** method of the {{domxref("LanguageModel")}} interface sends input to the language model and returns a {{jsxref("Promise")}} that resolves with the model's complete response as a string. + +## Syntax + +```js-nolint +prompt(input) +prompt(input, options) +``` + +### Parameters + +- `input` + - : The content to prompt the model with. This is either: + - A string — Shorthand for a single textual message. + - An array of objects, each representing a single message in a conversation with a language model. + Objects may have the following properties: + - `role` + - : A string indicating the point of view the message is phrased from. Must be one of: + - `"system"` + - : A system-level instruction that guides the model's overall behavior. This must be the first instruction passed to the model. + - `"user"` + - : A message from the user, which the API should respond to. + - `"assistant"` + - : An input that provides context for the AI assistant, such as its persona or the format of its responses. Such messages mainly serve to provide context/history, and further shape how the model responds. + - `content` + - : A string representing a textual prompt, or an array of objects. Each object includes the following properties: + - `type` + - : An enumerated value representing the type of content. This can be one of: + - `audio` + - : Audio content. + - `image` + - : Image content. + - `text` + - : Textual content. + - `"tool-call"` + - : A tool invocation issued by the model. + - `"tool-response"` + - : The result of a tool invocation. + - `value` + - : The content of the message. If the `type` is `text`, this is always a string. If the `type` is `audio` or `image`, the `value` can be one of several different object types; see [What data types are accepted?](/en-US/docs/Web/API/Prompt_API/Multimodal#what_data_types_are_accepted). + - `prefix` {{optional_inline}} + - : A boolean, defaulting to `false`. When `true`, the message is treated as a prefix for the model's next generated response rather than a complete turn. +- `options` {{optional_inline}} + - : Options for creating a prompt. Properties include: + - `responseConstraint` + - : An object following the structure defined by [JSON Schema](https://json-schema.org/) defining the precise format the model's output should be delivered in. When provided and `omitResponseConstraintInput` is `false`, any implementation-defined constraint-description message is included in the measurement. + - `omitResponseConstraintInput` + - : A boolean; when `true`, the automatic constraint-description message is excluded from the measurement. + - `signal` + - : An {{domxref("AbortSignal")}} to cancel the operation. + +### Return value + +A {{jsxref("Promise")}} that resolves with a {{jsxref("String")}} containing the model's complete response. + +### Exceptions + +- `AbortError` {{domxref("DOMException")}} + - : Thrown if the operation was cancelled via the `signal` option. +- `NotAllowedError` {{domxref("DOMException")}} + - : Thrown if usage of the method is blocked by a {{httpheader("Permissions-Policy/language-model", "language-model")}} {{httpheader("Permissions-Policy")}}. +- `NotSupportedError` {{domxref("DOMException")}} + - : Thrown if: + - A message's `role` is `"assistant"` and its `type` is anything other than `"text"`. + - A message's `type` is `text` and its `value` is not a string. + - The input or output text is in a language the user agent doesn't support for prompting. + - A message's `type` is `"image"` or `"audio"` but the type was not listed in `expectedInputs`, or the `value` is not an [accepted data type](/en-US/docs/Web/API/Prompt_API/Multimodal#what_data_types_are_accepted). +- `OperationError` {{domxref("DOMException")}} + - : Thrown if the prompt fails for any other reason not listed in the other exception types. +- `QuotaExceededError` {{domxref("DOMException")}} + - : Thrown if the prompt would cause the session's context usage to exceed the model's {{domxref("LanguageModel.contextWindow")}}. +- `SyntaxError` {{domxref("DOMException")}} + - : Thrown if: + - No messages are included in the messages array. + - A message's `prefix` property is set to `true` and: + - The message's `role` is not `assistant`. + - The message is not the last item in the messages array. +- `TypeError` + - : Thrown if: + - `omitResponseConstraintInput` is `true` but `responseConstraint` is not provided. + - A message's `role` is `system` but it was not the first message passed to the context. + +## Description + +The `prompt()` method is the primary mechanism for interacting with a language model session. It adds the provided input to the context window and generates a response. The entire response is buffered and returned as a single string when generation completes. + +For long responses or streaming use cases, use {{domxref("LanguageModel.promptStreaming()")}} instead to receive the response incrementally. To add content to the context window without generating a response, use {{domxref("LanguageModel.append()")}}. + +Each call to `prompt()` adds to the session's context. To branch from a given state without affecting the original session, call {{domxref("LanguageModel.clone()")}}. + +## Examples + +### Basic text prompt + +This example shows basic `prompt()` usage with a single user text input. + +```js +const session = await LanguageModel.create(); +const response = await session.prompt( + "Summarize the water cycle in one paragraph.", +); +console.log(response); +``` + +### Multi-turn conversation + +```js +const session = await LanguageModel.create(); + +const reply1 = await session.prompt("My name is Alex."); +console.log(reply1); // "Nice to meet you, Alex!" + +const reply2 = await session.prompt("What's my name?"); +console.log(reply2); // "Your name is Alex." +``` + +### Constrained JSON output + +The following example shows how do pass JSON to the `responseConstraint` option to specify that you want an array returned by the call to `prompt()`. + +```js +const session = await LanguageModel.create(); +const raw = await session.prompt("Name three planets in our solar system.", { + responseConstraint: { + type: "object", + properties: { + planets: { + type: "array", + items: { type: "string" }, + }, + }, + required: ["planets"], + }, +}); + +const { planets } = JSON.parse(raw); +console.log(planets); // ["Mercury", "Venus", "Earth"] +``` + +### Cancelling a prompt + +The following example shows how to enable a user to cancel a prompt with a button. It does this by creating an {{domxref("AbortController")}}. Its `abort()` is callable from a button's `click` handler. For this to work, a reference to the controller's `signal` property must be passed to `prompt()`. + +```js +const controller = new AbortController(); + +// Select your cancel button from the DOM +const cancelButton = document.querySelector("#btn-cancel"); + +// Trigger the abort when the user clicks the button +cancelButton.addEventListener("click", () => { + controller.abort(); +}); + +try { + const response = await session.prompt("write a very long story.", { + signal: controller.signal, + }); + console.log(response); +} catch (err) { + if (err.name === "AbortError") { + console.log("prompt was cancelled."); + } else { + console.error("An unexpected error occurred:", err); + } +} +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- {{domxref("LanguageModel.promptStreaming()")}} +- {{domxref("LanguageModel.append()")}} +- [Prompt API](/en-US/docs/Web/API/Prompt_API) +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) +- [Adding context with initial and ongoing prompt inputs](/en-US/docs/Web/API/Prompt_API/Adding_context) diff --git a/files/en-us/web/api/languagemodel/promptstreaming/index.md b/files/en-us/web/api/languagemodel/promptstreaming/index.md new file mode 100644 index 000000000000000..3c5191b20205fd4 --- /dev/null +++ b/files/en-us/web/api/languagemodel/promptstreaming/index.md @@ -0,0 +1,178 @@ +--- +title: "LanguageModel: promptStreaming() method" +short-title: promptStreaming() +slug: Web/API/LanguageModel/promptStreaming +page-type: web-api-instance-method +browser-compat: api.LanguageModel.promptStreaming +--- + +{{APIRef("Prompt API")}}{{SecureContext_Header}} + +The **`promptStreaming()`** method of the {{domxref("LanguageModel")}} interface sends input to the language model and returns a {{domxref("ReadableStream")}} that delivers the model's response incrementally as it is generated. + +This is useful for displaying responses to users incrementally for outputs that take a long time to complete, or for any scenario where perceived latency should be minimized. Consume the stream using `for await...of` or by attaching a reader via {{domxref("ReadableStream.getReader()")}}. + +## Syntax + +```js-nolint +promptStreaming(input) +promptStreaming(input, options) +``` + +### Parameters + +- `input` + - : The content to prompt the model with. This is either: + - A string — Shorthand for a single textual message. + - An array of objects, each representing a single message in a conversation with a language model. + Objects may have the following properties: + - `role` + - : A string indicating the point of view the message is phrased from. Must be one of: + - `"system"` + - : A system-level instruction that guides the model's overall behavior. This must be the first instruction passed to the model. + - `"user"` + - : A message from the user, which the API should respond to. + - `"assistant"` + - : An input that provides context for the AI assistant, such as its persona or the format of its responses. Such messages mainly serve to provide context/history, and further shape how the model responds. + - `content` + - : A string representing a textual prompt, or an array of objects. Each object includes the following properties: + - `type` + - : An enumerated value representing the type of content. This can be one of: + - `audio` + - : Audio content. + - `image` + - : Image content. + - `text` + - : Textual content. + - `"tool-call"` + - : A tool invocation issued by the model. + - `"tool-response"` + - : The result of a tool invocation. + - `value` + - : The content of the message. If the `type` is `text`, this is always a string. If the `type` is `audio` or `image`, the `value` can be one of several different object types; see [What data types are accepted?](/en-US/docs/Web/API/Prompt_API/Multimodal#what_data_types_are_accepted). + - `prefix` {{optional_inline}} + - : A boolean, defaulting to `false`. When `true`, the message is treated as a prefix for the model's next generated response rather than a complete turn. +- `options` {{optional_inline}} + - : Options for creating a prompt. Properties include: + - `responseConstraint` + - : An object following the structure defined by [JSON Schema](https://json-schema.org/) defining the precise format the model's output should be delivered in. When provided and `omitResponseConstraintInput` is `false`, any implementation-defined constraint-description message is included in the measurement. + - `omitResponseConstraintInput` + - : A boolean; when `true`, the automatic constraint-description message is excluded from the measurement. + - `signal` + - : An {{domxref("AbortSignal")}} to cancel the operation. + +### Return value + +A {{domxref("ReadableStream")}} of {{jsxref("String")}} chunks. Each chunk represents a piece of the model's response as it is generated. The stream closes when generation completes. + +### Exceptions + +Errors are surfaced as stream errors rather than as rejected promises. Consumers should handle errors using a stream's standard error-handling mechanisms. + +- `AbortError` {{domxref("DOMException")}} + - : Thrown if the operation was cancelled via the `signal` option. +- `NotAllowedError` {{domxref("DOMException")}} + - : Thrown if usage of the method is blocked by a {{httpheader("Permissions-Policy/language-model", "language-model")}} {{httpheader("Permissions-Policy")}}. +- `NotSupportedError` {{domxref("DOMException")}} + - : Thrown if: + - A message's `role` is `"assistant"` and its `type` is anything other than `"text"`. + - A message's `type` is `text` and its `value` is not a string. + - The input or output text is in a language the user agent doesn't support for prompting. + - A message's `type` is `"image"` or `"audio"` but the type was not listed in `expectedInputs`, or the `value` is not an [accepted data type](/en-US/docs/Web/API/Prompt_API/Multimodal#what_data_types_are_accepted). +- `OperationError` {{domxref("DOMException")}} + - : Thrown if the prompt fails for any other reason not listed in the other exception types. +- `QuotaExceededError` {{domxref("DOMException")}} + - : Thrown if the prompt would cause the session's context usage to exceed the model's {{domxref("LanguageModel.contextWindow")}}. +- `SyntaxError` {{domxref("DOMException")}} + - : Thrown if: + - No messages are included in the messages array. + - A message's `prefix` property is set to `true` and: + - The message's `role` is not `assistant`. + - The message is not the last item in the messages array. +- `TypeError` + - : Thrown if: + - `omitResponseConstraintInput` is `true` but `responseConstraint` is not provided. + - A message's `role` is `system` but it was not the first message passed to the context. + +## Description + +The `promptStreaming()` method adds the provided input to the context window and generates a response. The entire response is receives incrementally as a {{domxref("ReadableStream")}}. + +To receive the response as one complete string, use {{domxref("LanguageModel.prompt()")}} instead. To add content to the context window without generating a response, use {{domxref("LanguageModel.append()")}}. + +Each call to `promptStreaming()` adds to the session's context. To branch from a given state without affecting the original session, call {{domxref("LanguageModel.clone()")}}. + +## Examples + +### Streaming a response to the page + +This example writes out chunks from a `promptStreaming()` call's {{domxref("ReadableStream")}} as they arrive. + +```js +const session = await LanguageModel.create(); +const output = document.querySelector("#output"); + +const stream = session.promptStreaming("Write a short poem about the ocean."); + +for await (const chunk of stream) { + output.textContent += chunk; +} +``` + +### Streaming with an abort signal + +This example shows how to use an {{domxref("AbortController")}} with `promptStreaming()`. + +```js +const controller = new AbortController(); +document + .querySelector("#stop") + .addEventListener("click", () => controller.abort()); + +const stream = session.promptStreaming("Tell me a long story.", { + signal: controller.signal, +}); + +try { + for await (const chunk of stream) { + output.textContent += chunk; + } +} catch (err) { + if (err.name === "AbortError") { + console.log("Streaming was stopped by the user."); + } +} +``` + +### Collecting streamed chunks into a single string + +In this example, chunks from a {{domxref("ReadableStream")}} are collected before the whole stream is written out. + +```js +const session = await LanguageModel.create(); +const stream = session.promptStreaming("Explain quantum entanglement."); +const chunks = []; + +for await (const chunk of stream) { + chunks.push(chunk); +} + +const fullResponse = chunks.join(""); +console.log(fullResponse); +``` + +## Specifications + +{{Specifications}} + +## Browser compatibility + +{{Compat}} + +## See also + +- {{domxref("LanguageModel.prompt()")}} +- {{domxref("ReadableStream")}} +- [Prompt API](/en-US/docs/Web/API/Prompt_API) +- [Using the Prompt API](/en-US/docs/Web/API/Prompt_API/Using) +- [Adding context with initial and ongoing prompt inputs](/en-US/docs/Web/API/Prompt_API/Adding_context) diff --git a/files/en-us/web/api/prompt_api/adding_context/index.md b/files/en-us/web/api/prompt_api/adding_context/index.md new file mode 100644 index 000000000000000..48ef8077b5dffff --- /dev/null +++ b/files/en-us/web/api/prompt_api/adding_context/index.md @@ -0,0 +1,853 @@ +--- +title: Adding context with initial and ongoing prompt inputs +short-title: Adding context +slug: Web/API/Prompt_API/Adding_context +page-type: guide +--- + +{{DefaultAPISidebar("Prompt API")}} + +In our basic [Prompt API guide](/en-US/docs/Web/API/Prompt_API/Using), we covered everything you need to get up and running with the [Prompt API](/en-US/docs/Web/API/Prompt_API). However, this only really covers creating a generic AI prompt app. To give your app different personalities, get it to respond in different ways, and remember past conversations, you need to provide additional context. The Prompt API has a few different mechanisms to allow you to do this, which are covered in this article. + +## Prompt input syntax + +When {{domxref("LanguageModel.prompt()")}} is called, it takes an `input` parameter containing the inputs to respond to: + +```js +const response = await session.prompt(inputElem.value); +``` + +The previous `prompt()` call only receives a single string as a parameter. This is a shorthand form, available for the common situation where you only want to pass the model a single user text prompt. You can expand this to explicitly declare the `role` of the `input` object: + +```js +const response = await session.prompt([ + { + role: "user", + content: inputElem.value, + }, +]); +``` + +The three available `role` types are: + +- `user` + - : Inputs that come from the `user`, which the API should respond to. +- `assistant` + - : Inputs that are written from the point of view of the AI assistant, which mainly serve to provide context/history, and further shape how the model responds. These are commonly used for [preserving sessions](/en-US/docs/Web/API/Prompt_API/Preserving_sessions) and [few-shot prompts](#few-shot_prompts). +- `system` + - : Global inputs from the overall system that give the model instructions on how to respond. If a `system` input is included, it must come first in the provided inputs, otherwise an exception is thrown. As a result, `system` inputs are usually only included as [initial prompts](#providing_initial_prompts_during_session_creation). + +### Multiple inputs + +You can provide multiple inputs in the array, for example: + +```js +const response = await session.prompt([ + { + role: "user", + content: "The following is my favorite color. Do you like it?", + }, + { + role: "user", + content: inputElem.value, + }, +]); +``` + +This is useful because you can provide extra context to help the model build a response along with the actual input taken from the page, which might only be one word. + +### Specifying input type + +By default, the `input` type is `text`. To explicitly declare the `type`, you can further expand the previous form to the full longhand equivalent, which looks like this: + +```js +const response = await session.prompt([ + { + role: "user", + content: [ + { + type: "text", + value: inputElem.value, + }, + ], + }, +]); +``` + +You don't need this form unless you are providing the assistant with `image` and/or `audio` inputs (see [multimodal prompts](/en-US/docs/Web/API/Prompt_API/Multimodal)): + +```js +const response = await session.prompt([ + { + role: "user", + content: [ + { type: "text", value: "Describe my image and audio:" }, + { type: "image", value: imgElem }, + { type: "audio", value: audioBuffer }, + ], + }, +]); +``` + +However, you could rewrite the previous multiple-user-input example into this form, which includes both messages in a single input object. You might find this version easier to understand: + +```js +const response = await session.prompt([ + { + role: "user", + content: [ + { + type: "text", + value: "The following is my favorite color. Do you like it?", + }, + { type: "text", value: inputElem.value }, + ], + }, +]); +``` + +## Providing initial prompts during session creation + +The {{domxref("LanguageModel.create_static", "create()")}} method can take an [`initialPrompts`](/en-US/docs/Web/API/LanguageModel/create_static#initialprompts) option that contains an array of input prompts, just like the `inputs` array passed to `prompt()` and other methods. This allows you to pass an initial set of prompts into the session as it is created, so the model immediately has some context to work with. + +For example: + +```js +const session = await LanguageModel.create({ + initialPrompts: [ + { + role: "system", + content: "Respond like a pirate.", + }, + { + role: "assistant", + content: "Avast ye, pirate! I am Redbeard!", + }, + { + role: "user", + content: + "Yarrrr, matey! Well met. My name is Silas Blacktooth, the scourge of Blackpool!", + }, + ], +}); +``` + +As well as telling the model what kind of personality it should have, `initialPrompts` is also useful for loading a previous saved conversation into the session after a page reload or subsequent visit to the app. See [preserving sessions across reloads](/en-US/docs/Web/API/Prompt_API/Preserving_sessions). + +> [!NOTE] +> The text string shorthand form discussed at the top of [Prompt input syntax](#prompt_input_syntax) cannot be used in the `initialPrompts` option of a `create()` call. + +## Few-shot prompts + +A few-shot prompt is a set of `user` role and `assistant` role input pairs passed as an example to the API to train it how to respond to a particular type of input, before asking it to complete a similar task. + +The following example shows how to use a few-shot prompt to request a French translation in a specific format, providing sample inputs and outputs to demonstrate the expected structure. + +```js +const session = await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en", "fr"] }], + initialPrompts: [ + { + role: "system", + content: + "Translate the user's input to French. Use the output format 'English input: French output'", + }, + { role: "user", content: "Hello" }, + { role: "assistant", content: "Hello: Bonjour" }, + { role: "user", content: "Goodbye" }, + { role: "assistant", content: "Goodbye: Au revoir" }, + { role: "user", content: "The train is late" }, + { + role: "assistant", + content: "The train is late: Le train est en retard", + }, + { role: "user", content: "My shoes are pink" }, + { + role: "assistant", + content: "My shoes are pink: Mes chaussures sont roses", + }, + ], +}); + +const result = await session.prompt("Window"); +console.log(result); // "Window: Fenêtre" +``` + +You could include just the `system` initial prompt and the example would still work, but it would be less likely to provide responses in the desired format. + +## Initial and multiple inputs example + +Let's look at an example that makes use of initial and multiple inputs for extra context. In this example, the user is prompted to enter their name, and the API provides a whimsical review of it. + +Technically, this is very similar to the [complete example](/en-US/docs/Web/API/Prompt_API/Using#complete_example) from the previous guide, the only real differences being that user input is provided via a single line text {{htmlelement("input")}} rather than a {{htmlelement("textarea")}}, and the `create()` and `prompt()` calls are different. As a result, we won't walk through the entire codebase again. To check out the codebase in more detail, see the previous article's descriptions, and press the "Play" button in the rendered live output to open the full code in MDN Playground. + +```html hidden live-sample___rate-my-name +

Prompt API rate my name!

+

+ Enter your name (or someone else's name) into the input field and press the + rate button to have AI review your name. First released in Chrome 148. +

+ +

Input

+ +
+
+ + +
+ +
+ +

Output

+ +

+``` + +```css hidden live-sample___rate-my-name live-sample___excerpt-question live-sample___constraint-example +* { + box-sizing: border-box; +} + +html { + font-family: Arial, Helvetica, sans-serif; +} + +body { + max-width: 600px; + margin: 0 auto; +} + +form div { + display: flex; + flex-direction: column; + gap: 10px; + margin-bottom: 20px; +} + +input, +textarea, +.prompt-output { + padding: 5px; +} + +.prompt-output { + min-height: 150px; + border: 1px solid black; + width: 100%; + display: block; +} + +.error { + color: red; +} + +button { + margin-right: 10px; +} +``` + +```js hidden live-sample___rate-my-name +const form = document.querySelector("form"); +const inputElem = document.querySelector("input"); +const submitBtn = document.querySelector("#submit"); +const abortBtn = document.querySelector("#abort"); +abortBtn.disabled = true; +submitBtn.disabled = true; +const promptOutput = document.querySelector(".prompt-output"); + +let session; +inputElem.addEventListener("focus", () => { + if (!("LanguageModel" in window)) { + promptOutput.innerHTML = `Your browser doesn't support the Prompt API!`; + return; + } + + if (!session) { + init(); + } +}); + +async function init() { + session = await getSession(); + promptOutput.textContent = `Session created.`; + submitBtn.disabled = false; +} + +form.addEventListener("submit", handleSubmission); + +async function handleSubmission(e) { + e.preventDefault(); + + if (inputElem.value === "") { + promptOutput.innerHTML = `No text entered!`; + return; + } + + try { + promptOutput.textContent = "...generating response..."; + submitBtn.disabled = true; + abortBtn.disabled = false; + + const controller = new AbortController(); + abortBtn.addEventListener("click", () => { + controller.abort("Query aborted by user."); + submitBtn.disabled = false; + abortBtn.disabled = true; + }); + + const response = await session.prompt( + [ + { + role: "user", + content: "What do you think of my name?", + }, + { + role: "user", + content: inputElem.value, + }, + ], + { + signal: controller.signal, + }, + ); + + promptOutput.textContent = response; + + submitBtn.disabled = false; + abortBtn.disabled = true; + console.log(`${session.contextUsage}/${session.contextWindow}`); + } catch (e) { + promptOutput.innerHTML = `${e}`; + } +} + +async function getSession() { + const availability = await LanguageModel.availability({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + }); + if (availability === "unavailable") { + promptOutput.textContent = "Language model not available."; + return undefined; + } else if (availability === "available") { + return await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + initialPrompts: [ + { + role: "system", + content: + "In each case, respond with a short paragraph that pokes fun at the person's name in a sarcastic manner. Include a rating out of 10 at the end of the paragraph. The response should be cheeky, but not rude or offensive.", + }, + ], + }); + } else { + return await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + initialPrompts: [ + { + role: "system", + content: + "In each case, respond with a short paragraph that pokes fun at the person's name in a sarcastic manner. Include a rating out of 10 at the end of the paragraph. The response should be cheeky, but not rude or offensive.", + }, + ], + monitor(monitor) { + monitor.addEventListener("downloadprogress", (e) => { + promptOutput.textContent = `Downloading model data ${Math.floor(e.loaded * 100)}%`; + }); + }, + }); + } +} +``` + +### JavaScript + +When the {{domxref("LanguageModel.create_static", "create()")}} method is called to create the session `LanguageModel` instance, we pass in an `initialPrompts` option that includes a `system` input to tell the model exactly how we want it to respond to each user prompt: + +```js +return await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + initialPrompts: [ + { + role: "system", + content: + "In each case, respond with a short paragraph that pokes fun at the person's name in a sarcastic manner. Include a rating out of 10 at the end of the paragraph. The response should be cheeky, but not rude or offensive.", + }, + ], +}); +``` + +When we call {{domxref("LanguageModel.prompt", "prompt()")}} on our `session` object, we pass two `user` input objects to it. The first one makes it clear what the user is asking of the API, and the second provides the user's name entered into the `` element for the API to review. + +```js +const response = await session.prompt( + [ + { + role: "user", + content: "What do you think of my name?", + }, + { + role: "user", + content: inputElem.value, + }, + ], + { + signal: controller.signal, + }, +); +``` + +### Result + +{{EmbedLiveSample("rate-my-name", , "600px", , , , "language-model", "allow-forms")}} + +Try entering a name into the ``, then press the submit button to prompt the AI model for a whimsical review of the name. + +## Adding response constraints + +The `prompt()` and {{domxref("LanguageModel.promptStreaming", "promptStreaming()")}} methods both accept a [`responseConstraint`](/en-US/docs/Web/API/LanguageModel/prompt#responseconstraint) option that takes as its value a [JSON Schema](https://json-schema.org/) object that defines the exact format expected for the assistant's responses. This delivers more controlled results than just asking the API to respond in a particular way via a `system` prompt. + +A very simple schema might define a response that should contain a single boolean value: + +```js +const schema = { + type: "boolean", +}; +``` + +To use this, you set the schema as the value of the `responseConstraint` option: + +```js +const response = await session.prompt( + [ + { + role: "user", + content: `Is this a color: ${inputElem.value}?`, + }, + ], + { + responseConstraint: schema, + }, +); +``` + +In this case, we set the prompt content to "Is this a color:" followed by an `` element `value`. As a result, the API will evaluate whether the user's input is a color or not, and return a value of `true` or `false`. + +Let's look at a more complex example, to give you more of an idea of what is possible with response constraints. In this case, the schema specifies that the API response should be delivered as JSON containing: + +- A single string representing a summary description. +- An array of exactly three strings representing three supporting bullet points. + +```js +const schema = { + $schema: "https://json-schema.org/draft/2020-12/schema", + title: "Description with Three Bullets", + type: "object", + properties: { + description: { + type: "string", + description: "A descriptive sentence summarizing the content.", + minLength: 1, + }, + bullets: { + type: "array", + description: "Exactly three supporting bullet points.", + items: { + type: "string", + minLength: 1, + }, + minItems: 3, + maxItems: 3, + }, + }, + required: ["description", "bullets"], + additionalProperties: false, +}; +``` + +This is included in the `prompt()` call's `responseConstraint` option, as before: + +```js +const response = await session.prompt(textarea.value, { + responseConstraint: schema, +}); +``` + +Because the response is specified as a JSON string, we can parse the response into an object, and then use the object's properties in our response: + +```js +const structuredOutput = JSON.parse(response); + +promptOutput.innerHTML = `${structuredOutput.description}

- ${structuredOutput.bullets[0]}
- ${structuredOutput.bullets[1]}
- ${structuredOutput.bullets[2]}`; +``` + +You can try this demo out in the following live example: + +```html hidden live-sample___constraint-example +

Prompt API constraint demo

+

+ Type in a subject. The demo uses a JSON schema to constrain the API response + to a JSON string containing a summary string and an array containing three + supporting strings. Released in Chrome 148, but trialed since version 137. +

+ +

Input

+ +
+
+ + +
+ +
+ +

Output

+ +

+``` + +```js hidden live-sample___constraint-example +const form = document.querySelector("form"); +const textarea = document.querySelector("textarea"); +const submitBtn = document.querySelector("#submit"); +const abortBtn = document.querySelector("#abort"); +abortBtn.disabled = true; +submitBtn.disabled = true; +const promptOutput = document.querySelector(".prompt-output"); + +let session; +textarea.addEventListener("focus", () => { + if (!session) { + init(); + } +}); + +async function init() { + session = await getSession(); + promptOutput.textContent = `Session created.`; + submitBtn.disabled = false; +} + +form.addEventListener("submit", handleSubmission); + +async function handleSubmission(e) { + e.preventDefault(); + + if (textarea.value === "") { + promptOutput.innerHTML = `No text entered!`; + return; + } + + try { + promptOutput.textContent = "...generating response..."; + submitBtn.disabled = true; + abortBtn.disabled = false; + + const controller = new AbortController(); + abortBtn.addEventListener("click", () => { + controller.abort("Query aborted by user."); + submitBtn.disabled = false; + abortBtn.disabled = true; + }); + + const schema = { + $schema: "https://json-schema.org/draft/2020-12/schema", + title: "Description with Three Bullets", + type: "object", + properties: { + description: { + type: "string", + description: "A descriptive sentence summarizing the content.", + minLength: 1, + }, + bullets: { + type: "array", + description: "Exactly three supporting bullet points.", + items: { + type: "string", + minLength: 1, + }, + minItems: 3, + maxItems: 3, + }, + }, + required: ["description", "bullets"], + additionalProperties: false, + }; + + const response = await session.prompt(textarea.value, { + signal: controller.signal, + responseConstraint: schema, + }); + + const structuredOutput = JSON.parse(response); + + promptOutput.innerHTML = `${structuredOutput.description}

- ${structuredOutput.bullets[0]}
- ${structuredOutput.bullets[1]}
- ${structuredOutput.bullets[2]}`; + + submitBtn.disabled = false; + abortBtn.disabled = true; + console.log(`${session.contextUsage}/${session.contextWindow}`); + } catch (e) { + promptOutput.innerHTML = `${e}`; + } +} + +async function getSession() { + const availability = await LanguageModel.availability({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + }); + if (availability === "unavailable") { + promptOutput.textContent = "Language model not available."; + return undefined; + } else if (availability === "available") { + return await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + }); + } else { + return await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + monitor(monitor) { + monitor.addEventListener("downloadprogress", (e) => { + promptOutput.textContent = `Downloading model data ${Math.floor(e.loaded * 100)}%`; + }); + }, + }); + } +} +``` + +{{EmbedLiveSample("constraint-example", , "660px", , , , "language-model", "allow-forms")}} + +## Appending extra messages to the context + +Inferring a response to a user question or statement can take a long time, especially when the API has to deal with large, complex text inputs, or multimodal inputs. + +To reduce the perceived latency between the user's prompt and the response, it can be a good idea to start the API processing the request as soon as possible — providing useful context before the user submits their actual input — or adding further context afterwards. + +The {{domxref("LanguageModel.append()")}} method exists to provide such context — it adds further inputs for the API to process, without generating a model response. + +For example, in the following snippet we provide an excerpt from a fairly famous book. We use `append()` to feed the excerpt into the API session, and then ask a question about it using a `prompt()` call. The browser can get a headstart on processing the excerpt while it waits for the question to be asked. + +```js +const excerpt = + "The face of Elrond was ageless, neither old nor young, though in it was written the memory of many things both glad and sorrowful. His hair was dark as the shadows of twilight, and upon it was set a circlet of silver; his eyes were grey as a clear evening, and in them was a light like the light of stars. Venerable he seemed as a king crowned with many winters, and yet hale as a tried warrior in the fullness of his strength. He was Lord of Rivendell and mighty among both Elves and Men."; + +await session.append(excerpt); + +// ... + +const response = await session.prompt([ + { + role: "user", + content: "What book was the last entered text taken from?", + }, +]); +``` + +### An append example + +Let's look at a real implementation of the excerpt example mentioned previously. In this case, you can enter a passage of text into one input and a question about that text into another input. When submitted, the API response will answer the question specifically in the context of the provided text passage. + +It works similarly to previous examples, so we won't walk through all the code exhaustively. To study the full code, press the "Play" button in the rendered live output to open the full code in MDN Playground. + +```html hidden live-sample___excerpt-question +

Prompt API excerpt question demo

+

+ Enter a passage of text (such as a book excerpt) into the textarea, then enter + a question about the text into the single-line input. Press the submit button + to ask your question to the API. First released in Chrome 148. +

+ +

Input

+ +
+
+ + +
+
+ + +
+ +
+ +

Output

+ +

+``` + +```js hidden live-sample___excerpt-question +const form = document.querySelector("form"); +const textareaElem = document.querySelector("textarea"); +const inputElem = document.querySelector("input"); +const submitBtn = document.querySelector("#submit"); +const abortBtn = document.querySelector("#abort"); +abortBtn.disabled = true; +submitBtn.disabled = true; +const promptOutput = document.querySelector(".prompt-output"); + +let session; +textareaElem.addEventListener("focus", () => { + if (!("LanguageModel" in window)) { + promptOutput.innerHTML = `Your browser doesn't support the Prompt API!`; + return; + } + + if (!session) { + init(); + } +}); + +async function init() { + session = await getSession(); + promptOutput.textContent = `Session created.`; +} + +textareaElem.addEventListener("change", appendExcerpt); +form.addEventListener("submit", handleSubmission); + +async function appendExcerpt() { + if (textareaElem.value === "") { + promptOutput.innerHTML = `No passage entered!`; + return; + } + session.append(textareaElem.value); + submitBtn.disabled = false; +} + +async function handleSubmission(e) { + e.preventDefault(); + + if (inputElem.value === "") { + promptOutput.innerHTML = `No question entered!`; + return; + } + + try { + promptOutput.textContent = "...generating response..."; + submitBtn.disabled = true; + abortBtn.disabled = false; + + const controller = new AbortController(); + abortBtn.addEventListener("click", () => { + controller.abort("Query aborted by user."); + submitBtn.disabled = false; + abortBtn.disabled = true; + }); + + const response = await session.prompt( + [ + { + role: "user", + content: "I have a question for you about the provided text.", + }, + { + role: "user", + content: inputElem.value, + }, + ], + { + signal: controller.signal, + }, + ); + + promptOutput.textContent = response; + + submitBtn.disabled = false; + abortBtn.disabled = true; + console.log(`${session.contextUsage}/${session.contextWindow}`); + } catch (e) { + promptOutput.innerHTML = `${e}`; + } +} + +async function getSession() { + const availability = await LanguageModel.availability({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + }); + if (availability === "unavailable") { + promptOutput.textContent = "Language model not available."; + return undefined; + } else if (availability === "available") { + return await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + }); + } else { + return await LanguageModel.create({ + expectedInputs: [{ type: "text", languages: ["en"] }], + expectedOutputs: [{ type: "text", languages: ["en"] }], + monitor(monitor) { + monitor.addEventListener("downloadprogress", (e) => { + promptOutput.textContent = `Downloading model data ${Math.floor(e.loaded * 100)}%`; + }); + }, + }); + } +} +``` + +#### JavaScript + +In this example, the excerpt is entered into a ` + + + + + +

Output

+ +

+``` + +For brevity, we won't show the CSS; there is nothing significant to discuss, style-wise. + +## Retrieving the prompt history + +When the page first loads, we need to check whether we have any prompt history saved, and if so, load it into the session. + +We start by defining a variable called `promptHistory` to store the saved history: + +```js +let promptHistory; +``` + +We then check whether there is a property in {{domxref("Window.localStorage", "localStorage")}} called `promptHistory`, which is the key we will store our prompt history under. If there is, we retrieve that storage item using {{domxref("Storage.getItem", "getItem()")}}, parse it into an array using {{jsxref("JSON.parse()")}}, and store it in the variable. We also enable the delete ` + +``` + +```html live-sample___prompt-streaming-example +

Prompt API streaming demo

+

First released in Chrome 148.

+ +

Input

+
+
+ + +
+ +
+``` + +Next, we include a {{htmlelement("p")}} element to display the model's response to the user's prompt, plus details of any errors that are thrown. + +```html live-sample___prompt-example live-sample___prompt-streaming-example +

Output

+

+``` + +```css hidden live-sample___prompt-example live-sample___prompt-streaming-example +* { + box-sizing: border-box; +} + +html { + font-family: Arial, Helvetica, sans-serif; +} + +body { + max-width: 600px; + margin: 0 auto; +} + +form div { + display: flex; + flex-direction: column; + gap: 10px; + margin-bottom: 20px; +} + +textarea, +.prompt-output { + padding: 5px; +} + +.prompt-output { + min-height: 150px; + border: 1px solid black; + width: 100%; + display: block; +} + +.error { + color: red; +} + +button { + margin-right: 10px; +} +``` + +Note that we won't show the CSS for this example, as none of it is relevant to understanding the Prompt API. + +### JavaScript + +In our script, we start off by grabbing references to the `
`, `