Ways of telling by how much a too-large input is too large

[domenic]

It's possible an input fed to these APIs is too large. This will be signaled by a rejected promise, probably a "QuotaExceededError" DOMException.

However, @uskay points out that this does not allow you to give informative error messages, telling the user or web developer by how much the input is too large.

There are two possible APIs one could imagine here:

Measure, then summarize

const summarizer = await ai.summarizer.create();
const summarizerCapabilities = await ai.summarizer.capabilities();

const tokenCount = await summarizer.countTokens(input);
if (tokenCount > summarizerCapabilities.maxTokens) {
  console.error(`Too large! You tried to summarize ${tokenCount} tokens, but only up to ${summarizerCapabilities.maxTokens} is possible!`);
} else {
  console.log(await summarizer.summarize(input));
}

This API is probably bad because it requires two round-trips to the language model, one to tokenize, and then a second one to tokenize-plus-summarize.

More informative errors

This would probably look something like:

const summarizer = await ai.summarizer.create();
const summarizerCapabilities = await ai.summarizer.capabilities();

try {
  console.log(await summarizer.summarize(input));
} catch (e) {
  if (e.name === "TooManyTokensError") {
    console.error(`Too large! You tried to summarize ${e.tokenCount} tokens, but only up to ${summarizerCapabilities.maxTokens} is possible!`);
  } else {
    throw e;
  }
}

This is probably better since it only has one round trip.

[domenic]

In an separate thread @andreban pointed out that you probably need both. Above I pointed out situations where "more informative errors" is better, but his scenario is

A developer may want to provide feedback to the user how close they are to reaching the limit before the user submits the content to be summarized, so you'd need to know max tokens and be able to count tokens, without actually summarizing

which cannot really be handled without the "measure, then summarize" approach. We'll just have to be sure to put appropriate warnings on the countTokens() API.

[domenic]

I've put up an initial draft for this in #31.

The prompt API will change to align with this.