Skip to content

Commit 88edc28

Browse files
authored
feat (provider/openai): include more image generation response metadata (#10698)
## Background The OpenAI Images API returns useful metadata about each request in the response. Callers may want to use this data for purposes like cost computation. Reference: https://platform.openai.com/docs/pricing#image-generation To compute the cost for an image generation a caller has to know: - model - image quality - image dimensions ## Summary Parse additional detail from OpenAI API response and include in provider response metadata. ## Manual Verification Ran example scripts for openai. ## Checklist - [x] Tests have been added / updated (for bug fixes / features) - [x] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review)
1 parent 87f2d5a commit 88edc28

File tree

7 files changed

+69
-22
lines changed

7 files changed

+69
-22
lines changed

.changeset/heavy-suits-do.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@ai-sdk/openai': patch
3+
---
4+
5+
feat (provider/openai): include more image generation response metadata

content/providers/01-ai-sdk-providers/03-openai.mdx

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1479,7 +1479,20 @@ const { image, providerMetadata } = await generateImage({
14791479

14801480
For more on `generateImage()` see [Image Generation](/docs/ai-sdk-core/image-generation).
14811481

1482-
OpenAI's image models may return a revised prompt for each image. It can be access at `providerMetadata.openai.images[0]?.revisedPrompt`.
1482+
OpenAI's image models return additional metadata in the response that can be
1483+
accessed via `providerMetadata.openai`. The following OpenAI-specific metadata
1484+
is available:
1485+
1486+
- **images** _Array<object>_
1487+
1488+
Array of image-specific metadata. Each image object may contain:
1489+
1490+
- `revisedPrompt` _string_ - The revised prompt that was actually used to generate the image (OpenAI may modify your prompt for safety or clarity)
1491+
- `created` _number_ - The Unix timestamp (in seconds) of when the image was created
1492+
- `size` _string_ - The size of the generated image. One of `1024x1024`, `1024x1536`, or `1536x1024`
1493+
- `quality` _string_ - The quality of the generated image. One of `low`, `medium`, or `high`
1494+
- `background` _string_ - The background parameter used for the image generation. Either `transparent` or `opaque`
1495+
- `outputFormat` _string_ - The output format of the generated image. One of `png`, `webp`, or `jpeg`
14831496

14841497
For more information on the available OpenAI image model options, see the [OpenAI API reference](https://platform.openai.com/docs/api-reference/images/create).
14851498

examples/ai-core/src/generate-image/google-vertex.ts

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ import 'dotenv/config';
77
import { presentImages } from '../lib/present-image';
88

99
async function main() {
10-
const { image } = await generateImage({
10+
const result = await generateImage({
1111
model: vertex.image('imagen-3.0-generate-002'),
1212
prompt: 'A burrito launched through a tunnel',
1313
aspectRatio: '1:1',
@@ -18,7 +18,12 @@ async function main() {
1818
},
1919
});
2020

21-
await presentImages([image]);
21+
await presentImages(result.images);
22+
23+
console.log(
24+
'Provider metadata:',
25+
JSON.stringify(result.providerMetadata, null, 2),
26+
);
2227
}
2328

2429
main().catch(console.error);

examples/ai-core/src/generate-image/openai.ts

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,19 @@ import { presentImages } from '../lib/present-image';
44
import 'dotenv/config';
55

66
async function main() {
7-
const prompt = 'Santa Claus driving a Cadillac';
7+
const prompt = 'A blue cream Persian cat in Kyoto in the style of ukiyo-e';
88
const result = await generateImage({
99
model: openai.image('gpt-image-1-mini'),
1010
prompt,
11+
n: 3,
1112
});
1213

13-
// @ts-expect-error
14-
const revisedPrompt = result.providerMetadata.openai.images[0]?.revisedPrompt;
14+
await presentImages(result.images);
1515

16-
console.log({
17-
prompt,
18-
revisedPrompt,
19-
usage: result.usage,
20-
});
21-
22-
await presentImages([result.image]);
16+
console.log(
17+
'Provider metadata:',
18+
JSON.stringify(result.providerMetadata, null, 2),
19+
);
2320
}
2421

2522
main().catch(console.error);

packages/openai/src/image/openai-image-api.ts

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,17 @@ import { z } from 'zod/v4';
66
export const openaiImageResponseSchema = lazySchema(() =>
77
zodSchema(
88
z.object({
9+
created: z.number().nullish(),
910
data: z.array(
1011
z.object({
1112
b64_json: z.string(),
1213
revised_prompt: z.string().nullish(),
1314
}),
1415
),
16+
background: z.string().nullish(),
17+
output_format: z.string().nullish(),
18+
size: z.string().nullish(),
19+
quality: z.string().nullish(),
1520
usage: z
1621
.object({
1722
input_tokens: z.number().nullish(),

packages/openai/src/image/openai-image-model.test.ts

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -266,7 +266,15 @@ describe('doGenerate', () => {
266266
expect(result.warnings).toStrictEqual([]);
267267
expect(result.providerMetadata).toStrictEqual({
268268
openai: {
269-
images: [null],
269+
images: [
270+
{
271+
created: 1733837122,
272+
size: undefined,
273+
quality: undefined,
274+
background: undefined,
275+
outputFormat: undefined,
276+
},
277+
],
270278
},
271279
});
272280
});
@@ -306,8 +314,19 @@ describe('doGenerate', () => {
306314
{
307315
revisedPrompt:
308316
'A charming visual illustration of a baby sea otter swimming joyously.',
317+
created: 1733837122,
318+
size: undefined,
319+
quality: undefined,
320+
background: undefined,
321+
outputFormat: undefined,
322+
},
323+
{
324+
created: 1733837122,
325+
size: undefined,
326+
quality: undefined,
327+
background: undefined,
328+
outputFormat: undefined,
309329
},
310-
null,
311330
],
312331
},
313332
});

packages/openai/src/image/openai-image-model.ts

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -105,13 +105,16 @@ export class OpenAIImageModel implements ImageModelV3 {
105105
},
106106
providerMetadata: {
107107
openai: {
108-
images: response.data.map(item =>
109-
item.revised_prompt
110-
? {
111-
revisedPrompt: item.revised_prompt,
112-
}
113-
: null,
114-
),
108+
images: response.data.map(item => ({
109+
...(item.revised_prompt
110+
? { revisedPrompt: item.revised_prompt }
111+
: {}),
112+
created: response.created ?? undefined,
113+
size: response.size ?? undefined,
114+
quality: response.quality ?? undefined,
115+
background: response.background ?? undefined,
116+
outputFormat: response.output_format ?? undefined,
117+
})),
115118
},
116119
},
117120
};

0 commit comments

Comments
 (0)