|
| 1 | +--- |
| 2 | +title: AI SDK |
| 3 | +--- |
| 4 | + |
| 5 | +Evalite integrates deeply with the Vercel AI SDK to provide automatic tracing and caching of all LLM calls. |
| 6 | + |
| 7 | +## `wrapAISDKModel()` |
| 8 | + |
| 9 | +Wraps a Vercel AI SDK model to enable automatic tracing and caching of all LLM calls. |
| 10 | + |
| 11 | +```typescript |
| 12 | +import { openai } from "@ai-sdk/openai"; |
| 13 | +import { generateText } from "ai"; |
| 14 | +import { evalite } from "evalite"; |
| 15 | +import { wrapAISDKModel } from "evalite/ai-sdk"; |
| 16 | + |
| 17 | +// Wrap the model |
| 18 | +const model = wrapAISDKModel(openai("gpt-4o-mini")); |
| 19 | + |
| 20 | +evalite("My Eval", { |
| 21 | + data: [{ input: "Hello", expected: "Hi" }], |
| 22 | + task: async (input) => { |
| 23 | + // All calls are automatically traced and cached |
| 24 | + const result = await generateText({ |
| 25 | + model, |
| 26 | + prompt: input, |
| 27 | + }); |
| 28 | + |
| 29 | + return result.text; |
| 30 | + }, |
| 31 | +}); |
| 32 | +``` |
| 33 | + |
| 34 | +### Signature |
| 35 | + |
| 36 | +```typescript |
| 37 | +wrapAISDKModel( |
| 38 | + model: LanguageModelV2, |
| 39 | + options?: { |
| 40 | + tracing?: boolean; |
| 41 | + caching?: boolean; |
| 42 | + } |
| 43 | +): LanguageModelV2 |
| 44 | +``` |
| 45 | + |
| 46 | +**Parameters:** |
| 47 | + |
| 48 | +- `model` - A Vercel AI SDK language model (from `@ai-sdk/openai`, `@ai-sdk/anthropic`, etc.) |
| 49 | +- `options` (optional) - Configuration options: |
| 50 | + - `tracing` - Enable automatic trace capture (default: `true`) |
| 51 | + - `caching` - Enable response caching (default: `true`) |
| 52 | + |
| 53 | +**Returns:** A wrapped model with the same interface as the original. |
| 54 | + |
| 55 | +### Disabling Tracing |
| 56 | + |
| 57 | +```typescript |
| 58 | +const model = wrapAISDKModel(openai("gpt-4o-mini"), { |
| 59 | + tracing: false, // Disable automatic traces for this model |
| 60 | +}); |
| 61 | +``` |
| 62 | + |
| 63 | +### Disabling Caching |
| 64 | + |
| 65 | +```typescript |
| 66 | +const model = wrapAISDKModel(openai("gpt-4o-mini"), { |
| 67 | + caching: false, // Disable caching for this model |
| 68 | +}); |
| 69 | +``` |
| 70 | + |
| 71 | +## What Gets Captured |
| 72 | + |
| 73 | +### Tracing |
| 74 | + |
| 75 | +When tracing is enabled, `wrapAISDKModel` automatically captures: |
| 76 | + |
| 77 | +- Full prompt/messages sent to the model |
| 78 | +- Model responses (text and tool calls) |
| 79 | +- Token usage (input, output, total) |
| 80 | +- Timing information (start/end timestamps) |
| 81 | + |
| 82 | +Traces appear in the Evalite UI under each test case. |
| 83 | + |
| 84 | +### Caching |
| 85 | + |
| 86 | +When caching is enabled, `wrapAISDKModel` automatically: |
| 87 | + |
| 88 | +- Generates cache keys from model + parameters + prompt |
| 89 | +- Checks cache before making LLM calls |
| 90 | +- Returns cached responses (0 tokens used) on cache hits |
| 91 | +- Stores new responses in cache for future runs |
| 92 | +- Reports cache hits to the UI |
| 93 | + |
| 94 | +Cache hits are then tracked and displayed in the UI. |
| 95 | + |
| 96 | +### Persistent Caching |
| 97 | + |
| 98 | +By default, Evalite's uses an in-memory [storage](/guides/storage), both for caching and for storing results. |
| 99 | + |
| 100 | +If you want to persist the cache across runs, you can use the SQLite storage adapter. |
| 101 | + |
| 102 | +```ts |
| 103 | +// evalite.config.ts |
| 104 | +import { defineConfig } from "evalite/config"; |
| 105 | +import { createSqliteStorage } from "evalite/sqlite-storage"; |
| 106 | + |
| 107 | +export default defineConfig({ |
| 108 | + storage: () => createSqliteStorage("./evalite.db"), |
| 109 | +}); |
| 110 | +``` |
| 111 | + |
| 112 | +## Works With All AI SDK Methods |
| 113 | + |
| 114 | +`wrapAISDKModel` works with all Vercel AI SDK methods: |
| 115 | + |
| 116 | +**Generate:** |
| 117 | + |
| 118 | +```typescript |
| 119 | +import { generateText } from "ai"; |
| 120 | + |
| 121 | +const result = await generateText({ |
| 122 | + model: wrapAISDKModel(openai("gpt-4")), |
| 123 | + prompt: "Hello", |
| 124 | +}); |
| 125 | +``` |
| 126 | + |
| 127 | +**Stream:** |
| 128 | + |
| 129 | +```typescript |
| 130 | +import { streamText } from "ai"; |
| 131 | + |
| 132 | +const result = await streamText({ |
| 133 | + model: wrapAISDKModel(openai("gpt-4")), |
| 134 | + prompt: "Hello", |
| 135 | +}); |
| 136 | + |
| 137 | +const text = await result.text; |
| 138 | +``` |
| 139 | + |
| 140 | +**Generate Object:** |
| 141 | + |
| 142 | +```typescript |
| 143 | +import { generateObject } from "ai"; |
| 144 | +import { z } from "zod"; |
| 145 | + |
| 146 | +const result = await generateObject({ |
| 147 | + model: wrapAISDKModel(openai("gpt-4")), |
| 148 | + schema: z.object({ name: z.string() }), |
| 149 | + prompt: "Generate a person", |
| 150 | +}); |
| 151 | +``` |
| 152 | + |
| 153 | +**Stream Object:** |
| 154 | + |
| 155 | +```typescript |
| 156 | +import { streamObject } from "ai"; |
| 157 | +import { z } from "zod"; |
| 158 | + |
| 159 | +const result = await streamObject({ |
| 160 | + model: wrapAISDKModel(openai("gpt-4")), |
| 161 | + schema: z.object({ name: z.string() }), |
| 162 | + prompt: "Generate a person", |
| 163 | +}); |
| 164 | + |
| 165 | +const object = await result.object; |
| 166 | +``` |
| 167 | + |
| 168 | +## Behavior in Production |
| 169 | + |
| 170 | +`wrapAISDKModel` is a no-op when called outside an Evalite context: |
| 171 | + |
| 172 | +- Tracing: No traces are captured (no performance overhead) |
| 173 | +- Caching: No cache reads or writes occur (normal LLM behavior) |
| 174 | + |
| 175 | +This means you can safely use `wrapAISDKModel` in production code without any performance impact. |
| 176 | + |
| 177 | +## See Also |
| 178 | + |
| 179 | +- [Vercel AI SDK Guide](/tips/vercel-ai-sdk) - Complete integration guide with examples |
| 180 | +- [`reportTrace()` Reference](/api/report-trace) - Manual trace reporting for non-AI SDK calls |
| 181 | +- [Configuration Guide](/guides/configuration) - Global cache configuration options |
| 182 | +- [CLI Reference](/api/cli) - Command-line flags for controlling cache behavior |
0 commit comments