Skip to content

Caching AI SDK Models #317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 17 commits into from
Nov 14, 2025
Merged

Caching AI SDK Models #317

merged 17 commits into from
Nov 14, 2025
+2,097 −666

Conversation

@mattpocock
Copy link
Owner

@mattpocock mattpocock commented Nov 10, 2025 •
edited
Loading

Fixes #309

@changeset-bot
Copy link

changeset-bot bot commented Nov 10, 2025 •
edited
Loading

🦋 Changeset detected

Latest commit: b8a1dfc

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link

vercel bot commented Nov 10, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Preview Comments Updated (UTC)
evalite Ignored Ignored Nov 14, 2025 11:26am
evalite-beta-docs Ignored Ignored Nov 14, 2025 11:26am

@mattpocock mattpocock changed the base branch from main to v1 November 10, 2025 17:58
@mattpocock mattpocock changed the title matt/caching ai sdk models Caching AI SDK Models Nov 10, 2025
@pkg-pr-new
Copy link

pkg-pr-new bot commented Nov 10, 2025

Open in StackBlitz

npm i https://pkg.pr.new/mattpocock/evalite@317

commit: 0e0fcc0

@mattpocock mattpocock marked this pull request as draft November 10, 2025 17:59
@mattpocock mattpocock marked this pull request as ready for review November 10, 2025 21:29
@mattpocock mattpocock marked this pull request as draft November 10, 2025 21:29
@mattpocock mattpocock force-pushed the matt/caching-ai-sdk-models branch from c1a2208 to 86b625c Compare November 11, 2025 09:18
@mattpocock
Copy link
Owner Author

mattpocock commented Nov 11, 2025

There are a couple of things I'm not happy with about this PR.

Overall, the implementation is solid, and the UI is nice. But:

  • Scorers should show cache hits individually, since this gives a better breakdown to users

  • We should consolidate tracing and caching into a single wrapAISDKModel call

It doesn't really make sense to me to have caching without tracing, or tracing without caching. If you want a great AI SDK integration, you really just want to call one function, wrap your model, and you're good to go. We can have options to disable tracing or disable caching, but having a single function seems indispensable.

My thinking is that inside scorers, we would want to prevent tracing but allow caching, since scorer traces don't make sense in the trace view.

@mattpocock mattpocock force-pushed the matt/caching-ai-sdk-models branch from 9bf49cd to 0061595 Compare November 14, 2025 10:34
@mattpocock mattpocock force-pushed the matt/caching-ai-sdk-models branch from 0061595 to 61ece64 Compare November 14, 2025 10:41
@mattpocock mattpocock marked this pull request as ready for review November 14, 2025 11:14
@mattpocock mattpocock merged commit 4fd065e into v1 Nov 14, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mattpocock