Skip to content

Conversation

davidhewitt
Copy link
Contributor

Change Summary

This diff wraps all validators and serializers across the codebase in Arc, for two reasons:

  • This makes it possible to re-use simple validators, making progress to reduce memory usage observed in Memory Usage From Schema Validators Is Fairly High, Even With Simple Models pydantic#12272
  • It makes it possible to do further optimizations:
    • Re-use of portions of serializer / validator trees (left for future follow up, figuring out where to deduplicate is hard)
    • Tree traversal algorithms which rewrite portions of trees (this would allow schema cleaning to move into Rust)

In this PR, the memory usage from the repro in #12272 falls from about 600KB per model to 500 KB. We should be able to do more to improve this based on the follow ups.

image

Related issue number

Partial improvement for pydantic/pydantic#12272

Checklist

  • Unit tests for the changes exist
  • Documentation reflects the changes where applicable
  • Pydantic tests pass with this pydantic-core (except for expected changes)
  • My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Copy link

codspeed-hq bot commented Sep 26, 2025

CodSpeed Performance Report

Merging #1795 will not alter performance

Comparing dh/arc-validators (d9dbeeb) with main (ecbc893)

Summary

✅ 163 untouched

Copy link
Member

@Viicos Viicos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Seems like caching simple validators is straightforward, going to be more challenging for more complex cases.

@Viicos Viicos changed the title reuse basic validators and serializers Reuse basic validators and serializers Sep 29, 2025
@davidhewitt davidhewitt merged commit ed0d1ca into main Sep 29, 2025
32 checks passed
@davidhewitt davidhewitt deleted the dh/arc-validators branch September 29, 2025 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants