feat(ai): add Anthropic batch provider#7497
Draft
artell0 wants to merge 2 commits into
Draft
Conversation
Adds AnthropicBatchWrapper, an Anthropic Message Batches implementation that mirrors the public surface of GoogleGenAIBatchWrapper so the batch management commands can target either provider. The data model already declared LLMProvider.ANTHROPIC and references a Claude model in the api_model_name help text; this fills in the provider itself. Two differences from the Gemini path simplify the implementation: - messages.batches.create accepts the request list directly, so there is no intermediate JSONL file to upload. - Prompt caching is implicit prefix caching applied via a cache_control breakpoint on the system block, so there is no cache object to manage. Includes unit tests (SimpleTestCase, client mocked) covering model validation, request preparation (inline base64 document + text), system cache attachment, status retrieval, and result parsing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|
for more information, see https://pre-commit.ci
Member
|
Hi, I'm not sure this is something we need? Is there more (human) context for this? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes
Part of #6213 (Incorporate LLMs into Document Ingestion & Processing).
Summary
Adds
AnthropicBatchWrapper(cl/ai/llm_providers/anthropic.py), anAnthropic Message Batches implementation that mirrors the public surface
of
GoogleGenAIBatchWrapperso the batch management commands can targeteither provider.
LLMProvider.ANTHROPICwas already declared andLLMRequest.api_model_name's help text already references a Claude model —this fills in the provider itself.
Following the pattern noted in
check_gemini_batch_status.py("If you add other providers (OpenAI, Anthropic), create separate
commands"), I kept this as an independent wrapper alongside
google.py.Two differences from the Gemini path simplify the implementation:
messages.batches.createaccepts the request list directly — there is nointermediate JSONL file to upload.
cache_controlbreakpoint on the system block — there is no cache object to create or
look up, so
get_or_create_cachehas no analog here.Choices I made (happy to change if you'd prefer otherwise):
documentblocks (matches the Geminiwrapper's
input_file_path/input_texttask shape). Inline documentsare capped at 32 MB / 600 pages per request; if your scans routinely
exceed that, I'll switch to the Files API instead.
can be reviewed before I add
send_anthropic_file_batches/check_anthropic_batch_status. The S3/task-staging helpers in the Geminicommand are provider-agnostic — I can factor them into a shared module
when wiring the Anthropic command, if that's the direction you want.
max_tokensdefaults to 8192 per task (overridable onexecute_batch); no default model is hard-coded in the wrapper.Tests:
SimpleTestCasewith the Anthropic client mocked (no live calls,no DB) — model validation, request preparation (inline base64 document +
text, skip-missing-key), system
cache_controlattachment, statusretrieval, unfinished-batch guard, and result parsing (succeeded + errored).
Mirrors the existing
ValidateGeminiModelTest/GoogleGenAIBatchWrapperTeststyle.
The Anthropic API key is referenced by env var only and routed through the
same secret mechanism as the Gemini key; nothing is committed.
Deployment
This PR should:
skip-deploy(skips everything below)skip-web-deployskip-celery-deployskip-cronjob-deployskip-daemon-deploy