Skip to content

feat(ai): add Anthropic batch provider#7497

Draft
artell0 wants to merge 2 commits into
freelawproject:mainfrom
artell0:6213-anthropic-batch-provider
Draft

feat(ai): add Anthropic batch provider#7497
artell0 wants to merge 2 commits into
freelawproject:mainfrom
artell0:6213-anthropic-batch-provider

Conversation

@artell0

@artell0 artell0 commented Jun 25, 2026

Copy link
Copy Markdown

Fixes

Part of #6213 (Incorporate LLMs into Document Ingestion & Processing).

Summary

Adds AnthropicBatchWrapper (cl/ai/llm_providers/anthropic.py), an
Anthropic Message Batches implementation that mirrors the public surface
of GoogleGenAIBatchWrapper so the batch management commands can target
either provider. LLMProvider.ANTHROPIC was already declared and
LLMRequest.api_model_name's help text already references a Claude model —
this fills in the provider itself.

Following the pattern noted in check_gemini_batch_status.py
("If you add other providers (OpenAI, Anthropic), create separate
commands"), I kept this as an independent wrapper alongside google.py.

Two differences from the Gemini path simplify the implementation:

  • messages.batches.create accepts the request list directly — there is no
    intermediate JSONL file to upload.
  • Prompt caching is implicit prefix caching applied via a cache_control
    breakpoint on the system block — there is no cache object to create or
    look up, so get_or_create_cache has no analog here.

Choices I made (happy to change if you'd prefer otherwise):

  • File inputs: inline base64 document blocks (matches the Gemini
    wrapper's input_file_path / input_text task shape). Inline documents
    are capped at 32 MB / 600 pages per request; if your scans routinely
    exceed that, I'll switch to the Files API instead.
  • No commands yet: this PR is the provider + tests only, so the shape
    can be reviewed before I add send_anthropic_file_batches /
    check_anthropic_batch_status. The S3/task-staging helpers in the Gemini
    command are provider-agnostic — I can factor them into a shared module
    when wiring the Anthropic command, if that's the direction you want.
  • max_tokens defaults to 8192 per task (overridable on
    execute_batch); no default model is hard-coded in the wrapper.

Tests: SimpleTestCase with the Anthropic client mocked (no live calls,
no DB) — model validation, request preparation (inline base64 document +
text, skip-missing-key), system cache_control attachment, status
retrieval, unfinished-batch guard, and result parsing (succeeded + errored).
Mirrors the existing ValidateGeminiModelTest / GoogleGenAIBatchWrapperTest
style.

The Anthropic API key is referenced by env var only and routed through the
same secret mechanism as the Gemini key; nothing is committed.

Deployment

This PR should:

  • skip-deploy (skips everything below)
    • skip-web-deploy
    • skip-celery-deploy
    • skip-cronjob-deploy
    • skip-daemon-deploy

Adds AnthropicBatchWrapper, an Anthropic Message Batches implementation
that mirrors the public surface of GoogleGenAIBatchWrapper so the batch
management commands can target either provider. The data model already
declared LLMProvider.ANTHROPIC and references a Claude model in the
api_model_name help text; this fills in the provider itself.

Two differences from the Gemini path simplify the implementation:
- messages.batches.create accepts the request list directly, so there is
  no intermediate JSONL file to upload.
- Prompt caching is implicit prefix caching applied via a cache_control
  breakpoint on the system block, so there is no cache object to manage.

Includes unit tests (SimpleTestCase, client mocked) covering model
validation, request preparation (inline base64 document + text), system
cache attachment, status retrieval, and result parsing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@mlissner

Copy link
Copy Markdown
Member

Hi, I'm not sure this is something we need? Is there more (human) context for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants