Skip to content

kb: make scanned PDFs searchable (OCR) #941

Description

@clemenshelm

Wave 2. Depends on #936 (it hangs off the same spine: anything → PDF → text layer? → extract or OCR).

Why this is not optional polish

Measured on the real Noack corpus: 25 of 193 PDFs carry no text layer, and 19 of those 25 sit under 3_Appr_Certif_Spec/ — AFNOR validations, NordVal certificates, ISO 9001, BSI. The approval documents. For a technical-sales team, "is this method AFNOR-validated?" is close to the most frequently asked question, and it is the one category the index cannot answer today. See #935 for the visibility half of this problem.

OpenClaw already implements the decision rule — copy it, do not invent it

OpenClaw's native pdf tool, extraction-fallback mode (docs/tools/pdf.md, v2026.7.1):

  1. extract text for the selected pages via the bundled document-extract plugin (clawpdf, PDFium WebAssembly)
  2. if the extracted text is shorter than 200 characters, render the same pages to PNG
  3. render budget 4,000,000 pixels total, allocated proportionally across the pages that still need images — pages with enough text skip rendering
  4. send text + images to the model; if the model has no image input and there is no text, error

Use the same threshold, order and budget. We are not guessing at a sensible cut-off; there is a reference implementation in the runtime we ship.

packages/plugins/pinchy-files/pdf-render.ts already renders PDF pages to images (pdfjs + @napi-rs/canvas, 4M pixel budget) — the same building block, already in the repo.

Why we do not simply call the native tool

Its defaults show it is built for agent-time analysis of a few pages, not bulk indexing:

  • pdfMaxPages = 20, pdfMaxBytesMb = 10
  • the corpus's largest PDF is 268 MB; the next is 174 MB / 1159 pages; 6 PDFs exceed the 10 MB default

It also runs as an agent tool, while indexing runs server-side in the Pinchy container with no agent in the loop.

The decision that is not technical

Two paths, and they differ in what leaves the building:

  • Local OCR (e.g. Tesseract) — keeps the promise made to the customer on camera: only the question and the retrieved snippets go to the LLM provider.
  • Vision model — better on tables and certificates, but it sends every scanned document in full to the provider at index time. That is a materially different statement from the one the customer was given, and it must be an explicit, documented choice — not a default someone discovers later.

Recommendation: local OCR as the default, vision as an opt-in per instance, with the data-flow consequence stated in the docs. If vision is used, follow the capability+auth-gated model-selection pattern OpenClaw uses for tools.media rather than hard-coding a provider.

Acceptance

  • Text-layer detection uses the 200-character threshold; pages with text are never rendered
  • Render budget bounded (4M pixels), so a 1159-page scan cannot exhaust memory
  • OCR failures are document-level and land in kb: show which documents the index could not read #935, never silent
  • Default is local; enabling vision requires an explicit setting and is documented as sending document content to the provider
  • Measured on the 25 real corpus scans: how many become searchable, and how long it takes

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions