Preserve cross-page PDF table rows during indexing

[hedhoud]

Context

Large PDF tables can split one logical row over several pages. Continuation pages often omit the identifying cells, so retrieval loses the relationship between the row identity and its supporting text. Marker can also represent part of such a table as plain text or an image caption, which meant the first implementation only worked reliably with PyMuPDF parser output.

Expected behavior

Indexing presets can opt into conservative automatic reconstruction. Marker remains the primary parser, while a CPU-only PyMuPDF sidecar supplies layout evidence for matched table regions. Confirmed continuations are indexed as self-contained column: value row chunks; the original Marker blocks remain available for debugging. Ambiguous captions, headers, boundaries, or overlapping regions preserve the complete parser output unchanged. The feature remains disabled by default.

Chunk provenance records both the original parser anchor and the layout provider. Oversized rows split at semantic and source-page boundaries, repeat their identity columns, and cite the page containing each part's content.

Validation

• Backend unit suite: 2,242 passed.
• Admin UI suite: 171 passed.
• Backend and UI lint, formatting, and production build passed.
• The pages 803–805 regression reconstructs CST salarié / L. 421-1 across all three pages and keeps L. 421-3 as a separate row.
• Marker-shaped Markdown, plain-text continuation, synthetic-column, and image-caption paths are covered, including ambiguous fail-open cases.

No database or vector-store migration is required.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: eb8ae635-1020-4185-ad74-003bbd8610bf

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/cross-page-table-reconstruction

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}