Preserve cross-page PDF table rows during indexing - #828
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Superseded by #830, which uses the branch pushed directly to |
Context
Large PDF tables can split one logical row over several pages. Continuation pages often omit the identifying cells, so the parser output no longer carries enough context for reliable retrieval.
Problem
OpenRAG currently chunks those continuation fragments independently. This can separate supporting text from its row identity and allow unrelated rows to be mixed in retrieval results.
Expected behavior
Indexation presets can opt into conservative automatic reconstruction. High-confidence continuations across adjacent pages are indexed as self-contained, row-aware chunks with source provenance. Uncertain cases preserve the original parser output, and the feature remains disabled by default for backward compatibility.
Validation
CST salarié/L. 421-1row across all three pages while keeping the nextL. 421-3row separate.No database or vector-store migration is required.