Improve search context efficiency to prevent token overflow

[codegen-sh]

$(cat <<'EOF'

Summary

Addresses feedback from issue #12 to improve context efficiency and prevent token overflow errors when searching documents with OCR content.

Changes

1. Removed list_documents tool

• This tool was causing LLMs to fall back unnecessarily when search_documents failed
• Rarely useful for finding specific documents
• LLMs now focus on using search_documents with proper query syntax

2. Filter out content field from search results

• OCR content can contain thousands of words per document
• With page size of 10, this easily exceeds the 25k token limit
• Search results now return metadata without full text content
• Use get_document(id) to retrieve full details for specific documents

3. Remove long URLs from search results

• Dropped download_url and thumbnail_url fields (token-heavy)
• Document id is still included for reference
• URLs can be constructed if needed or accessed via get_document

4. Comprehensive search syntax documentation

Added full Paperless-NGX query language documentation to the tool description:

• Field-specific searches: tag:unpaid, type:invoice, correspondent:university
• Logical operators: term1 AND (term2 OR term3)
• Date ranges: created:[2020 to 2024], added:yesterday, modified:today
• Wildcards: prod*name

5. Pagination support

• Added page and page_size parameters to search_documents
• Helps manage large result sets without hitting token limits
• Smaller page sizes prevent token overflow with many matching documents

6. Updated tool descriptions

• Removed all references to list_documents throughout the codebase
• Updated get_document, download_document, and bulk_edit_documents descriptions

Improved Workflow

This creates a more efficient pattern where LLMs:

1. Search using search_documents with proper query syntax → get document IDs and metadata
2. Identify relevant documents from the search results
3. Retrieve full details using get_document only for documents of interest

This prevents token overflow while maintaining full functionality.

Testing

• ✅ TypeScript build passes without errors
• ✅ All tool descriptions updated consistently

Closes #12

🤖 Generated with Claude Code
EOF
)

---

💻 View my work • 👤 Initiated by @nloui • About Codegen
⛔ Remove Codegen from PR • 🚫 Ban action checks