Add SearXNG web search provider - #923
Conversation
Adds a third built-in web search provider for self-hosted SearXNG metasearch instances: no API key, no external dependency, which fits the self-hosted deployments that run this plugin against local OpenAI-compatible backends. Provider follows the Google/Brave pattern (GET /search?q=&format=json, results mapped to title/url/snippet). A 403 gets a targeted hint that SearXNG only serves json when it is listed under search.formats -- the most common misconfiguration. System console gains the provider option with Base URL and result limit; the base-URL help text points operators at AllowedUntrustedInternalConnections for instances on private addresses, since that guard silently blocks reserved ranges. Tests cover result mapping and trimming, limit truncation, trailing slash tolerance, the 403 hint, non-200 errors, empty results, and the nil-client guard. websearch, mmtools, and config suites pass; en.json regenerated via make i18n-extract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016e12GP7EHU2KeCGGYcfvBo
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
|
Warning Review limit reached
Next review available in: 24 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Repository UI (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughAdds self-hosted SearXNG web search support, including backend provider execution, configuration and tool wiring, result limits, system-console settings, translations, and automated provider tests. ChangesSearXNG provider integration
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant SearchTool
participant SearXNGProvider
participant SearXNGServer
SearchTool->>SearXNGProvider: Submit query and result limit
SearXNGProvider->>SearXNGServer: GET /search?q=query&format=json
SearXNGServer-->>SearXNGProvider: Return JSON results or HTTP error
SearXNGProvider-->>SearchTool: Return trimmed, limited SearchResponse
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@mmtools/web_search.go`:
- Around line 169-178: In Tool(), trim webCfg.SearXNG.BaseURL once before the
empty-value validation, validate the trimmed value, and pass that same value to
NewSearXNGProvider. Update SourceTool() to use the trimmed BaseURL for its empty
check as well, preserving the existing misconfiguration behavior.
In `@webapp/src/components/system_console/web_search/web_search_panel.tsx`:
- Line 91: Update the SearXNG option in the provider configuration to render its
“SearXNG (self-hosted)” label through react-intl’s FormattedMessage, matching
the surrounding translated labels, then rerun i18n-extract to add the message to
the catalog.
In `@websearch/searxng_test.go`:
- Around line 31-129: Convert TestSearXNGProvider into a table-driven test with
one case entry for each of the seven scenarios currently defined by t.Run.
Centralize provider creation, Search invocation, and common error/response
assertions where possible, while allowing each case to supply its own HTTP
handler and specialized expectations; preserve all existing request validation
and outcomes.
In `@websearch/searxng.go`:
- Around line 40-90: Update SearXNGProvider.Search so every early-return failure
records the error on the span, including request creation, missing HTTP client,
403, other non-OK responses, and response decoding. Add safe telemetry
attributes identifying the failure type and HTTP status code where available,
without including the search query; preserve the existing client.Do error
recording and avoid exposing sensitive data.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 6107e419-5376-486d-8e7d-49351e60a066
📒 Files selected for processing (7)
config/config.gommtools/web_search.gowebapp/src/components/system_console/config.tsxwebapp/src/components/system_console/web_search/web_search_panel.tsxwebapp/src/i18n/en.jsonwebsearch/searxng.gowebsearch/searxng_test.go
…, i18n label Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016e12GP7EHU2KeCGGYcfvBo
|
All four review findings addressed:
|
|
This PR has been automatically labelled "stale" because it hasn't had recent activity. |
|
Bumping to clear the stale label — all four review findings were addressed back in July and coderabbit approved the follow-up. Ready for a core team review whenever someone has cycles; happy to rebase if needed. |
2 participants
Summary
Adds SearXNG as a third built-in web search provider, alongside Google Custom Search and Brave. SearXNG is the de-facto self-hosted metasearch engine; this gives the plugin's self-hosting audience agent web search with no API key, no external dependency, and no per-query cost.
websearch/searxng.go-- provider following the Google/Brave pattern:GET {baseURL}/search?q=...&format=json, results mapped to title/URL/snippet, trimmed and truncated to the configured limit. A 403 response carries a targeted hint that SearXNG only servesjsonwhen listed undersearch.formatsin its settings (the most common misconfiguration).config/config.go--WebSearchSearXNGConfig(baseURL,resultLimit) onWebSearchConfig.mmtools/web_search.go--searxngcase in the provider switch, the SourceTool credential check (base URL required), and the result-limit resolution.AllowedUntrustedInternalConnectionsfor instances on private/reserved addresses, since that guard otherwise blocks them silently.webapp/src/i18n/en.jsonregenerated viamake i18n-extract.Built and verified against a live self-hosted SearXNG instance feeding a llama.cpp-backed agent.
QA steps
jsoninsearch.formats.Tests:
websearch/searxng_test.gocovers result mapping/trimming, limit truncation, trailing-slash tolerance, the 403 hint, non-200 errors, empty results, and the nil-client guard.go test ./websearch/ ./mmtools/ ./config/all pass; eslint clean on touched webapp files.Ticket Link
Fixes #922
Screenshots
N/A (new provider section follows the exact layout of the existing Google/Brave sections)
🤖 Generated with Claude Code
https://claude.ai/code/session_016e12GP7EHU2KeCGGYcfvBo
Summary by CodeRabbit