Skip to content

MCP tool for retrieving code from Freva data analysis plugins - #49

Open
alex-fischer-97 wants to merge 48 commits into
mainfrom
freva_plugins
Open

alex-fischer-97 wants to merge 48 commits into
mainfrom
freva_plugins

Conversation

@alex-fischer-97

@alex-fischer-97 alex-fischer-97 commented Jul 9, 2026 •

Copy link
Copy Markdown

This pull request adds support for a new MCP server, plugin-code-search-server, which enables automated and query-suited code context retrieval from GitLab repositories of Freva plugins for climate data analysis.

The tools makes several calls to the used LLM via LiteLLM to automatically scan (given the repository tree list) and select those parts of the repo's source code (realized in 2 stages: first scrape the base, then their dependency files) – coming from the most fitting Freva plugin –, that seems most relevant to the user's query.
The extracted code is then analyzed in the following to guide the answer of the user w.r.t. data analysis for climate-related questions.

It is supposed to be called by LiteLLM when the user

  • explicitly asks how a plugin's internal logic works, how to run or
    configure it, or when code snippets from the plugin should be translated or
    adapted into Python examples;
  • asks general questions about decadal climate prediction analysis, where repository-grounded code context could be useful to guide the answer to the question.

This PR also updates the documentation, environment, and Docker configurations to integrate this new service into the ClimateClaw stack. The most important changes are summarized below.


Plugin Code Search Server Integration

  • Added a new folder plugin_code_search (src/climateclaw/tools/plugin_code_search), with server.py for the main code discovery and fetching logic; as well as available_plugins.md with names & descriptions of all relevant plugins.
  • Added a new plugin-code-search-server MCP service, including its Dockerfile (docker/plugin-code-search-server/Dockerfile) and pyproject.toml config file
  • Added the plugin-code-search-server service into both docker-compose.yml and docker-compose.dev.yml with appropriate health checks, environment variables, and volume mounts

Documentation Updates

  • Expanded the README.md to document the new plugin-code-search-server, its purpose, and how it fits into the MCP tool ecosystem, including updates to the architecture and quick start sections.

Closes #115

Alexander Fischer and others added 22 commits April 10, 2026 18:56
… for searching and analyzing Freva plugins
update plugin code search functionality to include new file paths;
improve context retrieval for Freva plugins.
improve file selection logic for user queries to search for additional code dependencies
…de context and clarify Python snippet generation process.
…de and improve context collection process.
…and adjust max relevant files for plugin code search

Co-authored-by: Copilot <copilot@github.com>
enhance/refactor plugin code search functionality by Freva group (to allow for more than just ComDec);
start with implementation of security check to allow only granted group members access to tool
… visibility, user_id + access level of current user + repo to determine if user has read rights for requested codebase
…gnatures;

add user identification in GitLab access validation method via HTTP headers;
cleanup + polishing methods
- add user access denial message and update project names reference;
- refined system prompt for plugin_code_search tool in case of denied user access
- Updated the `plugin_code_search` function to auto-detect plugin and project names from user queries.
- Improved the scope description for `plugin_code_search` to clarify its usage.
- Added a new `available_plugins.md` file listing available Freva plugins and their functionalities.
…"available_plugins.md" with new project names + descriptions
- introduce dedicated plugin-code-search server for plugin_code_search MCP tool (Freva plugin code retrieval)
- cleanup of web-search server
- new docker container for this tool, populated docker-compose.dev.yml + .env.example accordingly
@alex-fischer-97
alex-fischer-97 requested a review from gekinci July 9, 2026 14:27
@alex-fischer-97 alex-fischer-97 self-assigned this Jul 9, 2026
@alex-fischer-97 alex-fischer-97 added documentation Improvements or additions to documentation enhancement New feature or request labels Jul 9, 2026
Comment thread scripts/evaluation/tool_evaluation_metrics_2026-08-13.png Outdated
Comment thread scripts/evaluation/benchmark-prompts-plugin-server.json
Comment thread .github/workflows/ci_job.yml Outdated
Comment thread .github/workflows/ci_job.yml Outdated
Comment thread src/climateclaw/prompt_library/baseline/starting_prompt.md
Comment thread src/climateclaw/core/prompting.py
Comment thread src/climateclaw/prompt_library/local_model/starting_prompt.md
out = "Unknown response."
web_sv = SVToolOutput(content=out, tool_name=tool_name, id=id) # type: ignore[arg-type]
web_msg = help_convert_sv_ccrm([web_sv])
yield web_sv

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output of the generic tool is not streamed to the client, only to the model.

Suggested change
yield web_sv

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the evaluation script, I implemented this though to evaluate if the correct tool got called. In my tests with localhost, this did not result in the generic tool output being displayed in the chat interface.

Comment thread scripts/dev_chat.py Outdated
Comment thread scripts/dev_chat.py Outdated
Comment thread src/climateclaw/prompt_library/baseline/summary_prompt.md Outdated

@gekinci gekinci left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to separate GPT-5 related changes to another PR.

For future PRs, it would also help to avoid unrelated or purely formatting changes, since keeping the diff focused makes the review process much easier.

@alex-fischer-97 alex-fischer-97 removed the documentation Improvements or additions to documentation label Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integration of Code Search MCP Tool into Backend

3 participants