Skip to content

refactor(knowledge): 提取可复用知识模块 - #3768

Open
sunnights wants to merge 57 commits into
wecode-ai:mainfrom
sunnights:refactor/pure-knowledge-contracts
Open

sunnights wants to merge 57 commits into
wecode-ai:mainfrom
sunnights:refactor/pure-knowledge-contracts

Conversation

@sunnights

@sunnights sunnights commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

本文仅说明相对 sunnights:refactor/rag-remove-local-mode 的新增改动。

背景

知识配置和文档处理规则与产品数据模型耦合,Backend 与 Runtime 也存在重复配置解析。本 PR 在已有远程 RAG 链路上抽取可复用的知识模块,集中维护通用规则,产品侧继续负责权限、数据存取和任务调度。

改动

  • 提取不依赖产品 ORM 的知识契约和模块,统一配置、查询规划、文档转换、索引和管理规则。
  • Backend 校验知识访问和资源使用权限,Runtime 根据授权引用加载资源,由模块解析执行配置。
  • 公开检索实际使用调用方选择的 Retriever 和 Embedding,并校验真实模型类别;空文档范围直接返回空结果。
  • 重建和删除先清理旧索引,清理失败保留记录;处理过期转换回调及删除后的迟到索引。
  • 补充模块复用、授权和文档生命周期测试,并将远程 E2E 接入 CI。

Move the runtime config, search hint, retrieval scope and splitter config
contracts into a new pure shared.knowledge_contracts package so the knowledge
execution kernel can import them without loading Wegent product ORM models,
database sessions or task workers.

shared.models still exposes the identical objects, so every existing caller
keeps one contract definition. The stale submodule paths
shared.models.{runtime_config,search_hints,splitter_config} and their importers
are gone, and backend-rs doc comments point at the new module.

Ticket: .scratch/open-source-knowledge-module/issues/01-pure-knowledge-contracts.md
Creating or editing a knowledge base now resolves the retriever, embedding model, retrieval mode and retrieval parameters in shared.knowledge_module. The module owns the composition priority (explicit input, valid system profile, authorized candidates) and the validation rules (resource category, mode, top_k, threshold, hybrid weights), while the Wegent side only reports the records it has authorized.

The system profile health check and the profile update schema reuse the module's rules, so the mode whitelist and the resource category check exist once. A minimal client that never loads Wegent ORM models proves the same entry point produces a saveable configuration.

Ticket: .scratch/open-source-knowledge-module/issues/02-knowledge-config-create-update.md
Creating a knowledge base now asks the adapter to resolve the finally selected retriever and embedding model, and rejects a reference the caller cannot use, one that resolves to another resource, or one whose category does not match its slot. The module keeps owning the rule and still queries no product table.

Wegent reuses the existing retriever service (its group permission check and public fallback) and the same caller-visible model resolution the runtime uses, so a legitimate public fallback stays accepted and no ACL is added. Profile health also requires the resolved records to correspond to the references the profile configures.

Ticket: .scratch/open-source-knowledge-module/issues/02-knowledge-config-create-update.md
…ng model

A group-owned Model Kind was resolved by name and namespace without checking that the caller may use that group, so a personal knowledge base could store an embedding reference to another group's model.

The Wegent adapter now applies the same namespace rule the retriever path already applies — the personal namespace belongs to its caller, organization namespaces are visible to everyone, and a group namespace needs at least Reporter membership — before reporting the record, and also checks the namespace a reference actually resolves to when it differs from the requested one. The module is unchanged: authorization stays in the adapter.

Ticket: .scratch/open-source-knowledge-module/issues/02-knowledge-config-create-update.md
A Backend product query now carries the retrieval resources it authorized for
this operation: the caller's knowledge base access and the knowledge base
owner's retriever / embedding usage are checked at the entry point, and the
remote request sends only those references. knowledge_runtime loads just the
authorized records and resolves the stored configuration once through
shared.knowledge_module, so the configuration Backend used to compute and the
remote request dropped is gone. A stored configuration edited to a resource
outside the authorized set fails instead of silently executing it.

Remote failures are surfaced; the deprecated local data plane is no longer a
query fallback. Historical stored configurations keep the 20/0.7 execution
fallback, and an empty document scope still returns no query instead of an
unscoped one. The shared internal service token only proves possession and is
documented as an MVP trust boundary, not an authorization credential.

Adds resolve_execution_config to the reusable module plus a minimal caller that
resolves a configuration and runs a scope-limited query without Wegent tables.

Ticket: .scratch/open-source-knowledge-module/issues/03-authorized-remote-query.md
Closing the three authorization gaps found reviewing the remote query closure:

- Backend now requires an explicit read subject and reuses the existing
  task-scoped read rule plus the knowledge base permission check, so a caller
  holding only the shared internal token can no longer query another user's
  knowledge base. Knowledge bases without an identity or without access are
  rejected before any gateway is selected.
- The knowledge base owner's retrieval resources are authorized through the
  same adapter the create path uses, so group membership, resolved ownership
  and the embedding category follow one rule. An owner removed from the group
  fails the next query instead of executing the group embedding model.
- A per-request retrieval override is passed to the module through
  resolve_execution_config; the runtime no longer replaces the resolved config
  wholesale, so effective parameters always pass the module's composition and
  validation, including inconsistent hybrid weights.

Focused regression tests cover an unreadable knowledge base, an owner whose
group membership was revoked, and an override applied (or rejected) by the
module; each asserts no gateway runs for the rejected request.

Ticket: .scratch/open-source-knowledge-module/issues/03-authorized-remote-query.md
Public /rag/retrieve accepted an explicit retriever_ref and
embedding_model_ref but the remote query only carried retrieval parameters,
so knowledge_runtime kept executing the knowledge base's stored resources.

Backend now authorizes the caller's explicit references for the operation
and sends them as the explicit selection next to the authorized set.
knowledge_runtime loads only that authorized set and resolves the selection
through shared.knowledge_module, which rejects a selection outside the set
instead of falling back to stored resources. Queries without a selection
keep resolving the stored configuration, and remote failures stay surfaced
without a local fallback.

Ticket: .scratch/open-source-knowledge-module/issues/04-explicit-retrieval-resources.md
Backend authorization accepted an approved shared Retriever reference, but
knowledge_runtime only searched the requested namespace and the public
fallback, so a valid shared reference could fail once the query reached the
runtime.

Move the direct -> approved reference -> public rule into
shared.db.capability_reference and use it from both Backend and the runtime.
Public queries now carry a single authorized resource set marked with
explicit_selection, so no second same-name selection can diverge, and the
stored-configuration match failure for ordinary queries is unchanged.

Ticket: .scratch/open-source-knowledge-module/issues/04-explicit-retrieval-resources.md
The shared Retriever lookup had put an approved reference ahead of a
same-name public Retriever in the default namespace, which could switch an
existing knowledge base to another index backend after upgrade.

Restore the pre-upgrade order: in default, the caller's own Retriever
precedes the public Retriever, which precedes an approved shared reference.
Other namespaces keep namespace resource -> approved reference -> public
fallback. Backend and knowledge_runtime already share the lookup function.

Ticket: .scratch/open-source-knowledge-module/issues/04-explicit-retrieval-resources.md
Document indexing now carries the retrieval resources Backend authorized for
the knowledge base owner, and knowledge_runtime resolves the index configuration
once through shared.knowledge_module inside that authorized set. The runtime
loads only the authorized retriever and embedding records, so a configuration
edited to a resource outside the set fails instead of silently indexing with it.

Index requests reach the runtime through the remote gateway only; remote
failures surface and no local index call is a fallback. The existing generation
guard still keeps an older indexing task from overwriting a newer result.

The Backend-to-runtime retrieval-resource protocol type is renamed to
RemoteAuthorizedRetrievalResources because indexing and querying now share it.

Ticket: .scratch/open-source-knowledge-module/issues/05-plain-document-remote-index.md
The runtime now rejects authorized retrieval references whose knowledge base
or owner does not match the knowledge base being indexed, before any content is
fetched or any index is written. A misplaced reference can no longer index into
another knowledge base with another owner's resources.

The remote index path no longer resolves the retriever and embedding
configuration the remote request drops. The gateway declares whether it
consumes a resolved configuration, so the local data plane keeps receiving one
while the runtime resolves configuration from the authorized references.

The product chain is covered end to end at each service boundary: the create
and rebuild entries enqueue generations, the task's request validates as the
runtime request model, the runtime index entry forwards it to the executor, and
a document-scoped product query returns the indexed knowledge base and document
reference. Local index and query calls fail the tests.

Ticket: .scratch/open-source-knowledge-module/issues/05-plain-document-remote-index.md
The document create and rebuild entries now run against the real
knowledge_runtime in CI: a plain document is indexed through the Celery task,
the runtime store serves a document-scoped query that returns the knowledge base
and document references, a runtime failure stays visible, and a stale indexing
generation stands down without overwriting the newest result. Local index and
query calls raise inside the same process, and the runtime is the only data
plane the queries accept.

The deterministic E2E embedding now keeps every component positive so a query
text and its indexed chunk still score above a zero threshold; the mock is not
semantic, and a signed distribution made every retrieval assertion a coin flip.

Ticket: .scratch/open-source-knowledge-module/issues/05-plain-document-remote-index.md
…broker

The remote index E2E now queues a superseded generation through the real Celery
broker and reads the embedded worker's decision, so the stale-generation rule is
verified on the product path instead of only in the test process. The query
evidence also pins the surfaced remote failure to the runtime's own status and
the returned detail, and the scenario is split into per-phase helpers with one
shared fixture teardown.

Ticket: .scratch/open-source-knowledge-module/issues/05-plain-document-remote-index.md
The local data plane is deprecated but any of its entry points could still serve
an operation this deployment configured to run remote, silently reading or
writing a different store. LocalRagGateway now refuses indexing, document-index
deletion, and standard retrieval before touching storage, so the Backend process
and the embedded Celery worker fail the same way the test process does. Auto and
direct-injection routing keep their existing local behaviour.

The remote index E2E replaces its process-local monkeypatch with that invariant:
it asserts the remote gateways are selected and that the gateway itself refuses
the local index, delete, and retrieval calls under the deployment settings.

Ticket: .scratch/open-source-knowledge-module/issues/05-plain-document-remote-index.md
…ario

The scenario file crossed the 1000-line split trigger, so the real fixture
builders, the document-scoped assertions, and the local-plane invariant checks
now live in `knowledge_remote_index_support`; the scenario keeps the create,
rebuild, stale-generation, and failure flows. The local query guard reads
`spec.route_mode` directly because the runtime spec always defines it.

Ticket: .scratch/open-source-knowledge-module/issues/05-plain-document-remote-index.md
Converted documents now enter the same remote index chain as plain ones. The
reusable module owns the conversion request rules, the converted filename, the
object-key prefix, the document identity every chunk carries, the normalized
index result, and the conversion lifecycle decisions, so both services convert
and index the same way. Wegent keeps its credentials, persistence, task
scheduling and worker protocol in adapters.

The converter worker builds its request through the module, the runtime index
executor indexes through the module, and the document service and conversion
callbacks read the same shared rules instead of repeating them.

The real remote E2E now converts a PDF end to end - product entry, converter
worker, callback, runtime index, document-scoped query - and covers a duplicate
and a superseded completion callback, a superseded index generation, and a
conversion failure that leaves nothing queryable. CI starts the converter
worker and serves the external MinerU parsing contract from the model mock.

Ticket: .scratch/open-source-knowledge-module/issues/06-converted-document-remote-index.md
…heck

The completed-conversion pre-check loaded the document as an entity, so a
Session that had already cached the document handed the identity-mapped object's
old generation and status to the decision. A callback superseded in the
meantime looked current and reached attachment creation before the conditional
update refused it.

The pre-check now reads only index_generation and index_status as stored column
values, which does not reuse the cached object, and hands those to the module
decision. The generation/status guarded update and its compensation path are
unchanged, so a change that happens after the pre-check is still refused there.

Ticket: .scratch/open-source-knowledge-module/issues/06-converted-document-remote-index.md
The document delete entry removed the row and only then asked
knowledge_runtime to drop the index, swallowing every remote failure: a failed
removal left chunks nothing could address while the caller saw a successful
delete. The removal now runs first and its failure propagates, so the document
stays and the caller can retry. A removal the runtime does not confirm, and a
retrieval configuration that cannot be resolved, are reported as failures
instead of reading as access errors or successes.

A rebuild treated a failed old-index removal as a warning and wrote the new
generation on top of the surviving one; it now stops with a visible failure.
The shared module owns the delete identity and the normalized, idempotent
result, the runtime delete executes through the module's adapter, the
Qdrant/Elasticsearch backends report an empty result when nothing was ever
written instead of failing on a missing collection, and the task compensates a
late index in the failure path as well.

The minimal non-Wegent caller indexes two documents and deletes one through the
module's public interface: only that reference disappears and a repeated delete
reports no removals. The real remote E2E asserts the delete entry resolves to
the remote gateway, deletes one of two documents, and checks the removed
reference stops being hit while the other stays searchable; a repeated delete
and a late index task change nothing, and an unreachable store makes the delete
fail visibly before a retry succeeds.

Ticket: .scratch/open-source-knowledge-module/issues/07-document-index-delete.md
The delete entry removed the remote index and only then deleted the document
row, so an in-flight indexing task could still write and finalize successfully
in that window, skip the late-index compensation, and leave a queryable orphan
once the row was gone. The delete now re-reads and locks the document row before
the removal and holds it until the commit: a task that starts after the delete
blocks and then finds no row, and a task that finishes blocks and then matches
nothing, which routes it into the existing late-index compensation.

A compensation that failed used to be logged and returned as a skipped task, so
nothing ever removed the late chunks again even though no product entry could
repeat the cleanup. A failed compensation now retries the task, a retried task
for a document that no longer exists only cleans its own reference up instead of
re-entering indexing, and the last allowed attempt re-raises so the failure
stays visible. The delete scenarios move to their own E2E module, which keeps
the shared script under the repository split threshold, and two focused
regressions cover the deletion race and the cleanup retry.

Ticket: .scratch/open-source-knowledge-module/issues/07-document-index-delete.md
Review follow-ups on the deletion race and its compensation.

A task whose document is gone but that could not re-acquire the distributed
index lock returned `skipped/lock_retry_exhausted`, which left the late chunks
in place with no retry left; that branch now runs the same cleanup whenever the
document no longer exists. The cleanup retry also normalized its failure: a
worker raises celery Retry, but outside a worker the retry re-raises the
original error, which would have fallen back into the indexing failure path and
removed the reference twice.

The stale skip result now states why it does not compensate - that path stops
before any write - the unreachable `if kb:` guard is gone, the cleanup helper
takes a `Task` hint, and the shared E2E script drops the imports the split left
behind.

New focused regressions: the lock query refreshes a row another writer
advanced, an exhausted lock retry still cleans up a deleted document, and a
direct (non worker) cleanup failure surfaces instead of running the removal
again.

Ticket: .scratch/open-source-knowledge-module/issues/07-document-index-delete.md
The public chunk listing, index purge and physical index drop now execute in
knowledge_runtime and surface its failures, instead of retrying the deprecated
local data plane. The remote path no longer builds the retriever configuration
the request drops, and LocalRagGateway refuses all three operations while the
deployment is configured remote.
Fold the runtime and local admin error mappings into one helper, align the
resolver flag, and reuse the shared remote-gateway and local-refusal E2E
assertions for the purge, drop, and chunk-listing entries.
Extract the shared setup, purge, rebuild, and failure-call helpers so each
scenario reads as one product flow.
Skipping the discarded execution config also skipped the retriever lookup that
carries the group permission verdict, so an owner removed from the retriever's
group could still read or drop the group store. The remote path now performs the
authorized lookup through the existing retriever service and only skips building
and decrypting the configuration the runtime resolves itself.
Fold the local and remote retriever handling into one resolver so the owner
access verdict is unconditional and only the configuration build is switched.
Add a database-backed group-permission regression for the three public
index-management entries, and require the cleared knowledge base to still answer
an empty query instead of accepting any visible failure.
…esolver

The resolver passed the 1000-line threshold. Extract the retriever and embedding
configuration resolution, including the owner access verdict and credential
decryption, into its own module so the resolver only composes runtime specs.
… module

Move the direct-injection capability (context fit decision plus reading the
original MySQL document bodies) out of `RetrievalService` and into
`app/services/rag/direct_injection.py`. The new module imports no vector store,
embedding model or query executor, so routing rules can change without pulling
in the retrieval execution stack.

The internal retrieve endpoint and the knowledge search service now enter the
injection path through that module instead of the local gateway's composite
retrieval entry. Injection payloads (mode, records, fields) and rejection
behaviour are unchanged, and a rejected injection still continues with the
pre-existing Backend-local retrieval, which a follow-up change moves to the
remote gateway. The knowledge runtime is still untouched by this change.

Route decisions that were previously only exercised through
`RetrievalService.decide_route_mode_for_chat_shell()` now live on the module;
tests were moved with them. Two guards assert the module neither imports nor
loads the execution kernel.
A direct injection that is rejected (truncated documents, context budget or
chunk limit) now continues as a knowledge_runtime query instead of the
Backend-local retrieval, and the knowledge search service resolves the
injection decision in the Backend and retrieves remotely otherwise. The
Backend no longer executes any retrieval in-process on these paths.

knowledge_runtime executes queries with the runtime configs the Backend already
resolved (`RemoteQueryRequest.knowledge_base_configs`), so it needs no Backend
database lookup for those requests; requests without configs keep resolving
them from the database as before.

Remote query failures are no longer retried locally: the internal retrieve
endpoint maps them to the existing failure response, and the search service
propagates the error, matching the previous remote-mode behaviour. Tests now
assert this at the knowledge-runtime HTTP call seam.
The scope-preservation test still built QueryExecutor with a `db` argument
the class no longer accepts, and mocked a `_config_resolver` attribute that
no longer exists. Both were left over from the switch to reference mode, so
the test failed before reaching any assertion and the suite stayed red.

Use the config loader the neighbouring tests use, matching the current
interface.
RAG execution now has exactly one home: knowledge_runtime. The Backend keeps
only the control plane, so "what runs in production" is readable from the code
instead of inferred from configuration.

Removed:

- `LocalRagGateway`, the local data plane (indexing, retrieval, administration)
  and the embedding compatibility package that only served it.
- `RetrievalService`, which held the last in-process storage backend, embedding
  and query executor usage. chat_shell now always retrieves through the Backend
  internal endpoint instead of falling back to an in-process path.
- `RAG_RUNTIME_MODE` with its value parsing, per-operation resolver and the
  per-operation gateway getters; the gateway factory now returns the
  knowledge_runtime gateway.
- `should_fallback_to_local` and every call site; a remote failure is reported
  instead of retried in process.
- The three internal endpoints that existed only for the local runtime
  (all-chunks, purge-knowledge-index, drop-knowledge-index) plus the unused
  gateway protocol `test_connection` and `ConnectionTestRuntimeSpec`.
- Local-only tests, the mode-parsing configuration tests and the deployment
  configuration that still set the removed mode variable.

The external knowledge MCP tool now sends the resolved runtime configs with its
remote query, matching the internal retrieve endpoint and the knowledge search
service, so knowledge_runtime needs no Backend database lookup for it.

Also fixes `test_processing_errors` scanning the whole error payload - including
the generated timestamp - for "0.5", which failed whenever the microseconds
started with 5.
Storage capability lookups (supported types and per-type retrieval methods)
no longer import the concrete backends, and the embedding capability and
dimension-contract modules no longer trigger the package initializers that
pull in llama-index. The storage factory imports a backend class only when it
must build one, so the retriever connection test keeps working.

Importing the retriever-config endpoint, the RAG runtime resolver and the
embedding dimension service now leaves llama_index, pymilvus, qdrant_client
and elasticsearch out of the process.
The Backend only routes retrieval now, so llama-index, pymilvus, the vector
store packages and the RAG document readers no longer belong in its dependency
list. wegent-knowledge-engine and wegent-knowledge-runtime keep declaring them;
the editable knowledge-engine dependency still installs them transitively for
the kernel's own use.

Guard the boundary with two regression tests:

- backend/pyproject.toml must not declare any RAG execution distribution, in
  any dependency group;
- Backend source must not import the execution kernel, and importing app.main
  must still succeed when those packages are unavailable.

Both tests read one shared list so the manifest and import checks stay in sync.
CALLBACK_RESULTS_TOTAL declares a request_type label, but three tests still
passed callback_type, so prometheus_client raised "Incorrect label names" and
the doc converter suite failed before any assertion ran. Pass the label the
counter actually declares.
The Backend stopped using the local RAG execution path, but the editable
wegent-knowledge-engine dependency still installed llama-index, the vector
store adapters, pymilvus, the OpenAI embeddings and the RAG document readers
into its environment and image, and chat_shell declared four of them directly
for a code path it no longer has.

Move the eight RAG execution distributions into a `retrieval` extra on the
kernel, then declare it where retrieval actually runs:

- knowledge_runtime indexes and queries through the kernel, so it depends on
  wegent-knowledge-engine[retrieval].
- knowledge_doc_converter only calls the conversion module (MinerU/S3/zip),
  which touches no retrieval dependency, so it stays on the plain kernel.
- backend and chat_shell declare no RAG execution dependency at all; the
  runtime Docker image installs the extra, the other images stay on the base.

The kernel's own dev environment opt into the extra so its retrieval tests
keep passing. Locks for the four projects are regenerated, and the Backend
lock and environment no longer contain any of the ten distributions.
RAG has no Backend-local execution path, so deployment can no longer treat
the knowledge runtime as optional:

- docker-compose: start knowledge_runtime by default, and make the Backend
  wait for its health; depend on MySQL instead of the Backend to avoid a
  dependency cycle.
- start.sh: always start and stop the Knowledge Runtime together with the
  Backend instead of requiring manual service selection.
- .env.example and user guides: describe the knowledge runtime as a
  required Backend dependency; record that RAG_RUNTIME_MODE is retired and
  that a leftover value is ignored instead of failing startup.
- the four RAG design documents and the external knowledge MCP guide (zh
  and en) now state that local mode was removed and why.
- frontend/e2e README no longer requires the mode variable.
- standalone mode documents that the single-container image does not
  provide knowledge retrieval.
- tests/install asserts the default standard compose starts
  knowledge_runtime and that the Backend depends on it.
The root cause is that standalone mode does not register the RAG endpoints;
the image missing the Knowledge Runtime service is only part of it. The
previous wording implied that bundling that service into the image would
restore knowledge features, which it would not.
The Backend no longer ships the vector store SDKs, so building a storage
backend in-process raised ModuleNotFoundError and the endpoint's fallback
turned every qdrant/elasticsearch/milvus test into success:false.

Forward the storage configuration to knowledge_runtime instead: it builds
the backend with the execution kernel and calls test_connection there.
The Backend keeps HTTP 200 + {success, message} and maps transport
failures to success:false.
Ticket 09: ticket 02 made the Backend put the resolved execution config
(retriever, embedding and their connection credentials) into the remote query
request. The knowledge module design spec rejects that and picks runtime-side
resolution instead: the Backend sends only knowledge base IDs and the retrieval
scope approved for the call, and knowledge_runtime resolves the config of each
knowledge base itself.

Removed:

- `RemoteQueryRequest.knowledge_base_configs` with its uniqueness and coverage
  validator, plus the now-unused `RemoteKnowledgeBaseQueryConfig` protocol type.
- `RagRuntimeResolver.with_query_knowledge_base_configs`, the forced-rag
  resolution inside `build_query_runtime_spec`, the
  `build_query_knowledge_base_configs*` helpers and
  `QueryRuntimeSpec.knowledge_base_configs`. A query runtime spec is
  reference-only, so `build_query_runtime_spec` no longer takes a database
  session and `build_public_query_runtime_spec` keeps only the KB access check
  and the retrieval overrides.
- The config attach points in the internal retrieve endpoint, the knowledge
  search service (including its `_prepare` worker step) and the external
  knowledge MCP tool, and the gateway argument that carried the config onto the
  wire.
- knowledge_runtime's request-carried-config branch: queries always resolve
  through `RuntimeConfigLoader`, and the log source is again
  `request_override` or `database`.
- The external knowledge MCP guide (zh and en) now describes reference mode.

Kept: a rejected direct injection still continues as a remote retrieval, and a
remote failure is reported instead of falling back to local execution. Ticket
08's connection test and the 04/05/06 dependency guards are untouched.

Known residue, recorded for a follow-up: `POST /api/rag/retrieve` still requires
`retriever_ref` and `embedding_model_ref` for client compatibility, but the
knowledge base record now decides which config runs.
Index, delete-document-index, purge-knowledge-index, drop-physical-index and
list-all-chunks still resolved the retriever and embedding configuration in the
Backend, only to discard it: the cross-process requests carry the knowledge base
id, the content reference and the index owner, nothing else.

Remove the resolution from those five paths so knowledge_runtime resolves the
configuration once through its own ConfigResolver:

- IndexRuntimeSpec, DeleteRuntimeSpec, PurgeKnowledgeRuntimeSpec,
  DropKnowledgeIndexRuntimeSpec and ListChunksRuntimeSpec no longer declare
  retriever_config or embedding_model_config.
- RagRuntimeResolver loses _build_resolved_retriever_config,
  _build_resolved_embedding_model_config, _get_model_kind and the credential
  decryption helper, plus the retriever_kinds_service, resolve_model_kind,
  decrypt_api_key and knowledge_engine.embedding.capabilities imports.
- The public purge/drop/list-chunks builders drop their now-unused user_name
  argument; callers and assertions follow.
- IndexRuntimeSpec.splitter_config stays; its normalization belongs to the
  ingestion path and dropping it is a separate contract change.

The prechecks bundled into the removed resolution keep the same status code and
message, now produced remotely: ConfigResolver raises ConfigResolutionError
(config_incomplete / config_not_found), knowledge_runtime maps it through the
ValueError handler to HTTP 400 invalid_request, and RemoteRagGatewayError
restores the same 400 and detail in the Backend. Tests pin the remote producers,
the error mapping and the three public admin routes; new per-path tests build
specs from knowledge bases without retrieval config, so a returning resolution
fails them. The knowledge-base-not-found precondition stays in the Backend and
is covered for delete, purge, drop and list-chunks.

Verification:
- cd backend && uv run pytest tests/services/rag tests/api/endpoints/test_rag_api.py
  tests/services/knowledge tests/integration/test_rag_remote_execution.py
  tests/tasks/test_knowledge_tasks.py tests/services/test_summary_service.py
  tests/mcp_server tests/test_dependency_constraints.py -q -> 1616 passed
- cd backend && uv run pytest -q -> 8372 passed, 1 skipped, 3 failed
  (tests/docker/test_device_executor_home_contract.py fails identically on the
  pristine tip: macOS bash 3.2 rejects ${value,,})
- cd knowledge_runtime && uv run pytest -q -> 121 passed
合并 sunnights/Wegent 的 refactor/rag-remove-local-mode,保留双方历史。
Backend 保留直接注入、身份和授权引用,Runtime Adapter 读取资源,
Module 统一解析执行配置和 QA 规划;删除 local 路径与 Backend 重依赖。
保留显式资源、scope、QA 元数据及 Code Wiki 失败保留记录的契约。

目标:aabc8b8cac4c2e2abcb8b4531730d18777143769
来源:8e25b7e215f084e34115d69e175f85b2f941bf21
merge-base:8fc1e1ba498eb5f0241b0247898e3519e8af3991

验证:Backend 定向集合 277 passed,审查修复后入口 56 passed;
Module 100+5、Chat Shell 77、内核 22(规划增量 17)、Converter 18 passed;
Runtime 148 个用例分阶段修复验证,最终 QA/规划增量 21 passed。
Standards 与 Spec 复审均无未解决项。未运行 E2E 或 ai:verify。
来源桌面 CI 的 fork 产物交接例外保持,不声称 CI 全绿。
@coderabbitai

coderabbitai Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Too many files!

This PR contains 214 files, which is 64 over the limit of 150.

To get a review, reduce the PR to 150 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 3d32f9e8-0315-4b7b-b8cd-365cafd93d3a
📥 Commits

Reviewing files that changed from the base of the PR and between dab9e70 and a7fb314.

⛔ Files ignored due to path filters (5)
  • backend/uv.lock is excluded by !**/*.lock
  • chat_shell/uv.lock is excluded by !**/*.lock
  • knowledge_doc_converter/uv.lock is excluded by !**/*.lock
  • knowledge_engine/uv.lock is excluded by !**/*.lock
  • knowledge_runtime/uv.lock is excluded by !**/*.lock
📒 Files selected for processing (214)
  • .env.example
  • .github/scripts/classify-ci-changes.sh
  • .github/workflows/e2e-tests.yml
  • .github/workflows/test.yml
  • backend-rs/src/knowledge_documents_list/splitter_config.rs
  • backend-rs/src/tables.rs
  • backend/app/api/endpoints/adapter/retrievers.py
  • backend/app/api/endpoints/internal/conversion_callback.py
  • backend/app/api/endpoints/internal/rag.py
  • backend/app/api/endpoints/knowledge.py
  • backend/app/api/endpoints/knowledge_open.py
  • backend/app/api/endpoints/rag.py
  • backend/app/core/config.py
  • backend/app/mcp_server/tools/knowledge_external.py
  • backend/app/schemas/kind.py
  • backend/app/schemas/knowledge.py
  • backend/app/schemas/rag.py
  • backend/app/services/adapters/retriever_kinds.py
  • backend/app/services/knowledge/code_wiki/side_effects.py
  • backend/app/services/knowledge/index_state_machine.py
  • backend/app/services/knowledge/indexing.py
  • backend/app/services/knowledge/knowledge_service.py
  • backend/app/services/knowledge/knowledge_transfer.py
  • backend/app/services/knowledge/orchestrator.py
  • backend/app/services/knowledge/retrieval_profile.py
  • backend/app/services/knowledge/retrieval_resource_resolver.py
  • backend/app/services/knowledge/search_execution.py
  • backend/app/services/knowledge/splitter_config.py
  • backend/app/services/rag/README.md
  • backend/app/services/rag/direct_injection.py
  • backend/app/services/rag/embedding/__init__.py
  • backend/app/services/rag/embedding/factory.py
  • backend/app/services/rag/gateway.py
  • backend/app/services/rag/gateway_factory.py
  • backend/app/services/rag/local_data_plane/__init__.py
  • backend/app/services/rag/local_data_plane/administration.py
  • backend/app/services/rag/local_data_plane/indexing.py
  • backend/app/services/rag/local_data_plane/retrieval.py
  • backend/app/services/rag/local_gateway.py
  • backend/app/services/rag/remote_gateway.py
  • backend/app/services/rag/retrieval_service.py
  • backend/app/services/rag/runtime_resolver.py
  • backend/app/services/rag/runtime_specs.py
  • backend/app/tasks/knowledge_tasks.py
  • backend/pyproject.toml
  • backend/tests/api/endpoints/internal/test_rag_retrieve_endpoint.py
  • backend/tests/api/endpoints/test_conversion_callback.py
  • backend/tests/api/endpoints/test_knowledge_retrieval_config_api.py
  • backend/tests/api/endpoints/test_rag_api.py
  • backend/tests/api/endpoints/test_retrievers_api.py
  • backend/tests/api/test_internal_rag_scope.py
  • backend/tests/api/test_knowledge_code_wiki.py
  • backend/tests/core/test_config.py
  • backend/tests/e2e/knowledge_remote_index.py
  • backend/tests/e2e/knowledge_remote_index_admin.py
  • backend/tests/e2e/knowledge_remote_index_delete.py
  • backend/tests/e2e/knowledge_remote_index_public.py
  • backend/tests/e2e/knowledge_remote_index_support.py
  • backend/tests/integration/test_rag_remote_execution.py
  • backend/tests/mcp_server/test_knowledge_external.py
  • backend/tests/mcp_server/test_knowledge_search_concurrency.py
  • backend/tests/services/knowledge/fixtures/run_code_wiki_purge.py
  • backend/tests/services/knowledge/fixtures/run_remote_qa_query.py
  • backend/tests/services/knowledge/test_authorized_remote_query.py
  • backend/tests/services/knowledge/test_code_wiki_index_cleanup.py
  • backend/tests/services/knowledge/test_code_wiki_retrieval_profile.py
  • backend/tests/services/knowledge/test_conversion_lifecycle.py
  • backend/tests/services/knowledge/test_document_transfer.py
  • backend/tests/services/knowledge/test_explicit_retrieval_resources.py
  • backend/tests/services/knowledge/test_external_document_import.py
  • backend/tests/services/knowledge/test_external_document_sync.py
  • backend/tests/services/knowledge/test_index_state_machine.py
  • backend/tests/services/knowledge/test_indexing.py
  • backend/tests/services/knowledge/test_knowledge_service.py
  • backend/tests/services/knowledge/test_orchestrator.py
  • backend/tests/services/knowledge/test_plain_document_remote_index.py
  • backend/tests/services/knowledge/test_processing_errors.py
  • backend/tests/services/knowledge/test_qa_index_query.py
  • backend/tests/services/knowledge/test_retrieval_resource_resolver.py
  • backend/tests/services/rag/execution_kernel_dependencies.py
  • backend/tests/services/rag/test_admin_retriever_access.py
  • backend/tests/services/rag/test_capability_import_boundary.py
  • backend/tests/services/rag/test_direct_injection.py
  • backend/tests/services/rag/test_embedding_factory.py
  • backend/tests/services/rag/test_local_data_plane_indexing.py
  • backend/tests/services/rag/test_local_data_plane_retrieval.py
  • backend/tests/services/rag/test_local_gateway.py
  • backend/tests/services/rag/test_remote_gateway.py
  • backend/tests/services/rag/test_resource_authorization.py
  • backend/tests/services/rag/test_retrieval_service.py
  • backend/tests/services/rag/test_runtime_resolver.py
  • backend/tests/services/rag/test_runtime_specs.py
  • backend/tests/services/test_summary_service.py
  • backend/tests/tasks/test_knowledge_tasks.py
  • backend/tests/test_dependency_constraints.py
  • backend/tests/utils/namespace_members.py
  • backend/tests/utils/query_knowledge.py
  • backend/tests/utils/remote_only.py
  • backend/tests/utils/retrieval_resources.py
  • chat_shell/chat_shell/tools/builtin/knowledge_base.py
  • chat_shell/chat_shell/tools/builtin/knowledge_listing.py
  • chat_shell/pyproject.toml
  • chat_shell/tests/test_knowledge_base_scoped_search.py
  • chat_shell/tests/test_knowledge_injection_strategy.py
  • chat_shell/tests/test_knowledge_injection_strategy_clean.py
  • chat_shell/tests/test_knowledge_listing.py
  • docker-compose.yml
  • docker/knowledge_doc_converter/Dockerfile
  • docker/knowledge_runtime/Dockerfile
  • docs/en/wegent/deployment/standalone-mode.md
  • docs/en/wegent/developer-guide/external-knowledge-mcp.md
  • docs/en/wegent/getting-started/installation.md
  • docs/en/wegent/getting-started/quick-start.md
  • docs/en/wegent/user-guide/knowledge/configuring-retrievers.md
  • docs/plans/2026-03-24-rag-service-split-plan.md
  • docs/plans/2026-04-04-rag-service-extraction-implementation-plan.md
  • docs/plans/2026-04-19-knowledge-runtime-architecture.md
  • docs/specs/knowledge/2026-04-03-rag-service-extraction-design.md
  • docs/specs/knowledge/2026-10-03-rag-module-merge-validation.md
  • docs/zh/wegent/deployment/standalone-mode.md
  • docs/zh/wegent/developer-guide/external-knowledge-mcp.md
  • docs/zh/wegent/getting-started/installation.md
  • docs/zh/wegent/getting-started/quick-start.md
  • docs/zh/wegent/user-guide/knowledge/configuring-retrievers.md
  • frontend/.husky/pre-commit
  • frontend/e2e/README.md
  • frontend/e2e/utils/mock-embedding.ts
  • frontend/e2e/utils/mock-mineru.ts
  • frontend/e2e/utils/mock-model-server.ts
  • frontend/src/__tests__/e2e/mock-embedding.test.ts
  • knowledge_doc_converter/knowledge_doc_converter/services/conversion_engine.py
  • knowledge_doc_converter/knowledge_doc_converter/tasks/conversion_task.py
  • knowledge_doc_converter/pyproject.toml
  • knowledge_doc_converter/tests/test_conversion_engine.py
  • knowledge_doc_converter/tests/test_conversion_task.py
  • knowledge_doc_converter/tests/test_metrics.py
  • knowledge_engine/knowledge_engine/embedding/__init__.py
  • knowledge_engine/knowledge_engine/embedding/factory.py
  • knowledge_engine/knowledge_engine/lazy_exports.py
  • knowledge_engine/knowledge_engine/query/executor.py
  • knowledge_engine/knowledge_engine/retrieval/search_hints.py
  • knowledge_engine/knowledge_engine/services/document_service.py
  • knowledge_engine/knowledge_engine/splitter/config.py
  • knowledge_engine/knowledge_engine/storage/__init__.py
  • knowledge_engine/knowledge_engine/storage/base.py
  • knowledge_engine/knowledge_engine/storage/capabilities.py
  • knowledge_engine/knowledge_engine/storage/elasticsearch_backend.py
  • knowledge_engine/knowledge_engine/storage/factory.py
  • knowledge_engine/knowledge_engine/storage/milvus_backend.py
  • knowledge_engine/knowledge_engine/storage/qdrant_backend.py
  • knowledge_engine/pyproject.toml
  • knowledge_engine/tests/retrieval/test_query_planning.py
  • knowledge_engine/tests/storage/test_elasticsearch_backend.py
  • knowledge_engine/tests/storage/test_qdrant_backend.py
  • knowledge_engine/tests/test_capability_import_boundary.py
  • knowledge_engine/tests/test_contract_import_boundary.py
  • knowledge_engine/tests/test_knowledge_module_execution.py
  • knowledge_engine/tests/test_query_executor.py
  • knowledge_engine/tests/test_storage_factory.py
  • knowledge_runtime/.env.example
  • knowledge_runtime/knowledge_runtime/api/endpoints/admin.py
  • knowledge_runtime/knowledge_runtime/models/knowledge_document.py
  • knowledge_runtime/knowledge_runtime/services/admin_executor.py
  • knowledge_runtime/knowledge_runtime/services/config_loader.py
  • knowledge_runtime/knowledge_runtime/services/config_resolver.py
  • knowledge_runtime/knowledge_runtime/services/document_index_adapter.py
  • knowledge_runtime/knowledge_runtime/services/index_executor.py
  • knowledge_runtime/knowledge_runtime/services/query_executor.py
  • knowledge_runtime/knowledge_runtime/services/query_planner.py
  • knowledge_runtime/pyproject.toml
  • knowledge_runtime/start.sh
  • knowledge_runtime/tests/conftest.py
  • knowledge_runtime/tests/test_admin_api.py
  • knowledge_runtime/tests/test_admin_executor.py
  • knowledge_runtime/tests/test_config_loader.py
  • knowledge_runtime/tests/test_config_resolver_builders.py
  • knowledge_runtime/tests/test_config_resolver_helpers.py
  • knowledge_runtime/tests/test_config_resolver_resolve.py
  • knowledge_runtime/tests/test_credential_environment.py
  • knowledge_runtime/tests/test_embedding_category_execution.py
  • knowledge_runtime/tests/test_index_endpoint.py
  • knowledge_runtime/tests/test_index_executor.py
  • knowledge_runtime/tests/test_main_errors.py
  • knowledge_runtime/tests/test_query_executor.py
  • knowledge_runtime/tests/test_remote_qa_query.py
  • shared/db/capability_reference.py
  • shared/knowledge_contracts/__init__.py
  • shared/knowledge_contracts/retrieval_scope.py
  • shared/knowledge_contracts/runtime_config.py
  • shared/knowledge_contracts/search_hints.py
  • shared/knowledge_contracts/splitter_config.py
  • shared/knowledge_module/__init__.py
  • shared/knowledge_module/adapter.py
  • shared/knowledge_module/config.py
  • shared/knowledge_module/documents.py
  • shared/knowledge_module/execution.py
  • shared/knowledge_module/index_state.py
  • shared/knowledge_module/operations.py
  • shared/knowledge_module/query_planning.py
  • shared/models/__init__.py
  • shared/models/knowledge_runtime_protocol.py
  • shared/pyproject.toml
  • shared/tests/test_capability_reference.py
  • shared/tests/test_knowledge_config_prepare.py
  • shared/tests/test_knowledge_document_pipeline.py
  • shared/tests/test_knowledge_execution_config.py
  • shared/tests/test_knowledge_index_state.py
  • shared/tests/test_knowledge_operations.py
  • shared/tests/test_knowledge_query_planning.py
  • shared/tests/test_knowledge_runtime_protocol.py
  • shared/tests/test_model_category.py
  • shared/utils/model_category.py
  • start.sh
  • tests/install/test_install_env.sh

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

来源 refactor/rag-remove-local-mode 已移除 8e25b7e,
最新 HEAD 为 b16227d。
完整撤销该提交的 Executor 代码和新增测试,不重写目标历史,
保留知识 Module、授权及远程 RAG 的全部合并适配。

验证:Executor 与最新来源及 wecode-ai/main 一致;
cargo fmt、1454 项库测试和 Clippy 均通过。
@sunnights sunnights changed the title refactor(knowledge): 提取可复用知识模块并统一远程 RAG 执行 refactor(knowledge): 提取可复用知识模块 Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant