Skip to content

Commit 7e5b2ab

Browse files
committed
feat(vis): read Rust-engine sessions from SQLite (records-based wire view)
vis was stranded after the engine switched to SQLite persistence (no more wire.jsonl). Restore it from the engine's own stores: - server: sqlite-store.ts read-only reader (node:sqlite, zero deps) for sessions.db/agent_tasks.db — list/detail/state/context snapshot/tasks/ cron/agents (flat); KIMI_VIS_SOURCE=legacy|sqlite|auto (auto = engine home readable); routes branch on detail.source - server: sqlite-records.ts projects records rows into the wire view — message.append -> context.append_message (normalized camelCase), tool call/result -> append_loop_event (pairing/hover/duration work as-is), usage/goal mapped, unknown types fall back to generic records; empty records return 200 + warnings instead of 404; context route rebuilds the full timeline (history=full semantics) from message.append - web: unknown-type generic rendering + search-crash fix, sqlite source badge on list/detail (i18n en/zh) - tests: vis/server 165 (incl. records fixture + wire projection), vis/web 20; root vitest 1168 green
1 parent 74768ea commit 7e5b2ab

31 files changed

Lines changed: 2239 additions & 23 deletions

CODEX_MIGRATION_PLAN.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,9 @@
4747
## 1.4 已知缺口(不阻塞主线的记录)
4848

4949
1. ~~**compaction summarizer 双通道**~~**已补(2026-08-10)**`LlmCompactionDelegate`(任意 `LLM` 实现)+ `ensure_compaction_delegate` 宿主分支——无 native LLM 的 host-proxy 会话经 `HostLlmProxy``HostCallbacks::llm_chat`)走宿主通道 compact(原报 `compaction.unable`);单测 + stdio 集成适配(`session_compact_host_proxy_gets_a_summarizer_delegate`
50-
2. ~~**子代理 replay 数据源**~~**已补(2026-08-10 数据源 + 2026-08-11 读取面)**:Task 工具子代理在任务被跟踪时(task_service 存在)经 `run_child_agent_persistent_with_model` 持久化对话到 session store(agent_id = task_id,swarm 同一机制);读取面:`session/list` 支持 `include_subagents`(默认过滤子代理记录,opt-in 全量),`session/get_context` 支持 `include_subagents` → 响应 `subagents` 摘要列表(agent_id/消息数/updated_at,`SubagentSummaryRpc`);单测 + stdio 集成(`session_list_and_context_include_subagents`)。**宿主接线待定**vscode harness 的 `includeSubagents` 尚未透传到 RPC、replay-adapter 消费的是完整 replay 而非摘要(`parentAgentId` 关联引擎未持久化)——vscode 为白名单包,改需单独确认
50+
2. ~~**子代理 replay 数据源**~~**已补(2026-08-10 数据源 + 2026-08-11 读取面 + 2026-08-12 精确配对**:Task 工具子代理在任务被跟踪时(task_service 存在)经 `run_child_agent_persistent_with_model` 持久化对话到 session store(agent_id = task_id,swarm 同一机制);读取面:`session/list` 支持 `include_subagents`(默认过滤子代理记录,opt-in 全量),`session/get_context` 支持 `include_subagents` → 响应 `subagents` 摘要列表(agent_id/消息数/updated_at,`SubagentSummaryRpc`);**配对已精确化(2026-08-12)**:子代理持久化时盖章父 Task/swarm 工具调用 id(`parent_tool_call_id``state_json` 顶层键,旧记录 None 兼容),摘要透出该字段,vscode 配对改为精确匹配 + 未盖章 FIFO 兜底 + 盖章孤儿丢弃;swarm 子代理合成 replay 仍缺失(需 XML 结果解析或 child-context 读 RPC,后续项)
5151
3. **用户真实 config.toml 损坏**`duplicate defaultModel`(defaultModel 与 default_model 并存)导致 Rust TOML 严格解析拒绝整个配置(用户禁止修改真实文件,隔离配置验证绕开;建议用户侧删 camelCase 行)
52+
4. ~~**vis 断代(Rust 引擎 SQLite 持久化后 wire.jsonl 不再写)**~~**已闭环(2026-08-12)**:引擎 `records` 表生产接线(9 类事件:message.append/turn.started·ended/tool.call·result/usage.updated/goal.updated/compaction.started·completed,SessionStore/RecordStore 共享同一 SQLite 连接,集成测试 `session_prompt_writes_wire_records` 锁定);vis/server 新增 SQLite reader(`sqlite-store.ts`,node:sqlite 只读,KIMI_VIS_SOURCE=legacy|sqlite|auto)恢复列表/detail/state/context 快照/tasks/cron/agents 平铺;wire 视图从 records 投影(`sqlite-records.ts`:message.append→context.append_message 等映射复用现有渲染器,空记录 200+warnings);context 路由用 records 重建 full timeline(含被 compaction 丢弃的历史);前端通用兜底 + sqlite 徽标。**剩余**:子代理不写 records(独立会话未接线)、TUI 直连路径(RUN_TURN)无 session_id 不写记录(需协议扩展)
5253

5354
> **2026-08-10 收口复核**:原第 4-6 项已消失——vscode typecheck 全过(`replay-adapter.ts` 错误已随 sdk-local 完成消除);kosong 已退役(flaky 测试随包);发布打包已接入 CI(`_rust-bin-build.yml` + release 注入);**vscode legacy backfill 已闭环**(kimi-runtime.ts 迁移会话读 `kimi_cli_source_path` → 写 `vscode_legacy_approval` metadata → session-runtime 消费,create/resume 全链路完整)。另:apps/kimi-code 的 `@moonshot-ai/kimi-agent` devDep 已删(全仓无真实 import,仅注释引用 wire.gen)。
5455

apps/vis/server/src/config.ts

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,3 +61,26 @@ export function resolveVisAuthToken(host: string = resolveHost()): string | unde
6161
}
6262

6363
export const KIMI_CODE_HOME: string = resolveKimiCodeHome();
64+
65+
// ── session-data source selection (legacy dirs vs Rust engine SQLite) ───────
66+
67+
export type VisSource = 'legacy' | 'sqlite' | 'auto';
68+
69+
/**
70+
* Which on-disk session source vis reads from:
71+
*
72+
* - `legacy` — the retired agent-core directory layout under
73+
* `<home>/sessions/<bucket>/<id>/` (state.json + wire.jsonl).
74+
* - `sqlite` — the Rust engine's `$KIMI_AGENT_HOME/sessions.db` +
75+
* `agent_tasks.db` (read-only).
76+
* - `auto` (default) — SQLite when the engine home's `sessions.db` is
77+
* readable, legacy directories otherwise.
78+
*
79+
* Read live from the env on every call (never cached) so tests and
80+
* long-running processes can switch sources without a restart.
81+
*/
82+
export function resolveVisSource(): VisSource {
83+
const raw = process.env['KIMI_VIS_SOURCE']?.trim().toLowerCase();
84+
if (raw === 'legacy' || raw === 'sqlite') return raw;
85+
return 'auto';
86+
}

apps/vis/server/src/lib/agent-record-types.ts

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,8 @@ export interface SessionDetail {
144144
sessionId: string;
145145
/** Canonical on-disk session directory. Routes derive agent wire paths
146146
* from this rather than the mutable `homedir` field inside `state.json`,
147-
* which can drift after fork/rename. */
147+
* which can drift after fork/rename. Empty for SQLite-sourced sessions
148+
* (no session directory exists). */
148149
sessionDir: string;
149150
workDir: string;
150151
state: unknown; // 原样透传,前端按 state.json 真实形状渲染
@@ -153,6 +154,10 @@ export interface SessionDetail {
153154
imported: boolean;
154155
/** Export/import provenance for imported sessions; null for local ones. */
155156
importMeta: ImportInfo | null;
157+
/** Which on-disk source produced this detail: `'sqlite'` when read from
158+
* the Rust engine's `sessions.db`, absent/`'legacy'` for the directory
159+
* layout. Route handlers switch their readers on this flag. */
160+
source?: 'sqlite' | 'legacy';
156161
}
157162

158163
/** One line of `wire.jsonl` after vis has parsed (and possibly migrated)
@@ -174,11 +179,26 @@ export interface WireEntry {
174179
raw: unknown;
175180
}
176181

182+
/** Parsed wire metadata. Legacy `wire.jsonl` carries only
183+
* `protocolVersion` / `createdAt`; the SQLite records source (Rust engine
184+
* `records` table) additionally reports `source: 'sqlite'` and the
185+
* distinct `record_type` values seen for the session. */
186+
export interface WireMetadata {
187+
protocolVersion: string;
188+
createdAt: number;
189+
/** `'sqlite'` when the wire view was reconstructed from the engine's
190+
* `records` table instead of a legacy `wire.jsonl`. */
191+
source?: 'sqlite';
192+
/** Distinct engine record types (`message.append`, `tool.call`, …) in
193+
* first-seen order; absent for legacy wires. */
194+
recordTypes?: string[];
195+
}
196+
177197
export interface WireResponse {
178198
sessionId: string;
179199
agentId: string;
180200
protocolVersion: string;
181-
metadata: { protocolVersion: string; createdAt: number };
201+
metadata: WireMetadata;
182202
records: readonly WireEntry[];
183203
warnings: string[];
184204
}

apps/vis/server/src/lib/session-store.ts

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import { createInterface } from 'node:readline';
66
import type { SessionSummary, SessionDetail, AgentInfo, SessionHealth, ImportInfo } from './agent-record-types';
77
import { compareAgentIds } from './agent-tree';
88
import { importedDirOf, isImportId, listImportedIds, readImportMeta } from './import-store';
9+
import { isSqliteSourceActive, listSqliteSessions, readSqliteSessionDetail } from './sqlite-store';
910

1011
const SESSION_ID_RE = /^session_[A-Za-z0-9._-]+$/;
1112
const AGENT_ID_RE = /^[A-Za-z0-9._-]+$/;
@@ -38,7 +39,17 @@ interface StateJson {
3839
custom?: Record<string, unknown>;
3940
}
4041

42+
/**
43+
* List sessions, newest first. Source is selected per `KIMI_VIS_SOURCE`:
44+
* `sqlite` (Rust engine `sessions.db`, read-only), `legacy` (directory
45+
* layout), or `auto` (default) — SQLite when the engine home's
46+
* `sessions.db` is readable, legacy directories otherwise. The legacy scan
47+
* below only runs for the directory source.
48+
*/
4149
export async function listSessions(home: string): Promise<SessionSummary[]> {
50+
if (isSqliteSourceActive()) {
51+
return listSqliteSessions();
52+
}
4253
const sessionsDir = join(home, 'sessions');
4354
const buckets = await readdir(sessionsDir, { withFileTypes: true }).catch(() => []);
4455
const index = await readSessionIndex(home);
@@ -69,6 +80,9 @@ export async function listSessions(home: string): Promise<SessionSummary[]> {
6980
}
7081

7182
export async function readSessionDetail(home: string, sessionId: string): Promise<SessionDetail | null> {
83+
if (isSqliteSourceActive()) {
84+
return readSqliteSessionDetail(undefined, sessionId);
85+
}
7286
if (isImportId(sessionId)) return readImportedDetail(home, sessionId);
7387
const sessionDir = await findSessionDir(home, sessionId);
7488
if (sessionDir === null) return null;
@@ -159,7 +173,7 @@ async function discoverAgentsFromDisk(sessionDir: string): Promise<AgentInfo[]>
159173
swarmItem: null,
160174
});
161175
}
162-
return out.sort((a, b) => compareAgentIds(a.agentId, b.agentId));
176+
return out.toSorted((a, b) => compareAgentIds(a.agentId, b.agentId));
163177
}
164178

165179
async function tryReadSummary(
@@ -294,7 +308,7 @@ async function inventoryAgents(sessionDir: string, state: StateJson): Promise<Ag
294308
swarmItem: meta.swarmItem ?? null,
295309
});
296310
}
297-
return result.sort((a, b) => compareAgentIds(a.agentId, b.agentId));
311+
return result.toSorted((a, b) => compareAgentIds(a.agentId, b.agentId));
298312
}
299313

300314
async function readState(sessionDir: string): Promise<StateJson | null> {

0 commit comments

Comments
 (0)