Test PR，忽略#671
Closed
guoyi8 wants to merge 8 commits intoxerrors:mainfrom
Closed
Conversation
added 8 commits
May 1, 2026 16:45
- 实现了 tiktoken 离线补丁，避免运行时从互联网下载 BPE 文件 - 添加了字符型分词器作为 tiktoken 不可用时的备选方案 - 为 LightRAG 实例注入离线分词器支持 - 适配推理模型输出格式，提取被包装在推理文本中的 JSON 内容 - 解决 LightRAG 索引参数配置中 chunk_size 和 chunk_overlap 参数传递问题 - 统一文件表组件中块大小重叠参数的显示逻辑
naive_merge 不保证输出 chunk 在 token 上限内，当单行内容超过 chunk_token_num 时会产生超大 chunk，导致 LightRAG 报错 "Chunk token length 3140 exceeds chunk_token_size 1200"。 - nlp.py: 新增 hard_split_by_token_limit 公共硬切分函数 - general.py: 新增 _ensure_chunk_token_limit 兜底保护 - laws.py: 删除本地重复函数，改用 nlp 版本（DRY）
# Conflicts: # docker-compose.yml
V2 模型 spec 格式为 provider_id:model_id（冒号分隔）， 但 handleLLMSelect 使用斜杠拆分，导致 provider 和 model_name 均为空。 新增 model_spec 字段直接存储完整 spec，抽取 parseModelSpec/buildDisplaySpec 工具函数统一处理 V1/V2 两种格式，消除 3 处重复逻辑。
Author
|
误操作，关闭后重新提交 |
Author
|
This pull request was opened in error and contained unintended internal configuration details. The correct and sanitized change is available in #672. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan