-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
检查清单
- 1. 我已经搜索过相关问题,但未能获得预期的帮助
- 2. 该问题在最新版本中尚未修复
- 3. 请注意,如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例,我们将难以复现和定位问题,降低获得反馈的可能性
- 4. 如果您提出的不是bug而是问题,请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
- 5. 为方便社区交流,我将使用中文/英文或附上中文/英文翻译(如使用其他语言)。未附带翻译的非中文/英语内容可能会被关闭
问题描述
运行 KT 0.3.2 版本时出错,请帮忙看一下什么原因
复现步骤
(kt2) xstdsr1@xstdsr1-System-Product-Name:~/ktransformers$ python ktransformers/server/main.py \
--port 10002
--host 10.97.14.98
--model_path /home/xstdsr1/DeepSeek-R1
--gguf_path /home/xstdsr1/DeepSeek-R1-GGUF/DeepSeek-R1-UD-IQ1_S
--optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml
--max_new_tokens 1024
--cpu_infer 14
--cache_lens 32768
--chunk_size 256
--max_batch_size 4
--backend_type balance_serve
--use_cuda_graph
--force_think
Traceback (most recent call last):
File "/home/xstdsr1/ktransformers/ktransformers/server/main.py", line 10, in
from ktransformers.server.args import ArgumentParser
File "/home/xstdsr1/.conda/envs/kt2/lib/python3.11/site-packages/ktransformers/server/args.py", line 2, in
from ktransformers.server.backend.args import ConfigArgs, default_args
File "/home/xstdsr1/.conda/envs/kt2/lib/python3.11/site-packages/ktransformers/server/backend/args.py", line 79, in
cfg = Config()
^^^^^^^^
File "/home/xstdsr1/.conda/envs/kt2/lib/python3.11/site-packages/ktransformers/server/config/singleton.py", line 26, in call
cls._instances[cls] = super(Singleton, cls).call(*args, **kwds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xstdsr1/.conda/envs/kt2/lib/python3.11/site-packages/ktransformers/server/config/config.py", line 206, in init
self.kvc2_disk_path = cfg["kvc2"]["disk_path"]
~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'disk_path'
(kt2) xstdsr1@xstdsr1-System-Product-Name:~/ktransformers$
环境信息
Version: 0.3.2+cu128torch27avx2
NVIDIA-SMI 570.169 Driver Version: 570.169 CUDA Version: 12.8
Ubuntu 2404
i9/192G内存/4090-24G