Skip to content

Commit 6cf9967

Browse files
zhaochen20zhaochen20
committed
Update 2025-05-02 04:40:46
1 parent 029be76 commit 6cf9967

File tree

80 files changed

+7913
-7768
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

80 files changed

+7913
-7768
lines changed

‎README.html

+2-2
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@
5252
<link rel="search" title="Search" href="search.html" />
5353
<meta name="viewport" content="width=device-width, initial-scale=1"/>
5454
<meta name="docsearch:language" content="en"/>
55-
<meta name="docbuild:last-update" content="May 01, 2025"/>
55+
<meta name="docbuild:last-update" content="May 02, 2025"/>
5656
</head>
5757

5858

@@ -643,7 +643,7 @@ <h3><strong>Prompt Alignment Example</strong><a class="headerlink" href="#prompt
643643

644644
<div class="footer-item">
645645
<p class="last-updated">
646-
Last updated on May 01, 2025.
646+
Last updated on May 02, 2025.
647647
<br/>
648648
</p>
649649
</div>

‎_sources/backend/function_calling.ipynb

+209-181
Large diffs are not rendered by default.

‎_sources/backend/lora.ipynb

+267-254
Large diffs are not rendered by default.

‎_sources/backend/native_api.ipynb

+320-332
Large diffs are not rendered by default.

‎_sources/backend/offline_engine_api.ipynb

+452-436
Large diffs are not rendered by default.

‎_sources/backend/openai_api_completions.ipynb

+236-219
Large diffs are not rendered by default.

‎_sources/backend/openai_api_embeddings.ipynb

+66-60
Large diffs are not rendered by default.

‎_sources/backend/openai_api_vision.ipynb

+86-101
Large diffs are not rendered by default.

‎_sources/backend/send_request.ipynb

+105-91
Large diffs are not rendered by default.

‎_sources/backend/separate_reasoning.ipynb

+126-133
Large diffs are not rendered by default.

‎_sources/backend/speculative_decoding.ipynb

+271-291
Large diffs are not rendered by default.

‎_sources/backend/structured_outputs.ipynb

+146-139
Large diffs are not rendered by default.

‎_sources/backend/structured_outputs_for_reasoning_models.ipynb

+532-499
Large diffs are not rendered by default.

‎_sources/frontend/frontend.ipynb

+211-217
Large diffs are not rendered by default.

‎_sources/references/deepseek.md

+4-3
Original file line numberDiff line numberDiff line change
@@ -171,10 +171,10 @@ See [Separate Reasoning](https://docs.sglang.ai/backend/separate_reasoning.html)
171171

172172
### Function calling for DeepSeek Models
173173

174-
Add arguments `--tool-call-parser deepseekv3` to enable this feature. For example (running on 1 * H20 node):
174+
Add arguments `--tool-call-parser deepseekv3` and `--chat-template ./examples/chat_template/tool_chat_template_deepseekv3.jinja`(recommended) to enable this feature. For example (running on 1 * H20 node):
175175

176176
```
177-
python3 -m sglang.launch_server --model deepseek-ai/DeepSeek-V3-0324 --tp 8 --port 30000 --host 0.0.0.0 --mem-fraction-static 0.9 --disable-cuda-graph --tool-call-parser deepseekv3
177+
python3 -m sglang.launch_server --model deepseek-ai/DeepSeek-V3-0324 --tp 8 --port 30000 --host 0.0.0.0 --mem-fraction-static 0.9 --disable-cuda-graph --tool-call-parser deepseekv3 --chat-template ./examples/chat_template/tool_chat_template_deepseekv3.jinja
178178
```
179179

180180
Sample Request:
@@ -188,7 +188,7 @@ curl "http://127.0.0.1:30000/v1/chat/completions" \
188188
Expected Response
189189

190190
```
191-
{"id": "62af80528930423a82c806651ec66e7c", "object": "chat.completion", "created": 1744431333, "model": "deepseek-ai/DeepSeek-V3-0324", "choices": [{"index": 0, "message": {"role": "assistant", "content": null, "reasoning_content": null, "tool_calls": [{"id": "0", "type": "function", "function": {"name": "query_weather", "arguments": "{\\"city\\": \\"Guangzhou\\"}"}}]}, "logprobs": null, "finish_reason": "tool_calls", "matched_stop": null}], "usage": {"prompt_tokens": 118, "total_tokens": 140, "completion_tokens": 22, "prompt_tokens_details": null}}
191+
{"id":"6501ef8e2d874006bf555bc80cddc7c5","object":"chat.completion","created":1745993638,"model":"deepseek-ai/DeepSeek-V3-0324","choices":[{"index":0,"message":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":[{"id":"0","index":null,"type":"function","function":{"name":"query_weather","arguments":"{\"city\": \"Qingdao\"}"}}]},"logprobs":null,"finish_reason":"tool_calls","matched_stop":null}],"usage":{"prompt_tokens":116,"total_tokens":138,"completion_tokens":22,"prompt_tokens_details":null}}
192192
193193
```
194194
Sample Streaming Request:
@@ -215,6 +215,7 @@ The client needs to concatenate all arguments fragments to reconstruct the compl
215215
```
216216
Important Notes:
217217
1. Use a lower `"temperature"` value for better results.
218+
2. To receive more consistent tool call results, it is recommended to use `--chat-template examples/chat_template/tool_chat_template_deepseekv3.jinja`. It provides an improved unified prompt.
218219

219220

220221

‎backend/attention_backend.html

+2-2
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@
5454
<link rel="prev" title="Hyperparameter Tuning" href="hyperparameter_tuning.html" />
5555
<meta name="viewport" content="width=device-width, initial-scale=1"/>
5656
<meta name="docsearch:language" content="en"/>
57-
<meta name="docbuild:last-update" content="May 01, 2025"/>
57+
<meta name="docbuild:last-update" content="May 02, 2025"/>
5858
</head>
5959

6060

@@ -625,7 +625,7 @@ <h3>Launch command for different attention backends.<a class="headerlink" href="
625625

626626
<div class="footer-item">
627627
<p class="last-updated">
628-
Last updated on May 01, 2025.
628+
Last updated on May 02, 2025.
629629
<br/>
630630
</p>
631631
</div>

‎backend/custom_chat_template.html

+2-2
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@
5454
<link rel="prev" title="Structured Outputs For Reasoning Models" href="structured_outputs_for_reasoning_models.html" />
5555
<meta name="viewport" content="width=device-width, initial-scale=1"/>
5656
<meta name="docsearch:language" content="en"/>
57-
<meta name="docbuild:last-update" content="May 01, 2025"/>
57+
<meta name="docbuild:last-update" content="May 02, 2025"/>
5858
</head>
5959

6060

@@ -574,7 +574,7 @@ <h2>Jinja Format<a class="headerlink" href="#jinja-format" title="Link to this h
574574

575575
<div class="footer-item">
576576
<p class="last-updated">
577-
Last updated on May 01, 2025.
577+
Last updated on May 02, 2025.
578578
<br/>
579579
</p>
580580
</div>

‎backend/function_calling.html

+118-102
Large diffs are not rendered by default.

‎backend/function_calling.ipynb

+209-181
Large diffs are not rendered by default.

‎backend/hyperparameter_tuning.html

+2-2
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@
5454
<link rel="prev" title="Sampling Parameters" href="sampling_params.html" />
5555
<meta name="viewport" content="width=device-width, initial-scale=1"/>
5656
<meta name="docsearch:language" content="en"/>
57-
<meta name="docbuild:last-update" content="May 01, 2025"/>
57+
<meta name="docbuild:last-update" content="May 02, 2025"/>
5858
</head>
5959

6060

@@ -610,7 +610,7 @@ <h2>Tune <code class="docutils literal notranslate"><span class="pre">--schedule
610610

611611
<div class="footer-item">
612612
<p class="last-updated">
613-
Last updated on May 01, 2025.
613+
Last updated on May 02, 2025.
614614
<br/>
615615
</p>
616616
</div>

0 commit comments

Comments
 (0)