Skip to content

Commit 7c341a8

Browse files
committed
CANN: Add support for async operator submission
Submit operators using asynchronous threads to improve performance. Use the environment variable GGML_CANN_ASYNC_MODE to control whether asynchronous submission is enabled. It is disabled by default. Testing shows a 10%–20% performance improvement in scenarios with small parameter sizes, especially in quantized models.
1 parent 0019279 commit 7c341a8

File tree

4 files changed

+650
-357
lines changed

4 files changed

+650
-357
lines changed

0 commit comments

Comments
 (0)