使用批量处理embeddings来加速本地推理效率

[TriDefender]

Problem / Motivation

目前各处有关于embeddin的都是顺序调用，也就是每次只发1个请求。这对于推理来说产生了很高的overhead（对我来说，一个28的batch能有90tps，但是如果一次只发一个就只有6-7tps）主要的瓶颈就是没法并发来增加效率。

Proposed Solution

如果不是代码逻辑上强制要求串行向量化，可以考虑在支持batching的断点上攒一个batch再去call向量化

Alternatives Considered

No response

Area

Embedding

Additional Context

No response

[AliceLJY]

Good point — the serial embedding calls are a known bottleneck, especially for local models where batch throughput is much higher than single-request throughput.

I see you've already opened PR #561 with an implementation. We'll review it there.