Can I directly use vllm for inference？

[dingjingzhen]

Great job! Are there any plans to integrate this job into the vllm?

[Zhichenzzz]

Great job! Are there any plans to integrate this job into the vllm?

Thank you for your interest! We don’t support vllm serve yet, but contributions are very welcome. Please feel free to open a PR.

[terarachang]

Hi, thanks for the great work! Does eval/reasoning_tasks/eval_hf.py use Tilelang or Triton kernel?

[Zhichenzzz]

Hi, thanks for the great work! Does eval/reasoning_tasks/eval_hf.py use Tilelang or Triton kernel?

Thank you for your interest! We are using both Triton and Tilelang kernels to improve the efficiency. BTW, the tilelang community supported our seer_attn kernel few months ago, please referring to https://github.com/tile-ai/tilelang/tree/main/examples/seer_attention.