Code

fix cuda graphs for qwen2-vl#2708

Code

Merged

Narsil merged 6 commits intomainhuggingface/text-generation-inference:mainfrom qwen2-vl-support-cuda-graphshuggingface/text-generation-inference:qwen2-vl-support-cuda-graphsCopy head branch name to clipboardNov 1, 2024

+93-18

Commits

Commits on Oct 30, 2024

• feat: support multidimensional position ids on batch to enable cuda graphs on qwen2-vl

  drbh

  committed
  8648212

  Copy full SHA for 8648212

• fix: only check model type if config exists

  drbh

  committed
  7d97ee8

  Copy full SHA for 7d97ee8

Commits on Oct 31, 2024

• fix: adjust sharding and lm head logic

  drbh

  committed
  bfa16a5

  Copy full SHA for bfa16a5

• fix qwen2 failure in intel cpu

  Show description for 17de599
  drbh

  committed
  17de599

  Copy full SHA for 17de599

• fix: return correct shape logits and add streaming test

  drbh

  committed
  e2b394e

  Copy full SHA for e2b394e

• fix: remove unused import and refactor test

  drbh

  committed
  d375e1e

  Copy full SHA for d375e1e