Code

ROCm and sliding windows fixes#2033

Code

Merged

fxmarty merged 10 commits intomainhuggingface/text-generation-inference:mainfrom rocm-fixeshuggingface/text-generation-inference:rocm-fixesCopy head branch name to clipboardJun 10, 2024

+36-24

Commits

Commits on Jun 6, 2024

• update vllm commit & fix models using sliding window

  fxmarty

  committed
  0d3cc03

  Copy full SHA for 0d3cc03

• update

  fxmarty

  committed
  e9b9a96

  Copy full SHA for e9b9a96

• update commit

  fxmarty

  committed
  35d1946

  Copy full SHA for 35d1946

• fix bug where tunableop is bound to cuda graph even when cuda graph are disabled

  fxmarty

  committed
  c36c7ec

  Copy full SHA for c36c7ec

• enable tunableop by default

  fxmarty

  committed
  0d9b2f2

  Copy full SHA for 0d9b2f2

Commits on Jun 7, 2024

• fix sliding window

  fxmarty

  committed
  4220423

  Copy full SHA for 4220423

• address review

  fxmarty

  committed
  b884d2b

  Copy full SHA for b884d2b

• dead code

  fxmarty

  committed
  fb5487d

  Copy full SHA for fb5487d

• precise comment

  fxmarty

  committed
  b8ac9ba

  Copy full SHA for b8ac9ba

• is it flaky?

  fxmarty

  committed
  979b670

  Copy full SHA for 979b670