Code

Dev/split#1

Code

Draft

tianmu-li wants to merge 19 commits intomlperf_featurestianmu-li/vllm-fork:mlperf_featuresfrom dev/splittianmu-li/vllm-fork:dev/splitCopy head branch name to clipboard

+401-99

Commits

Commits on Dec 16, 2024

• add const_norm option

  Show description for 2576619
  xuechendi

  committed
  2576619

  Copy full SHA for 2576619

• Enable TP splitting on seq_len

  Show description for d6bdc90
  

  jikunshang

  authored and

  xuechendi

  committed
  d6bdc90

  Copy full SHA for d6bdc90

• make both TP overlap splitting by batch and seq work together (HabanaAI#630)

  Show description for a23e1a1
  

  xuechendi

  and

  jikunshang

  authored
  a23e1a1

  Copy full SHA for a23e1a1

Commits on Dec 17, 2024

• fix for FP8 (HabanaAI#639)

  Show description for e2ea481
  xuechendi

  authored
  e2ea481

  Copy full SHA for e2ea481

Commits on Dec 19, 2024

• update vllm-hpu-extention to const_pipeline_pa (HabanaAI#655)

  Show description for 291a1c4
  xuechendi

  committed
  291a1c4

  Copy full SHA for 291a1c4

Commits on Dec 20, 2024

• use env var to control

  

  jikunshang

  authored and

  tianmu-li

  committed
  2f7adb5

  Copy full SHA for 2f7adb5

• update strategy only split when tensor is big

  Show description for c73ff19
  

  xuechendi

  authored and

  tianmu-li

  committed
  c73ff19

  Copy full SHA for c73ff19

• [SW-209737] prepare sin/cos buffers for rope outside model forward

  

  bgoldberg-habana

  authored and

  tianmu-li

  committed
  94897d0

  Copy full SHA for 94897d0

• Split qkv

  

  nirda7

  authored and

  tianmu-li

  committed
  8bf84bc

  Copy full SHA for 8bf84bc

• WIP

  tianmu-li

  committed
  388388b

  Copy full SHA for 388388b

• Resolve rebase issues

  tianmu-li

  committed
  d4eb9a7

  Copy full SHA for d4eb9a7

• Remove print

  tianmu-li

  committed
  217c58a

  Copy full SHA for 217c58a

• Remove old split_qk_v implementation

  tianmu-li

  committed
  a76a6b8

  Copy full SHA for a76a6b8

• Remove additional prints

  tianmu-li

  committed
  e95ea97

  Copy full SHA for e95ea97

• Remove old code

  tianmu-li

  committed
  f6bcffb

  Copy full SHA for f6bcffb

• Cleanup

  tianmu-li

  committed
  4d508b8

  Copy full SHA for 4d508b8

• Remove unneeded changes

  tianmu-li

  committed
  e011f91

  Copy full SHA for e011f91

• Typo

  tianmu-li

  committed
  69ffcb2

  Copy full SHA for 69ffcb2

Commits on Dec 25, 2024

• seq_len=512 prefill w/a

  tianmu-li

  committed
  b19dbf8

  Copy full SHA for b19dbf8