Code

Jiayi dev v2#11

Code

Merged

KuntaiDu merged 24 commits intokuntai-disagg-refactorKuntaiDu/vllm:kuntai-disagg-refactorfrom jiayi-dev-v2KuntaiDu/vllm:jiayi-dev-v2Copy head branch name to clipboardSep 16, 2024

+29-13

Commits

Commits on Sep 15, 2024

• use new round robin proxy in performance benchmark

  ApostaC

  committed
  01b2fd3

  Copy full SHA for 01b2fd3

• update

  ApostaC

  committed
  54bd11f

  Copy full SHA for 54bd11f

• update benchmarking script

  ApostaC

  committed
  b19f346

  Copy full SHA for b19f346

• revert changes in model_runner.py --- no change needed for disagg prefill

  ApostaC

  committed
  cb7ff06

  Copy full SHA for cb7ff06

• no I was wrong

  ApostaC

  committed
  dd8c86d

  Copy full SHA for dd8c86d

• update benchmark

  ApostaC

  committed
  4e8043c

  Copy full SHA for 4e8043c

• remove sonnet 4x --- it can be automatically generated via benchmarking script

  ApostaC

  committed
  b51f891

  Copy full SHA for b51f891

• revert change in flash attn and flash infer to clean up the diff

  ApostaC

  committed
  168452f

  Copy full SHA for 168452f

• update the example

  ApostaC

  committed
  784d905

  Copy full SHA for 784d905

• make format checker happy

  ApostaC

  committed
  17d2505

  Copy full SHA for 17d2505

• resolve circular import

  ApostaC

  committed
  36a382c

  Copy full SHA for 36a382c

• fix redundant import

  ApostaC

  committed
  a0867dd

  Copy full SHA for a0867dd

• rename to a shorter name

  ApostaC

  committed
  7f90903

  Copy full SHA for 7f90903

Commits on Sep 16, 2024

• remove unnecessary file

  ApostaC

  committed
  5ca22fb

  Copy full SHA for 5ca22fb

• update kv transfer test

  ApostaC

  committed
  073642b

  Copy full SHA for 073642b

• update tests

  ApostaC

  committed
  70d6571

  Copy full SHA for 70d6571

• make fmt checker happy

  ApostaC

  committed
  4d6b00a

  Copy full SHA for 4d6b00a

• constraint the model length

  ApostaC

  committed
  7c13e03

  Copy full SHA for 7c13e03

• adjust path

  ApostaC

  committed
  cf5b84c

  Copy full SHA for cf5b84c

• add disagg prefill test to test pipeline

  ApostaC

  committed
  eb751d6

  Copy full SHA for eb751d6

• Merge pull request #9 from KuntaiDu/kuntai-disagg-refactor

  Show description for f101b40
  KuntaiDu

  authored
  f101b40

  Copy full SHA for f101b40

• Merge pull request #10 from KuntaiDu/kuntai-disagg-refactor

  Show description for 0d81aaf
  KuntaiDu

  authored
  0d81aaf

  Copy full SHA for 0d81aaf

• bugfix

  YaoJiayi

  committed
  0df7566

  Copy full SHA for 0df7566

• bugfix

  YaoJiayi

  committed
  73c1683

  Copy full SHA for 73c1683