Commit c39c9e3
WIP: DP4AMatMul fix matmul for subgoup size 64 GPUs (#23637)
### Description
This change moves away from using subgroup ops for quantization. This is
because on AMD GPUs subgroup size is 64 and that is not handled in our
quantization function, resulting in garbage output. Implementing
subgroup size 64 quantization requires changing the workgroup size and
then implementing support for subgroup size 128 becomes a challenge.
With the new implementation perf on intel ALD remains about the same
4.36s for 1000K prefill.
Tests for this change are present here
https://github.com/microsoft/onnxruntime/blob/e66650350b85cb5e3a408f6576fe6a7f4f4ddebc/onnxruntime/test/contrib_ops/matmul_4bits_test.cc
However, to trigger the current issue they must be run on a GPU with
subgroup size 64.1 parent 755f28a commit c39c9e3
File tree
1 file changed
+13
-32
lines changed- onnxruntime/contrib_ops/webgpu/quantization
1 file changed
+13
-32
lines changedLines changed: 13 additions & 32 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
535 | 535 | | |
536 | 536 | | |
537 | 537 | | |
538 | | - | |
539 | | - | |
540 | | - | |
541 | | - | |
542 | 538 | | |
543 | | - | |
544 | | - | |
545 | | - | |
546 | | - | |
547 | | - | |
548 | | - | |
549 | | - | |
550 | | - | |
551 | | - | |
552 | | - | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
553 | 542 | | |
554 | | - | |
555 | | - | |
556 | | - | |
| 543 | + | |
| 544 | + | |
557 | 545 | | |
558 | | - | |
559 | | - | |
560 | | - | |
561 | | - | |
562 | | - | |
563 | | - | |
564 | | - | |
565 | | - | |
566 | | - | |
567 | | - | |
568 | | - | |
569 | | - | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
570 | 550 | | |
571 | | - | |
572 | | - | |
| 551 | + | |
573 | 552 | | |
| 553 | + | |
| 554 | + | |
574 | 555 | | |
575 | 556 | | |
576 | 557 | | |
| |||
838 | 819 | | |
839 | 820 | | |
840 | 821 | | |
841 | | - | |
| 822 | + | |
842 | 823 | | |
843 | 824 | | |
844 | 825 | | |
| |||
0 commit comments