(reference-only) Multi backend refactor -> main (full diff of all already merged PRs) #1220

Titus-von-Koeller · 2024-05-25T16:46:27Z

This PR to main serves the purpose to keep an overview of all the extensive changes that have been introduced to multi-backend-refactor to the iterative PRs around this topic.

This will not be merged into master and instead the changes will be ported to the new custom_ops API that's already merged to main. Future backend PRs should be addressed directly at main.

This reverts commit b7ca5cf.

…bytes into fix_igemmlt_int

Enable igemmlt int test on rocm

IFU master 2024 01 24

…OCm/hipBLASLt@3aad0d8

…tion

…yproject.toml

github-actions · 2025-02-10T20:14:29Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

* fix xpu dtypoe Signed-off-by: jiqing-feng <[email protected]> * fix nf4 dtype Signed-off-by: jiqing-feng <[email protected]> --------- Signed-off-by: jiqing-feng <[email protected]>

* fix version Signed-off-by: jiqing-feng <[email protected]> * fix setup version Signed-off-by: jiqing-feng <[email protected]> --------- Signed-off-by: jiqing-feng <[email protected]>

* enable benchmark script Signed-off-by: jiqing-feng <[email protected]> * Small fixes to non_cuda_backends.mdx --------- Signed-off-by: jiqing-feng <[email protected]> Co-authored-by: Titus <[email protected]>

Signed-off-by: jiqing-feng <[email protected]>

* enable quant storage Signed-off-by: jiqing-feng <[email protected]> * fix to numpy Signed-off-by: jiqing-feng <[email protected]> --------- Signed-off-by: jiqing-feng <[email protected]>

Signed-off-by: jiqing-feng <[email protected]>

* fix 4bit XPU dequant 4bit Signed-off-by: jiqing-feng <[email protected]> * fix default value Signed-off-by: jiqing-feng <[email protected]> * fix ipex linear set Signed-off-by: jiqing-feng <[email protected]> * fix ipex linear set to false when calling state dict Signed-off-by: jiqing-feng <[email protected]> * fix Int8Param device patch Signed-off-by: jiqing-feng <[email protected]> --------- Signed-off-by: jiqing-feng <[email protected]>

* fix xpu to cpu Signed-off-by: jiqing-feng <[email protected]> * fix xpu cpu data device Signed-off-by: jiqing-feng <[email protected]> --------- Signed-off-by: jiqing-feng <[email protected]>

Signed-off-by: jiqing-feng <[email protected]>

* fix intel cpu/xpu warning Signed-off-by: jiqing-feng <[email protected]> * fix error log Signed-off-by: jiqing-feng <[email protected]> * fix lib Signed-off-by: jiqing-feng <[email protected]> * rm return Nonr Signed-off-by: jiqing-feng <[email protected]> * error log only without ipex Signed-off-by: jiqing-feng <[email protected]> * fix import eerror Signed-off-by: jiqing-feng <[email protected]> * fix format Signed-off-by: jiqing-feng <[email protected]> --------- Signed-off-by: jiqing-feng <[email protected]>

anadon · 2025-04-14T20:23:32Z

Could someone post about the status/progress of this PR? Like a list of checked and unchecked known items to do.

* enable xpu 8bit optim * add deqaunt_blockwise * dequantize_blockwise * add bakcend synchronize * refine code * ipex dep * ipex dep too * ipex version check --------- Co-authored-by: jiqing-feng <[email protected]>

Authored by: Chetan Kumar Verma <[email protected]> Co-authored-by: Ruheena Suhani Shaik <[email protected]> Co-authored-by: Bhargav Eede <[email protected]> Co-authored-by: Vivek Goel <[email protected]> Co-authored-by: Ruheena Suhani Shaik <[email protected]>

Titus-von-Koeller · 2025-04-15T17:24:31Z

Please see this short update about the multi-backend refactor #1596.

cc @anadon

Signed-off-by: jiqing-feng <[email protected]>

Titus-von-Koeller · 2025-08-22T15:26:31Z

We're closing this PR, because the functionality contained therein has been merged to main following the torch.library API as a integration mechanism. Therefore, this branch here will remain undeleted for a while, in case anyone still wants to reference it and we'll leave the build up, until the official release from main.

In the mean-time the already merged code can be used by downloading the wheel from CI:
https://github.com/bitsandbytes-foundation/bitsandbytes/releases/tag/continuous-release_main

jianan-gu and others added 30 commits December 4, 2023 20:23

minor fix

59facc8

final refinement

066d0dc

Enable col to row transformation

657ca4b

Add make functions for row to col transformation

a390e0c

Update get_transform_buffer for row to col in HIP

99ad6b5

Update igemmlt for col format

039b808

Unskip test_igemmlt_int on ROCm

1a052ee

Update igemmlt_int test for col inputs

b7ca5cf

Skip transpose igemmlt test on ROCm

a2cd90d

Revert "Update igemmlt_int test for col inputs"

5b6c5ac

This reverts commit b7ca5cf.

Return nvidia_transform from transform for HIP

218bf66

Fix syntax error

8bb5c2f

Add comment for shape change

eb2edf7

Enable nvidia_transform tests

a38ea0f

Merge branch 'fix_igemmlt_int' of https://github.com/pnunna93/bitsand…

fbacd7a

…bytes into fix_igemmlt_int

Enable igemmlt_half tests

67c383b

Revert col32 check in nvidia_transform test

42b860f

Merge pull request #3 from pnunna93/fix_igemmlt_int

7198d6b

Enable igemmlt int test on rocm

Merge remote-tracking branch 'upstream/main' into IFU-master-2024-01-24

b1d484a

Update README.md

c36085d

Update hip files with upstream changes

0e91e48

Skip failing tests for now

1295d53

Merge pull request #4 from ROCm/IFU-master-2024-01-24

48b7fa9

IFU master 2024 01 24

ops.hip: adapt to enum naming changes in ROCm/hipBLASLt@95131d6 and R…

f1a0b8b

…OCm/hipBLASLt@3aad0d8

Merge remote-tracking branch 'main/main' into upstream_device_abstrac…

e34c30e

…tion

refine backend register with base-backend

cebd83c

Merge remote-tracking branch 'main/main' into upstream_device_abstrac…

e0f2e18

…tion

minor clean format

d20c017

fix wmma api parity

a84c369

hipify wmma datatype

b044010

matthewdouglas added 2 commits February 10, 2025 14:17

Fix

d3ead1e

Build: use setuptools_scm for dynamic versioning compatibility with p…

6c4d878

…yproject.toml

matthewdouglas and others added 5 commits February 10, 2025 15:40

Update wheel build

2d06869

Add rocm6.3.2

7c917b0

setuptools_scm update

fdbbfb6

fix xpu woq linear dtype (#1506)

89373b8

* fix xpu dtypoe Signed-off-by: jiqing-feng <[email protected]> * fix nf4 dtype Signed-off-by: jiqing-feng <[email protected]> --------- Signed-off-by: jiqing-feng <[email protected]>

fix version (#1532)

2640753

* fix version Signed-off-by: jiqing-feng <[email protected]> * fix setup version Signed-off-by: jiqing-feng <[email protected]> --------- Signed-off-by: jiqing-feng <[email protected]>

matthewdouglas added the Cross Platform label Feb 28, 2025

jiqing-feng and others added 9 commits March 4, 2025 20:39

enable benchmark script (#1554)

c66e137

* enable benchmark script Signed-off-by: jiqing-feng <[email protected]> * Small fixes to non_cuda_backends.mdx --------- Signed-off-by: jiqing-feng <[email protected]> Co-authored-by: Titus <[email protected]>

update comments (#1562)

83c147d

Signed-off-by: jiqing-feng <[email protected]>

enable quant storage (#1563)

0cd87aa

* enable quant storage Signed-off-by: jiqing-feng <[email protected]> * fix to numpy Signed-off-by: jiqing-feng <[email protected]> --------- Signed-off-by: jiqing-feng <[email protected]>

fix meta device dispatch (#1564)

2354bdd

Signed-off-by: jiqing-feng <[email protected]>

Enable XPU int matmul (#1547)

249a3cd

Signed-off-by: jiqing-feng <[email protected]>

Fix xpu to cpu (#1570)

d3658c5

* fix xpu to cpu Signed-off-by: jiqing-feng <[email protected]> * fix xpu cpu data device Signed-off-by: jiqing-feng <[email protected]> --------- Signed-off-by: jiqing-feng <[email protected]>

fix double compress 8bit precision (#1582)

d180d8e

Signed-off-by: jiqing-feng <[email protected]>

Liangliang-Ma and others added 2 commits April 15, 2025 11:13

XPU backend support 8bit optimizer (#1565)

5c48b33

* enable xpu 8bit optim * add deqaunt_blockwise * dequantize_blockwise * add bakcend synchronize * refine code * ipex dep * ipex dep too * ipex version check --------- Co-authored-by: jiqing-feng <[email protected]>

fix log (#1604)

5027e64

Signed-off-by: jiqing-feng <[email protected]>

iwr-redmond mentioned this pull request Apr 23, 2025

[Feature Request] Supporting quantization for DiT models (Flux, SD3.5) Teriks/dgenerate#14

Closed

jiqing-feng and others added 2 commits April 29, 2025 09:31

fix xpu ipex linear in torch2.7 (#1618)

263179a

Signed-off-by: jiqing-feng <[email protected]>

update compute_type_is_set attr (#1623)

5e267f5

Titus-von-Koeller changed the title ~~(WIP) Multi backend refactor -> main (full diff of all already merged PRs)~~ (reference-only) Multi backend refactor -> main (full diff of all already merged PRs) May 8, 2025

supports HPU double quant (#1630)

c3eac42

Titus-von-Koeller closed this Aug 22, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

(reference-only) Multi backend refactor -> main (full diff of all already merged PRs) #1220

(reference-only) Multi backend refactor -> main (full diff of all already merged PRs) #1220

Uh oh!

Titus-von-Koeller commented May 25, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Feb 10, 2025

Uh oh!

anadon commented Apr 14, 2025

Uh oh!

Titus-von-Koeller commented Apr 15, 2025

Uh oh!

Titus-von-Koeller commented Aug 22, 2025

Uh oh!

Uh oh!

Uh oh!

(reference-only) Multi backend refactor -> main (full diff of all already merged PRs) #1220

(reference-only) Multi backend refactor -> main (full diff of all already merged PRs) #1220

Uh oh!

Conversation

Titus-von-Koeller commented May 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 10, 2025

Uh oh!

anadon commented Apr 14, 2025

Uh oh!

Titus-von-Koeller commented Apr 15, 2025

Uh oh!

Titus-von-Koeller commented Aug 22, 2025

Uh oh!

Uh oh!

Titus-von-Koeller commented May 25, 2024 •

edited

Loading