initial support blackwell

[johnnynunez]

10.0 blackwell b100/b200
12.0 blackwell rtx50

[yzh119]

Hi @johnnynunez , thanks for bringing this up! Could we hold this PR and wait for the official release of torch 2.6 and blackwell software stack?

[johnnynunez]

Hi @johnnynunez , thanks for bringing this up! Could be hold this PR and wait for the official release of torch 2.6 and blackwell software stack?

Yeah for sure! I put all codegen blackwell family on pytorch.
Also you have references here:
NVIDIA/cccl#3493

[johnnynunez]

Hi @johnnynunez , thanks for bringing this up! Could we hold this PR and wait for the official release of torch 2.6 and blackwell software stack?

FYI: pytorch/pytorch#145436

[johnnynunez]

FYI: https://docs.nvidia.com/cuda/pdf/ptx_isa_8.7.pdf

[yzh119]

FYI: https://docs.nvidia.com/cuda/pdf/ptx_isa_8.7.pdf

This is huge!

[johnnynunez]

@yzh119 can you merge?

[zhyncs]

@yzh119 can you merge?

@johnnynunez remind #747 (comment)

[johnnynunez]

well, sure... pytorch is coming this week : M6: Release Day (1/29/25)

[ghostplant]

Is there a prebuilt that can work for B200?

[YavorGIvanov]

What performance improvement should we expect out of the box on B200 compared to H100 SXM5 for different size models ? 8B, 70B, 400B. I expected to get some benefit even for 8B (e.g. 30% for low batch sizes), but I am getting no benefit using Llama 8B.

Also is there any planned on in-progress work on flashinfer utilizing B200 specific capabilities (e.g. Tensor Memory Accelerator) ?