Skip to content

[Kernel]Remove recast iter from sliding window attention kernel. - #997

Merged
Phil-amd merged 15 commits into
ROCm:mainfrom
amd-nprotaso:main
Aug 11, 2026
Merged

[Kernel]Remove recast iter from sliding window attention kernel.#997
Phil-amd merged 15 commits into
ROCm:mainfrom
amd-nprotaso:main

Conversation

@amd-nprotaso

Copy link
Copy Markdown
Contributor

#933 was merged, now we can use add_offset without recast_iter, with equivalent performance:

B seq win prev ms cur ms prev/cur prev TF cur TF
1 1024 (512,0) 0.0297 0.0300 0.99x 217.6 215.3
1 2048 (512,0) 0.0353 0.0350 1.01x 427.3 430.6
1 2048 (1024,0) 0.0469 0.0475 0.99x 549.9 543.0
1 4096 (512,0) 0.0734 0.0697 1.05x 440.0 463.3
1 4096 (1024,0) 0.1049 0.1012 1.04x 573.9 594.5
1 8192 (512,0) 0.1581 0.1616 0.98x 422.0 412.7
1 8192 (1024,0) 0.2325 0.2398 0.97x 554.8 538.0
1 8192 (4096,0) 0.5402 0.5459 0.99x 763.5 755.5
1 16384 (4096,0) 1.1505 1.1471 1.00x 836.4 838.9
1 32768 (8192,0) 4.3768 4.3778 1.00x 879.4 879.2
1 65536 (8192,0) 9.2751 9.2596 1.00x 889.2 890.7
1 131072 (8192,0) 19.0403 19.0032 1.00x 895.2 896.9

@Phil-amd
Phil-amd self-requested a review August 11, 2026 07:34
@Phil-amd

Copy link
Copy Markdown
Member

LGTM. v_smem is already a bf16 pointer, and add_offset preserves its element type, so removing the redundant recast_iter is semantically safe. The gfx950 tests and performance results also look good.

Phil-amd

This comment was marked as abuse.

@Phil-amd
Phil-amd merged commit b425145 into ROCm:main Aug 11, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants