Add KV cache for paged/non-paged attention

[cyanguwa]

Description

This PR adds KV cache support for FusedAttention, FlashAttention, and UnfusedDotProductAttention backends in TE-PyTorch.

backend  | precision      |    KV cache     | architecture | qkv_format    | page_size
---------------------------------------------------------------------------------------
Fused    | FP16/BF16      | non-paged/paged | sm80+        | bshd,sbhd,thd | >= 1
Flash v2 | FP16/BF16      | non-paged/paged | sm80+        | bshd,sbhd,thd | >= 256
Flash v3 | FP16/BF16      | non-paged/paged | sm90         | bshd,sbhd,thd | >= 1
         | FP8            | non-paged/paged | sm90         | thd           | >= 1
Unfused  | FP32/FP16/BF16 | non-paged/paged | all          | bshd,sbhd,thd | >= 1

• KV cache is in bshd format, and incoming tokens can be in bshd, sbhd, or thd
• FusedAttention processes QKV in bshd, sbhd_2bshd, or thd_2bshd format directly
• FlashAttention v2 converts Q to thd and uses flash_attn_varlen_func for attention
• FlashAttention v3 (must be 39e7197 or after) converts Q to thd and uses flash_attn_with_kvcache for attention
• UnfusedDotProductAttention converts Q to bshd for attention; for paged, it converts the cache tensors to non-paged first, based on the page table
• All backends support pure context, pure generation, and mixed context/generation phases
• FusedAttention and FlashAttention support CUDA graph
• K cache and V cache should have the same page table
• fp8_dpa=True is supported (KV cache is still in FP16/BF16 precision); fp8_mha=True is not
• Context parallelism is not supported
• RoPE for inference will be fixed in RoPE enhancements #1478
• Requires FE1.11 from Update FE to 1.11 #1580

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refractor

Changes

Please list the changes introduced in this PR:

• Add KV caching support for FusedAttention, FlashAttention, and UnfusedDotProductAttention
• Add mixed q/kv format support to FusedAttention for F16
• Adapt to the new FA3 APIs from FA2.7.3+/hopper for CP and non-CP and CP cases

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[cyanguwa]

/te-ci pytorch L0

[cyanguwa]

/te-ci pytorch L0

[cyanguwa]

/te-ci pytorch L0

[cyanguwa]

/te-ci pytorch L1

[cyanguwa]

/te-ci pytorch L3

[cyanguwa]

/te-ci pytorch L0 L1 L3

[cyanguwa]

/te-ci jax L0

[cyanguwa]

/te-ci pytorch L0

[cyanguwa]

/te-ci pytorch L0 L1 L3