Skip to content

varlen attention tutorial #3660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
svekars merged 6 commits into from
Jan 19, 2026
Merged

varlen attention tutorial #3660

svekars merged 6 commits into from
Jan 19, 2026

Conversation

@liangel-02
Copy link
Contributor

@liangel-02 liangel-02 commented Nov 24, 2025 •
edited
Loading

This PR is a tutorial for a variable length attention API

@pytorch-bot
Copy link

pytorch-bot bot commented Nov 24, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3660

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit af7fe97 with merge base 63e3575 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the cla signed label Nov 24, 2025
@liangel-02 liangel-02 force-pushed the varlen_tutorial branch 3 times, most recently from a6cb078 to 19a4ad9 Compare November 24, 2025 21:59
@liangel-02 liangel-02 requested a review from drisspg November 25, 2025 01:42
@drisspg
Copy link
Contributor

drisspg commented Nov 25, 2025

@svekars any tips for reviewing this code in a more human friendly way?

@liangel-02 liangel-02 marked this pull request as ready for review December 1, 2025 19:53
@liangel-02 liangel-02 force-pushed the varlen_tutorial branch 3 times, most recently from 46751dd to 38867c9 Compare December 1, 2025 21:10
svekars
svekars requested changes Dec 1, 2025
View reviewed changes
Copy link
Contributor

@svekars svekars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please submit this as a .py - all our tutorials are either .py or .rst and .py tutorials are converted into html and .ipynb on build. You can follow this template: https://github.com/pytorch/tutorials/blob/main/beginner_source/template_tutorial.py

@svekars
Copy link
Contributor

svekars commented Dec 1, 2025 •
edited
Loading

There is more info here: https://github.com/pytorch/tutorials#contributing and there is a script that will help you convert your .ipynb to .py

@liangel-02 liangel-02 requested a review from svekars December 2, 2025 15:51
svekars
svekars reviewed Dec 2, 2025
View reviewed changes
Copy link
Contributor

@svekars svekars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good overall. Need to add to compilers_index.rst and I added some suggestions that would help it make more discoverable on the website/google. Can also add to the What's new in the PyTorch tutorials in the index.rst.

@svekars svekars added the 2.10 2.10 PyTorch release label label Dec 2, 2025
@svekars
Copy link
Contributor

svekars commented Dec 2, 2025

There is also an error here: https://github.com/pytorch/tutorials/actions/runs/19864783846/job/56924391840?pr=3660#step:9:4582 - let me check how should we test against 2.10.

@liangel-02 liangel-02 force-pushed the varlen_tutorial branch 4 times, most recently from aac18ab to 25db45f Compare December 2, 2025 19:09
@liangel-02 liangel-02 requested a review from svekars December 2, 2025 19:10
@svekars
Copy link
Contributor

svekars commented Jan 5, 2026

@AlannaBurke can you please review this 2.10 tutorial.

@liangel-02 liangel-02 mentioned this pull request Jan 6, 2026
Open
@svekars
Copy link
Contributor

svekars commented Jan 6, 2026

Nightly preview: https://docs-preview.pytorch.org/pytorch/tutorials-nightly-preview/3660/intermediate/variable_length_attention_tutorial.html

svekars
svekars reviewed Jan 6, 2026
View reviewed changes
svekars
svekars reviewed Jan 7, 2026
View reviewed changes
intermediate_source/variable_length_attention_tutorial.py Outdated
# is_causal: bool = False,
# return_aux: AuxRequest | None = None,
# ) -> torch.Tensor | tuple[torch.Tensor, torch.Tensor]:

Copy link
Contributor

@svekars svekars Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#

svekars
svekars reviewed Jan 7, 2026
View reviewed changes
intermediate_source/variable_length_attention_tutorial.py
# and ``max_k`` are the maximum sequence lengths of query and key,
# respectively. ``is_causal`` applies causal masking if set to True and
# ``return_aux`` specifies which auxiliary outputs to return (ie ``lse``).

Copy link
Contributor

@svekars svekars Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#

@liangel-02 liangel-02 force-pushed the varlen_tutorial branch 2 times, most recently from df452db to b7c5dbf Compare January 7, 2026 19:59
svekars
svekars reviewed Jan 7, 2026
View reviewed changes
intermediate_source/variable_length_attention_tutorial.py
# Variable length attention handles sequences of varying length by
# **packing** the tensors in a batch together and essentially collapsing
# the batch dimension.

Copy link
Contributor

@svekars svekars Jan 7, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes the "Variable length attention" part got fixed but the para below still shows as code block: https://docs-preview.pytorch.org/pytorch/tutorials-nightly-preview/3660/intermediate/variable_length_attention_tutorial.html#overview-of-variable-length-attention

need to make sure, every empty line starts with a #. if you add an empty line the new line needs to have ###### and then your text. The figure below is not showing too. Here is more about sphinx-gallery syntax: https://sphinx-gallery.github.io/stable/syntax.html#embed-rest-in-your-example-python-files

@liangel-02 liangel-02 force-pushed the varlen_tutorial branch 11 times, most recently from dae4bed to 9c8d9c7 Compare January 8, 2026 17:38
AlannaBurke
AlannaBurke reviewed Jan 12, 2026
View reviewed changes
Copy link
Contributor

@AlannaBurke AlannaBurke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment on the diagram, but otherwise with Svetlana's changes this looks good to me!

_static/img/varlen_diagram.png
Copy link
Contributor

@AlannaBurke AlannaBurke Jan 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any way to make this diagram accessible? If not, can you make sure that there's a thorough description of it?

Copy link
Contributor Author

@liangel-02 liangel-02 Jan 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added more alt text!

Copy link
Contributor

@AlannaBurke AlannaBurke Jan 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

excellent, thanks!

@AlannaBurke
Copy link
Contributor

AlannaBurke commented Jan 15, 2026

looks great, make sure you've resolved all of @svekars 's comments

@liangel-02
Copy link
Contributor Author

liangel-02 commented Jan 15, 2026

looks great, make sure you've resolved all of @svekars 's comments

yep resolved

@svekars svekars self-requested a review January 15, 2026 22:47
@svekars svekars merged commit e156b07 into main Jan 19, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AlannaBurke AlannaBurke AlannaBurke approved these changes

@svekars svekars svekars approved these changes

@drisspg drisspg drisspg approved these changes

Assignees

No one assigned

Labels

2.10 2.10 PyTorch release label cla signed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@liangel-02 @drisspg @svekars @AlannaBurke