Add Ascend NPU Support #1521
Add Ascend NPU Support #1521
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1521
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit c4aba33 with merge base f7c4f14 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
Hi @xuedinge233! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at [email protected]. Thanks! |
facebook-github-bot
added
the
CLA Signed
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
xuedinge233
changed the title
xuedinge233
changed the title
|
@Jack-Khuu @mikekgfb Hello，can you help review this PR? |
install/install_requirements.sh
Outdated
| @@ -71,6 +71,9 @@ then | |||
| elif [[ -x "$(command -v xpu-smi)" ]]; | |||
| then | |||
| TORCH_NIGHTLY_URL="https://download.pytorch.org/whl/nightly/xpu" | |||
| elif [[ -x "$(command -v npu-smi)" ]] | |||
| then | |||
| TORCH_NIGHTLY_URL="https://download.pytorch.org/whl/test/cpu" | |||
There was a problem hiding this comment.
Why are we using a test wheel?
There was a problem hiding this comment.
Hey @Jack-Khuu thanks for your review. IMO, we should use nightly pytorch wheels like other backends. What do you think? @hipudding @xuedinge233
There was a problem hiding this comment.
But, we usually use pytorch RC versions, not sure if everything works fine on nightly
There was a problem hiding this comment.
Torch_npu is currently released following the test version of the PyTorch wheels. but after testing, nightly is also effective, so following the source code, modify it to the nightly version.
The latest version of torch_npu is 2.7.0, so torch has been fixed at version 2.7.0
done. |
This PR introduces native support for Huawei Ascend NPU acceleration in generative AI tasks and chat applications. The implementation leverages Ascend's CANN computing architecture to optimize tensor operations for large language models.
Compatibility Matrix:
Python 3.10.15
torch 2.7.0+cpu
torch_npu 2.7.0+git36b5d8c (nightly)
torchvision 0.22.0
torchtune 0.6.0
The following is the result generated using Llama3.1.
