TST Accelerate mixed adapter batches tests - #3630
Open
BenjaminBossan wants to merge 1 commit into
Open
Conversation
The mixed adapter batches tests are relatively slow on CI, especially with MoE models. They constititute 4 out of the 5 slowest tests, with ~100 sec total: https://github.com/huggingface/peft/actions/runs/33380424434/job/99451723715 The tests are a bit wasteful right now for MoE LoRA because target_parameters does not support this feature, so we check that trying to use it raises an appropriate error. The problem is that before checking that, we first run 3 expensive generate calls, which are not required to check the error. This PR reorders the generate calls to call with mixed adapter batches first, triggering the error immediately. This is fine, as the order does not affect the results. In my local testing on CPU, the total test time for this test went down from >90 sec to 30 sec, of which 20 sec are for test discovery. To help a little more with speeding things up, I also decreased the max length and number of beams. The test_mixed_adapter_batches test was updated too, mainly for consistency.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Member
Author
|
Update: With the CI finished, we can verify that the https://github.com/huggingface/peft/actions/runs/33393694722/job/99493615262?pr=3630#step:8:1512 We can't see how long exactly they took, but it means that they went from > 25 sec to < 9 sec, saving us at least one minute on the CI. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The mixed adapter batches tests are relatively slow on CI, especially with MoE models. They constitute 4 out of the 5 slowest tests, with ~100 sec total:
https://github.com/huggingface/peft/actions/runs/33380424434/job/99451723715#step:8:1512
The tests are a bit wasteful right now for MoE LoRA because
target_parametersdoes not support this feature, so we check that trying to use it raises an appropriate error. The problem is that before checking that, we first run 3 expensive generate calls, which are not required to check the error.This PR reorders the generate calls to call with mixed adapter batches first, triggering the error immediately. This is fine, as the order does not affect the results.
In my local testing on CPU, the total test time for this test went down from >90 sec to 30 sec, of which 20 sec are for test discovery.
To help a little more with speeding things up, I also decreased the max length and number of beams.
The
test_mixed_adapter_batchestest was updated too, mainly for consistency.