Add PRC models to test_text_generation_example.py #1695
Conversation
Signed-off-by: Chen, Wenbin <[email protected]>
|
@wenbinc-Bin would you please post the test results with these additions. |
>>> RUN_SLOW=true GAUDI2_CI=1 python -m pytest tests/test_text_generation_example.py -s -v -k qwen --token $(TOKEN)
12 passed, 64 deselected, 1 warning in 640.85s (0:10:40) |
|
Also chaglm2 |
|
@wenbinc-Bin do we need to add all the tests? Maybe we can remove |
Signed-off-by: Chen, Wenbin <[email protected]>
I can remove them. 14B models are relatively less important. |
| @@ -41,6 +41,7 @@ | |||
| ("bigcode/starcoder2-3b", 1, False, 261.07213776344133, True), | |||
| ("adept/persimmon-8b-base", 4, False, 366.73968820698406, False), | |||
| ("Qwen/Qwen1.5-7B", 4, False, 490.8621617893209, False), | |||
| ("Qwen/Qwen1.5-32B", 4, False, 120, False), | |||
There was a problem hiding this comment.
why do we need both 7B and 32B of 1.5 if we already have 2.5?
There was a problem hiding this comment.
Because we said we support these models, we need to make sure there are no regression. We don't know what models users are using.
There was a problem hiding this comment.
one type of each is enough, we can't test all. but if you want to test on your own for each release, you can do that.
| ("Qwen/Qwen2.5-7B", 4, False, 490, False), | ||
| ("Qwen/Qwen2.5-32B", 4, False, 120, False), |
There was a problem hiding this comment.
can you remove 7B from multi test?
There was a problem hiding this comment.
Do you mean remove "Qwen/Qwen2.5-7B"? I think we need to test small models. Customers use small models to test and build their APPs.
| @@ -87,6 +91,9 @@ | |||
| ("meta-llama/Meta-Llama-3-70B-Instruct", 8, 1, 64), | |||
| ("facebook/opt-66b", 2, 1, 28.48069266504111), | |||
| ("google/gemma-2-9b", 8, 1, 110.12610917383735), | |||
| ("Qwen/Qwen1.5-72B", 2, 1, 26), | |||
There was a problem hiding this comment.
why do we need 1.5 and 2.0 if we have 2.5?
There was a problem hiding this comment.
Because we said we support these models, we need to make sure there are no regression. We don't know what models users are using.
|
@wenbinc-Bin can you please remove the extra ones as Libin suggested and only keep one type of each, so we can merge and close this PR. Thanks |
Signed-off-by: Chen, Wenbin <[email protected]>
|
Ok, I update the PR. |
What does this PR do?
Add Chinese models to test_text_generation_example.py