-
Notifications
You must be signed in to change notification settings - Fork 298
[FX] Dynamic Shapes Support #3225
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
AlexanderDokuchaev
merged 19 commits into
openvinotoolkit:develop
from
anzr299:fx/dynamic_shapes
Mar 5, 2025
Merged
Changes from 13 commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
8cebf99
init
anzr299 fbd5779
fix
anzr299 eda0549
pre-commit fix
anzr299 36aca2f
Add test case to check data type on edge tensor shape
anzr299 3057cf2
clean
anzr299 e25e594
pre commit fix
anzr299 bdc43c0
Merge branch 'openvinotoolkit:develop' into fx/dynamic_shapes
anzr299 c56ff98
update test
anzr299 8a6a923
Merge branch 'fx/dynamic_shapes' of https://github.com/anzr299/nncf i…
anzr299 f77b395
pre commit fix && clean
anzr299 ced6be1
Comment Fix
anzr299 62979bf
comment fixes
anzr299 292c5f1
Merge branch 'openvinotoolkit:develop' into fx/dynamic_shapes
anzr299 da74767
update reference graphs
anzr299 107c711
apply suggested changes
anzr299 ef28db1
pre commit fix
anzr299 6ccbba8
update reference graph
anzr299 ad9f638
remove dynamic shapes extra test
anzr299 253d703
review changes
anzr299 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1,110 changes: 1,110 additions & 0 deletions
1,110
...ta/reference_graphs/fx/dynamic_shapes/post_quantization_compressed/mobilenet_v3_small.dot
Large diffs are not rendered by default.
Oops, something went wrong.
497 changes: 497 additions & 0 deletions
497
...s/torch/data/reference_graphs/fx/dynamic_shapes/post_quantization_compressed/resnet18.dot
Large diffs are not rendered by default.
Oops, something went wrong.
5,924 changes: 5,924 additions & 0 deletions
5,924
.../torch/data/reference_graphs/fx/dynamic_shapes/post_quantization_compressed/swin_v2_s.dot
Large diffs are not rendered by default.
Oops, something went wrong.
49 changes: 49 additions & 0 deletions
49
...reference_graphs/fx/dynamic_shapes/post_quantization_compressed/synthetic_transformer.dot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| strict digraph { | ||
| "0 wte_weight" [id=0, type=get_attr]; | ||
| "1 linear_bias" [id=1, type=get_attr]; | ||
| "2 lm_head_bias" [id=2, type=get_attr]; | ||
| "3 input_ids" [id=3, type=input]; | ||
| "4 embedding" [id=4, type=embedding]; | ||
| "5 embedding_0_0_nncf_smooth_quant_0" [id=5, type=call_module]; | ||
| "6 quantize_per_tensor_default" [id=6, type=quantize_per_tensor]; | ||
| "7 dequantize_per_tensor_default" [id=7, type=dequantize_per_tensor]; | ||
| "8 scale_updated_constant0" [id=8, type=get_attr]; | ||
| "9 compressed_weight_updated_constant0" [id=9, type=get_attr]; | ||
| "10 mul_tensor" [id=10, type=mul]; | ||
| "11 zero_point_updated_constant0" [id=11, type=get_attr]; | ||
| "12 sub_tensor" [id=12, type=sub]; | ||
| "13 linear" [id=13, type=linear]; | ||
| "14 linear_0_0_nncf_smooth_quant_0" [id=14, type=call_module]; | ||
| "15 quantize_per_tensor_default_1" [id=15, type=quantize_per_tensor]; | ||
| "16 dequantize_per_tensor_default_1" [id=16, type=dequantize_per_tensor]; | ||
| "17 scale_updated_constant1" [id=17, type=get_attr]; | ||
| "18 compressed_weight_updated_constant1" [id=18, type=get_attr]; | ||
| "19 mul_tensor_1" [id=19, type=mul]; | ||
| "20 zero_point_updated_constant1" [id=20, type=get_attr]; | ||
| "21 sub_tensor_1" [id=21, type=sub]; | ||
| "22 linear_1" [id=22, type=linear]; | ||
| "23 output" [id=23, type=output]; | ||
| "0 wte_weight" -> "4 embedding" [label="(10, 5)", style=solid]; | ||
| "1 linear_bias" -> "13 linear" [label="(5,)", style=solid]; | ||
| "2 lm_head_bias" -> "22 linear_1" [label="(10,)", style=solid]; | ||
| "3 input_ids" -> "4 embedding" [label="(-1,)", style=solid]; | ||
| "4 embedding" -> "5 embedding_0_0_nncf_smooth_quant_0" [label="(-1, 5)", style=solid]; | ||
| "5 embedding_0_0_nncf_smooth_quant_0" -> "6 quantize_per_tensor_default" [label="(-1, 5)", style=solid]; | ||
| "6 quantize_per_tensor_default" -> "7 dequantize_per_tensor_default" [label="(-1, 5)", style=solid]; | ||
| "7 dequantize_per_tensor_default" -> "13 linear" [label="(-1, 5)", style=solid]; | ||
| "8 scale_updated_constant0" -> "10 mul_tensor" [label="(5, 1)", style=solid]; | ||
| "9 compressed_weight_updated_constant0" -> "10 mul_tensor" [label="(5, 5)", style=solid]; | ||
| "10 mul_tensor" -> "12 sub_tensor" [label="(5, 5)", style=solid]; | ||
| "11 zero_point_updated_constant0" -> "12 sub_tensor" [label="(5, 1)", style=solid]; | ||
| "12 sub_tensor" -> "13 linear" [label="(5, 5)", style=solid]; | ||
| "13 linear" -> "14 linear_0_0_nncf_smooth_quant_0" [label="(-1, 5)", style=solid]; | ||
| "14 linear_0_0_nncf_smooth_quant_0" -> "15 quantize_per_tensor_default_1" [label="(-1, 5)", style=solid]; | ||
| "15 quantize_per_tensor_default_1" -> "16 dequantize_per_tensor_default_1" [label="(-1, 5)", style=solid]; | ||
| "16 dequantize_per_tensor_default_1" -> "22 linear_1" [label="(-1, 5)", style=solid]; | ||
| "17 scale_updated_constant1" -> "19 mul_tensor_1" [label="(10, 1)", style=solid]; | ||
| "18 compressed_weight_updated_constant1" -> "19 mul_tensor_1" [label="(10, 5)", style=solid]; | ||
| "19 mul_tensor_1" -> "21 sub_tensor_1" [label="(10, 5)", style=solid]; | ||
| "20 zero_point_updated_constant1" -> "21 sub_tensor_1" [label="(10, 1)", style=solid]; | ||
| "21 sub_tensor_1" -> "22 linear_1" [label="(10, 5)", style=solid]; | ||
| "22 linear_1" -> "23 output" [label="(-1, 10)", style=solid]; | ||
| } |
515 changes: 515 additions & 0 deletions
515
tests/torch/data/reference_graphs/fx/dynamic_shapes/post_quantization_compressed/unet.dot
Large diffs are not rendered by default.
Oops, something went wrong.
2,011 changes: 2,011 additions & 0 deletions
2,011
...s/torch/data/reference_graphs/fx/dynamic_shapes/post_quantization_compressed/vit_b_16.dot
Large diffs are not rendered by default.
Oops, something went wrong.
63 changes: 63 additions & 0 deletions
63
...ta/reference_graphs/fx/dynamic_shapes/post_quantization_compressed/yolo11n_sdpa_block.dot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| strict digraph { | ||
| "0 x" [id=0, type=input]; | ||
| "1 x_0_0_nncf_smooth_quant_0" [id=1, type=call_module]; | ||
| "2 quantize_per_tensor_default" [id=2, type=quantize_per_tensor]; | ||
| "3 dequantize_per_tensor_default" [id=3, type=dequantize_per_tensor]; | ||
| "4 scale_updated_constant0" [id=4, type=get_attr]; | ||
| "5 compressed_weight_updated_constant0" [id=5, type=get_attr]; | ||
| "6 mul_tensor" [id=6, type=mul]; | ||
| "7 zero_point_updated_constant0" [id=7, type=get_attr]; | ||
| "8 sub_tensor" [id=8, type=sub]; | ||
| "9 linear" [id=9, type=linear]; | ||
| "10 quantize_per_tensor_default_1" [id=10, type=quantize_per_tensor]; | ||
| "11 dequantize_per_tensor_default_1" [id=11, type=dequantize_per_tensor]; | ||
| "12 slice_1" [id=12, type=slice]; | ||
| "13 slice_2" [id=13, type=slice]; | ||
| "14 slice_3" [id=14, type=slice]; | ||
| "15 quantize_per_tensor_default_2" [id=15, type=quantize_per_tensor]; | ||
| "16 dequantize_per_tensor_default_2" [id=16, type=dequantize_per_tensor]; | ||
| "17 slice_4" [id=17, type=slice]; | ||
| "18 slice_5" [id=18, type=slice]; | ||
| "19 slice_6" [id=19, type=slice]; | ||
| "20 slice_7" [id=20, type=slice]; | ||
| "21 slice_8" [id=21, type=slice]; | ||
| "22 slice_9" [id=22, type=slice]; | ||
| "23 transpose" [id=23, type=transpose]; | ||
| "24 matmul" [id=24, type=matmul]; | ||
| "25 div_" [id=25, type=div_]; | ||
| "26 softmax" [id=26, type=softmax]; | ||
| "27 transpose_1" [id=27, type=transpose]; | ||
| "28 matmul_1" [id=28, type=matmul]; | ||
| "29 output" [id=29, type=output]; | ||
| "0 x" -> "1 x_0_0_nncf_smooth_quant_0" [label="(1, -1, -1)", style=solid]; | ||
| "1 x_0_0_nncf_smooth_quant_0" -> "2 quantize_per_tensor_default" [label="(1, -1, 4)", style=solid]; | ||
| "2 quantize_per_tensor_default" -> "3 dequantize_per_tensor_default" [label="(1, -1, 4)", style=solid]; | ||
| "3 dequantize_per_tensor_default" -> "9 linear" [label="(1, -1, 4)", style=solid]; | ||
| "4 scale_updated_constant0" -> "6 mul_tensor" [label="(12, 1)", style=solid]; | ||
| "5 compressed_weight_updated_constant0" -> "6 mul_tensor" [label="(12, 4)", style=solid]; | ||
| "6 mul_tensor" -> "8 sub_tensor" [label="(12, 4)", style=solid]; | ||
| "7 zero_point_updated_constant0" -> "8 sub_tensor" [label="(12, 1)", style=solid]; | ||
| "8 sub_tensor" -> "9 linear" [label="(12, 4)", style=solid]; | ||
| "9 linear" -> "10 quantize_per_tensor_default_1" [label="(1, -1, 12)", style=solid]; | ||
| "9 linear" -> "15 quantize_per_tensor_default_2" [label="(1, -1, 12)", style=solid]; | ||
| "9 linear" -> "20 slice_7" [label="(1, -1, 12)", style=solid]; | ||
| "10 quantize_per_tensor_default_1" -> "11 dequantize_per_tensor_default_1" [label="(1, -1, 12)", style=solid]; | ||
| "11 dequantize_per_tensor_default_1" -> "12 slice_1" [label="(1, -1, 12)", style=solid]; | ||
| "12 slice_1" -> "13 slice_2" [label="(1, -1, 12)", style=solid]; | ||
| "13 slice_2" -> "14 slice_3" [label="(1, -1, 12)", style=solid]; | ||
| "14 slice_3" -> "24 matmul" [label="(1, -1, 4)", style=solid]; | ||
| "15 quantize_per_tensor_default_2" -> "16 dequantize_per_tensor_default_2" [label="(1, -1, 12)", style=solid]; | ||
| "16 dequantize_per_tensor_default_2" -> "17 slice_4" [label="(1, -1, 12)", style=solid]; | ||
| "17 slice_4" -> "18 slice_5" [label="(1, -1, 12)", style=solid]; | ||
| "18 slice_5" -> "19 slice_6" [label="(1, -1, 12)", style=solid]; | ||
| "19 slice_6" -> "23 transpose" [label="(1, -1, 4)", style=solid]; | ||
| "20 slice_7" -> "21 slice_8" [label="(1, -1, 12)", style=solid]; | ||
| "21 slice_8" -> "22 slice_9" [label="(1, -1, 12)", style=solid]; | ||
| "22 slice_9" -> "28 matmul_1" [label="(1, -1, 4)", style=solid]; | ||
| "23 transpose" -> "24 matmul" [label="(1, 4, -1)", style=solid]; | ||
| "24 matmul" -> "25 div_" [label="(1, -1, -1)", style=solid]; | ||
| "25 div_" -> "26 softmax" [label="(1, -1, -1)", style=solid]; | ||
| "26 softmax" -> "27 transpose_1" [label="(1, -1, -1)", style=solid]; | ||
| "27 transpose_1" -> "28 matmul_1" [label="(1, -1, -1)", style=solid]; | ||
| "28 matmul_1" -> "29 output" [label="(1, -1, 4)", style=solid]; | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.