[Tracker] All the issue related with ONNX model zoo models #886
Description
We have enabled total 2219 unique models from ONNX library. This issue is to track all the issues seen in the models.
Passing Summary
TOTAL TESTS = 2219
| Stage | # Passing | % of Total | % of Attempted |
|---|---|---|---|
| Setup | 2217 | 99.9% | 99.9% |
| IREE Compilation | 2149 | 96.8% | 96.9% |
| Gold Inference | 2149 | 96.8% | 100.0% |
| IREE Inference Invocation | 2146 | 96.7% | 99.9% |
| Inference Comparison (PASS) | 2011 | 90.6% | 93.7% |
Fail Summary
TOTAL TESTS = 2219
| Stage | # Failed at Stage | % of Total |
|---|---|---|
| Setup | 2 | 0.1% |
| IREE Compilation | 68 | 3.1% |
| Gold Inference | 0 | 0.0% |
| IREE Inference Invocation | 3 | 0.1% |
| Inference Comparison | 135 | 6.1% |
CPU
| # | issue type | issue no | #model impacted | list of model | assignee | status |
|---|---|---|---|---|---|---|
| 1 | failed to legalize operation 'torch.operator' -> QLinearAdd / QLinearLeakyRelu / QLinearConcat / onnx.QLinearGlobalAveragePool | 949 | 15 | QLinearAdd | Importer - Torch MLIR - (Assigned) | |
| 2 | failed to legalize operation 'torch.constant.float | 908 | 8 | Roi Align | Torch MLIR (In Progress) | |
| 3 | failed to legalize operation 'onnx.Loop' | 948 | 7 | Loop | Torch MLIR - (In Progress) | |
| 4 | Assertion `result && ""Zero-sized vectors are not allowed""' failed.\n' | 913 | 3 | If | Torch MLIR - (Unassigned) | |
| 5 | expected offsets to be non-negative, but got -1 | 19935 | 2 | Offsets | IREE - (Unassigned) | |
| 6 | Incorrect imported IR | 905 | 8 | Incorrect IR | Torch MLIR - (Unassigned) | |
| 7 | failed to legalize operation 'torch.operator' -> AvgPool | 895 | 3 | AvgPool | Torch MLIR (Review) - 3923 | |
| 8 | 'tensor.cast' op operand type 'tensor<80x15130xf32>' and result type 'tensor<15130xf32>' are cast incompatible | 891 | 3 | NMS | Torch MLIR - Review 3999, IREE - Assigned19474 | |
| 9 | error: invalid size -9223372036854775808 for !torch.tensor type | 0 | Invalid | ✅Can be solved by adding model to the optimization using onnx.runtime list | ||
| 10 | failed to legalize operation "onnx.CategoryMapper" | 910 | 2 | CategoryMapper | torch-mlir(Unassigned) | |
| 11 | 'hal.tensor.barrier' op failed to verify that all of {sources, results} have same type | 2 | Barrier | Resolves with onnx runtime shape infer | ||
| 12 | failed to legalize operation 'torch.operator' that was explicitly marked illegal onnx.If |
930 | 1 | Onnx.If | Importer issue | |
| 13 | failed to legalize operation 'onnx.LSTM' | 947 | 1 | onnx.LSTM | ✅Torch MLIR- (Completed) | |
| 14 | 'linalg.generic' op write affecting operations on global resources are restricted to workgroup distributed contexts | 20156 | 2 | Conv | IREE - (Unassigned) -Upstream 🔴Vectorization does not support such case, so we should not use CPUDefault for this anyway, otherwise all the ops are not vectorized. | |
| 15 | One or more operations with large vector sizes (32768 bytes) were found | 21643 | 25 | Large Vector Sizes | ✅IREE |