Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] yolov7 onnx #86

Open
wants to merge 5 commits into
base: DEPLOY
Choose a base branch
from
Open

Conversation

f-fl0
Copy link
Contributor

@f-fl0 f-fl0 commented Aug 23, 2024

Description

This PR addresses the following issues when trying to use the yolov7 model with ONNX. Running the following command leads to this error:

python yolo/lazy.py task=inference name=v7-onnx device=cpu model=v7 task.nms.min_confidence=0.1 task.fast_inference=onnx task.data.source=./demo/images/inference/image.png
[08/23 02:45:49]   INFO  | 📄 Created log folder: runs/inference/v7-onnx
/usr/local/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:65: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'
  warnings.warn(
[08/23 02:45:49] WARNING | 🈳 Error loading ONNX model: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from v7.onnx failed:Load model v7.onnx failed. File doesn't exist
[08/23 02:45:49]   INFO  | 🚜 Building YOLO
[08/23 02:45:49]   INFO  |   🏗️  Building backbone
[08/23 02:45:49]   INFO  |   🏗️  Building head
/ws/tmp/YOLO/yolo/model/yolo.py:126: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  weights = torch.load(weights, map_location=torch.device("cpu"))
[08/23 02:45:50]   INFO  | ✅ Success load model & weight
[08/23 02:45:52]   INFO  | 📥 ONNX model saved to v7.onnx
[08/23 02:45:52]   INFO  | 🈶 Found stride of model [8, 16, 32]
Error executing job with overrides: ['task=inference', 'name=v7-onnx', 'device=cpu', 'model=v7', 'task.nms.min_confidence=0.1', 'task.fast_inference=onnx', 'task.data.source=./demo/images/inference/image.png']
Traceback (most recent call last):
  File "/ws/tmp/YOLO/yolo/lazy.py", line 30, in main
    converter = create_converter(cfg.model.name, model, cfg.model.anchor, cfg.image_size, device)
  File "/ws/tmp/YOLO/yolo/utils/bounding_box_utils.py", line 373, in create_converter
    converter = Anc2Box(*args, **kwargs)
  File "/ws/tmp/YOLO/yolo/utils/bounding_box_utils.py", line 326, in __init__
    self.class_num = model.num_classes
AttributeError: 'InferenceSession' object has no attribute 'num_classes'

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Once num_classes is added to ort_session, running the same command leads another error:

[08/23 02:48:16]   INFO  | 📄 Created log folder: runs/inference/v7-onnx
/usr/local/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:65: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'
  warnings.warn(
[08/23 02:48:16]   INFO  | 🚀 Using ONNX as MODEL frameworks!
[08/23 02:48:16]   INFO  | 🈶 Found stride of model [8, 16, 32]
[08/23 02:48:16]   INFO  | 👀 Start Inference!
Error executing job with overrides: ['task=inference', 'name=v7-onnx', 'device=cpu', 'model=v7', 'task.nms.min_confidence=0.1', 'task.fast_inference=onnx', 'task.data.source=./demo/images/inference/image.png']
Traceback (most recent call last):
  File "/ws/tmp/YOLO/yolo/lazy.py", line 39, in main
    solver.solve(dataloader)
  File "/ws/tmp/YOLO/yolo/tools/solver.py", line 207, in solve
    raise e
  File "/ws/tmp/YOLO/yolo/tools/solver.py", line 183, in solve
    predicts = self.post_proccess(predicts, rev_tensor)
  File "/ws/tmp/YOLO/yolo/utils/model_utils.py", line 133, in __call__
    prediction = self.converter(predict["Main"])
  File "/ws/tmp/YOLO/yolo/utils/bounding_box_utils.py", line 352, in __call__
    predict = rearrange(predict, "B (L C) h w -> B L h w C", L=self.anchor_num)
  File "/usr/local/lib/python3.10/site-packages/einops/einops.py", line 451, in rearrange
    tensor = get_backend(tensor[0]).stack_on_zeroth_dimension(tensor)
  File "/usr/local/lib/python3.10/site-packages/einops/_backends.py", line 334, in stack_on_zeroth_dimension
    return self.torch.stack(tensors)
RuntimeError: stack expects each tensor to be equal size, but got [1, 255, 80, 80] at entry 0 and [1, 255, 40, 40] at entry 1

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Type of Change

Bug fix (non-breaking change which fixes an issue)

Checklist:

  • The code follows the Python style guide.
  • Code and files are well organized.
  • All tests pass.
  • New code is covered by tests.
  • The pull request is directed to the corresponding topic branch.

Licensing:

By submitting this pull request, I confirm that:

  • My contribution is made under the MIT License.
  • I have not included any code from questionable or non-compliant sources (GPL, AGPL, ... etc).
  • I understand that all contributions to this repository must comply with the MIT License, and I promise that my contributions do not violate this license.
  • I have not used any code or content from sources that conflict with the MIT License or are otherwise legally questionable.

@f-fl0
Copy link
Contributor Author

f-fl0 commented Nov 5, 2024

After rebasing to the latest DEPLOY branch, the test about bbox_nms no longer passes. The changes from 2522f72 should fix the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant