Struggling with YOLO setup – First attempts, but stuck

[Rexem99]

Hi guys,

I recently started working with computer vision as part of my thesis. Initially, I achieved good results using the Ultralytics YOLO model, but its license poses a challenge for future use cases. After some research, I believe this YOLO repository is the best fit for my needs. I also tried YOLOX, DAMO-YOLO, and YOLO-NAS but couldn’t get them to work.

I now have some questions and problems regarding the training process and on how to correctly load a trained model for predictions. I would be very happy if someone can give me some ideas on how to solve these problems.

I am training on a custom dataset in the normalized yolo labeling format (c,x,y,w,h) with the following folder structure and configs:
#######################################
dataset/
├── images
│ ├── train
│ └── val
├── labels
│ ├── train
│ └── val
└── classes.txt
#######################################
dataset.yaml:
path: dataset_dog/
train: train
validation: val

class_num: 4
class_list: ["Bicycle","Car","Cat","Dog"]
#######################################
config.yaml:

hydra:
  run:
    dir: runs

name: v9-train-dog

defaults:
  - _self_
  - task: train
  - dataset: yolo_dog
  - model: v9-m
  - general

#######################################
general.yaml:

device: cuda
cpu_num: 8

image_size: [640, 640]

out_path: runs
exist_ok: True

lucky_number: 10
use_wandb: False
use_tensorboard: True

weight: yolo/v9-m.pt

#######################################

1. While training on my custom dataset the training metrics look like this:
   

2. After training the last checkpoint is being saved in the runs folder. I am currently loading the model how it is suggested in the lazy.py and I do not get any predictions on my pictures. I am not sure if I am doing something wrong with the training or the prediction and on how to validate if the model was trained correctly or not.

The configs for predictions look like this:
#######################################
config_inference.yaml

hydra:
  run:
    dir: runs

name: v9-inf

defaults:
  - _self_
  - task: inference
  - dataset: yolo_dog
  - model: v9-m 
  - general_inference

#######################################
general_inference.yaml

device: cuda
cpu_num: 8

image_size: [640, 640]

out_path: runs
exist_ok: True

lucky_number: 10
use_wandb: False
use_tensorboard: True

weight: yolo/v9-m-custom.ckpt

#######################################

What is the correct way to load the model for predictions? I want to setup a pipeline in which I can predict on camera (not webcam) pictures in real time inside a loop with images in opencv format and get the results to work with (which class was predictied etc. like in ultralytics).

[divyanidhi]

Hi @Rexem99
Were you able to solve this issue?

[fjodborg]

Hi @divyanidhi and @Rexem99 .
If i remembered right, i had the same problem and it was fixed by adding pull request #175. It was related to how this repository loads in the yolo annotation format for bounding boxes.

Please reply if that worked, just so i know.

[divyanidhi]

Hi @fjodborg

Adding this PR to code does solve the issue of yolo format, but I am not able to get correct inferencing on the custom trained model. Adding image as a reference.

Do you have any suggestions on what could be the issue?

[fjodborg]

I can't remember having that problem.

Is it a model you trained and if so, for how long, what is the step size and what is the loss during training?

Is the image resized to the model input size? (I can't remember if this is part of the inference step)

EDIT 1:
I just noticed that the end corner of both bounding boxes are pointing to the center of the object. This looks like a format issue.
Is the format center format (cx, cy, w, h) or corner format (x0, y0, x1, y1)?

[divyanidhi]

My training data is in YOLO detection format (cx, cy, w, h). While training, since I had incorporated #PR175, it converts bounding boxes to YOLO segmentation format if they are in YOLO detection format.
I had trained a model with 30 epochs. Loss looks something like this:

I am using below code for inferencing.

!python yolo/lazy.py task=inference model=v9-c device=gpu weight=custom-trained-model task.nms.min_confidence=0.1 task.data.source=test_image.png use_wandb=False

[fjodborg]

Hmm, to me the training looks alright. But the output image puts all my suspicion on the the bounding box format somewhere not being correct.
Again due to the bounding box having its corner in the center of a correct object. Which to me seems like a cx and cy is being interpreted as a corner instead of a center.

I can't think of other things, sorry.

[pzoltowski]

did anyone can just update readme / docs and put some very tiny custom dataset ? And exact command line for training and inference? This repo is very poor quality regarding that imho and I struggled already the whole week. I know I'm ranting but this is terrible engineering. It's not well documented if we supposed to use YOLO format or COCO format and the whole folder structure is completely off. README having too many magic numbers like path and don't know if its absolute or relative (but to what?).

[fjodborg]

I'm not the author of the repo and it isn't a plug and play repo, on that we can agree. But honestly, all of it has most likely been freetime work and people need to understand that. If i maintained an open source project for fun i would most likely also focus on the fun parts, which usually isn't the documentation or small edge cases. They would probably be happy if someone volunteered to help maintain it, but few do so.

Honestly, just be thankful that all the hardwork is already complete in this repo and only a few minor documentation things are missing, which can be derived if you read through the code or the open pull requests. They made it open source under MIT license, so if you aren't happy, just fork it and make your changes or perhaps contribute to the repo? Maybe you could find a fork that has better documentation, or if you really want something plug and play, perhaps use ultralyics and pay for their commercial license if you need that.

[pzoltowski]

I know. It's just hard to even contribute to docs o repo if someone cannot even test it. Half of issues on this repo are just marked as questions