bbox_nms doesn't fully reduce boxes

[agriic]

Describe the bug

During training, in validation step bbox_nms returns boxes which should have been clearly merged (1px shifts and the same class). This produces wrong metrics.

This happens only for big validation batch sizes (default 32).
It happens randomly and not for all images (i see ~ 20% of images from batch having problems).

I am avoiding this, using smaller validation batch size (8). I did also increased min_confidence in nms config to 0.4

System Info (please complete the following ## information):

• OS: ubuntu 24.04.01 (using one GPU - 4090)
• Python Version: 3.10.14
• PyTorch Version: 2.5.1
• CUDA/cuDNN/MPS Version: 12.6
• YOLO Model Version: YOLOv9-c