Errors in uploading images and annotations could be handeled better

[siddagra]

Actions before raising this issue

• I searched the existing issues and did not find anything similar.
• I read/searched the docs

Is your feature request related to a problem? Please describe.

CVAT has the tendency to completely abort any upload operation the moment it finds a single error. This results in huge time loss just trying to debug CVAT errors.

If a few extra images are annotated in COCO, it will throw an error instead of just annotating the images present in the job. rather it will completely abort the operation.

If a single annotation has an error, it will abort the operation again, uploading/importing nothing.

I have a single .txt file in a folder full of .pngs, it will throw an error and upload nothing.

Sometimes things that should not even be errors like uploading valid png files (that load fine on other annotation tools) causes errors and aborts the entire upload operation

Sometimes things that should not cause errors like trying to upload image1.png and image2.jpg COCO annotations causes errors and it will not import any annotations (#7523)

Have a whole terabyte free on your 10TB drive? Well too bad, CVAT developers know better than you and will not let you use their app AT ALL because you exceed 90% disk usage even though you have a full terabyte of storage left.

Often times it throws errors for pointless reasons. But even if that wasn't the case, and there is legitimately a slight error, it ultimately seems like a poor design philosophy to abort the whole operation and not do anything in my opinion.

It also makes working on smaller subsets of the data impossible as it will throw error that the COCO.json has more labels than the images in the job.

Instead, it should just warn you and try and import as many annotations/images as possible, like roboflow does. It should throw a warning or prompt the user to confirm instead of throwing an error and aborting the entire operation. Even if it helps a bit in maintaining data integrity, the usability of the software becomes a lot lower by aborting the entire operation.

Describe the solution you'd like

Instead, it should just warn you and try and import as many annotations/images as possible, like roboflow does. It should throw a warning or prompt the user to confirm instead of throwing an error and aborting the entire operation. Even if it helps a bit in maintaining data integrity, the usability of the software becomes a lot lower by aborting the entire operation.

Describe alternatives you've considered

One could try and tailor themselves to CVAT and have a perfect dataset with no errors + also fix errors that aren't errors but CVAT cannot process them regardless (#7523), but honestly, real world is not perfect and is messy. not to mention the developers often cannot make sure that CVAT itself will be 100% bug free (#7523). When the same images and annotation files work everywhere but in CVAT, people get turned off from using CVAT. And this is a recurring issue where it will excessively throw errors for everything it can, trying to abort the operation as much as possible, sometimes for the most pointless non-issues, or even genuinely correct formats that it simply cant process because of a bug in CVAT.

Additional context

No response

[siddagra]

I assume that since there is no response, CVAT has no intention of inculcating that feedback. Which is fine and the choice of the authors. However I have to mention that I spent 2 days trying to upload COCO annotations from my trained model to CVAT. Just resolving minutest errors (sometimes an error on CVAT itself and not even with my annotation file).

By this point we would have already gotten done with manual annotation.

All of this could largely be avoided if there were just a few images which were left un-annotated with a warning, but instead the authors have inserted assert everywhere so it completely discards the whole job and throws the baby out with the bath water.

[corentin87]

I definitely backup your idea. I'm trying to understand why my import annotations in MOT format has been failing for hours.
All I am getting is a message saying Cannot read properties of undefined (reading 'attributes'). A better log error message would definitely help the user understand where there is a problem.

[siddagra]

Thank you for the support though I don't think CVAT/opencv will do much for this. You can try and add a try: and except: block manually if you would like but ultimately it seems like poor design for an annotation tool. Seems like they have went with C++ like philosophy of fail first, but that never really worked well for most websites, especially not an annotation tool where a few images if left un-annotated would not be as big of an issue, as compared to never being able to start your work in the first place.

I think if you dig through the code you can add an alert or console.log as a warning as well if u would like, or something more fancy too.

Unfortunately, even setting up development environment to do this is also buggy now and does not work but I never got any resolution on that issue and they closed/ignored it by saying that they had to improve the documentation.