-
Notifications
You must be signed in to change notification settings - Fork 28.7k
Verify interpolation of image processors #28180
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
@NielsRogge Thanks for opening the issue! It's fine to open up to the community but you'll need to add a checklist of the image processors so it's clear who is working on what and what's done as well as ideally some instructions on what it means for each one to be "done" e.g. making sure to run slow tests for models. |
If I understand it correctly, we need to match the interpolation: For example for convnext: convnext should be changed to Bicubic as per timm/convnext . If that's correct , I can take this up for all the models. Let me know. |
Yes that is correct, see also the original implementation. Thanks for spotting that. Hence feel free to open a PR to update this, along with the image processor created in the conversion script. Ideally we assert the pixel values created by it against the original implementation, like done here for DINOv2. |
Sure! thanks for the pointers, will work on it. |
DieT and DPT default interpolation types matches with the original implementation types to BICUBIC . That's what I see it. Let me know if I overlooked. |
would you have a look ? |
@NielsRogge Can you please complete the checklist here? |
@NielsRogge |
@NielsRogge , @nileshkokane01 Can I work on this issue and help in completing the checklist ? |
@NielsRogge , is it possible for me to contribute ? Thanks. |
@NielsRogge , can I work on this issue? |
Please don't ask to work on it, just open a PR for one of the models which are still open/which doesn't have a merged PR yet :) Thanks. |
Feature request
As pointed out in #27742, some image processors might need a correction on the default interpolation method being used (resampling in Pillow). We could check this on a per-model basis.
Motivation
Interpolation methods have a slight (often minimal) impact on performance. However it would be great to verify this on a per-model basis.
e.g. ViT's image processor defaults to BILINEAR but should use BICUBIC as seen here. We can update the default values of the image processors, but can't update the configs on the hub as this would break people's fine-tuned models.
Your contribution
I could work on this, but this seems like a good first issue for first contributors.
To be checked (by comparing against original implementation):
The text was updated successfully, but these errors were encountered: