Skip to content

Add D-FINE to KerasHub #2318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

harshaljanjani
Copy link
Collaborator

@harshaljanjani harshaljanjani commented Jul 9, 2025

Description of the change

Welcome D-FINE to the KerasHub family of models!
D-FINE, a powerful real-time object detector, sets a new state-of-the-art benchmark for object detection on KerasHub. It achieves outstanding localization precision by redefining the bounding box regression task in DETR models. Additionally, it incorporates lightweight optimizations in computationally intensive modules and operations, striking a better balance between speed and accuracy. Specifically, D-FINE-L/X achieves 54.0%/55.8% AP on the COCO dataset at 124/78 FPS on an NVIDIA T4 GPU. When pretrained on Objects365, D-FINE-L/X attains 57.1%/59.3% AP, surpassing all existing real-time detectors.

Closes the second half and thus, the complete issue #2271

Results in Action of KerasHub's D-FINE

Model Predictions Numerics Matching (Visit the Colab notebook for the complete results)

Colab Notebook

D-FINE: Complete Workflow with Predictions and Numerics Matching

Checklist

  • I have added all the necessary unit tests for my change.
  • I have verified that my change does not break existing code and works with all backends (TensorFlow, JAX, and PyTorch).
  • My PR is based on the latest changes of the main branch (if unsure, rebase the code).
  • I have followed the Keras Hub Model contribution guidelines in making these changes.
  • I have followed the Keras Hub API design guidelines in making these changes.
  • I have signed the Contributor License Agreement.

@harshaljanjani harshaljanjani self-assigned this Jul 9, 2025
@harshaljanjani
Copy link
Collaborator Author

The tests fail due to pending reviews on the HGNetV2 dependency. Once it is merged, the D-FINE PR will be open for review, and the tests will pass, as demonstrated in the notebook.

@divyashreepathihalli
Copy link
Collaborator

/gemini review

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the D-FINE model to KerasHub, including its architecture, layers, tests, and a checkpoint conversion script. The implementation is comprehensive and well-structured. I've provided a few suggestions to improve code clarity, maintainability, and correctness. Overall, this is a solid contribution.

@harshaljanjani harshaljanjani marked this pull request as ready for review July 12, 2025 19:25
@harshaljanjani
Copy link
Collaborator Author

harshaljanjani commented Jul 13, 2025

@divyashreepathihalli @mattdangerw D-FINE is ready for its first round of reviews!
As discussed, we're leaving out the task model from the scope of this PR, given the sheer volume of the code, since the task model is a 1000+ LOC effort in itself. I've not only covered the numerics check, but also the examples we'd add to the quickstart notebook on Kaggle once merged, in the Colab notebook linked in the PR description!

Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Nice work. Just some initial comments.

In general, now that this is up and working let's see if we can find anywhere to cut complexity if we can. Anything we can do to same lines of code (without playing code golf) will probably help keep this maintainable for the future.

@harshaljanjani
Copy link
Collaborator Author

Thanks for the reviews @mattdangerw. Yeah let's definitely cut down the complexity wherever possible for maintainability, I'll look into it!

@harshaljanjani
Copy link
Collaborator Author

@mattdangerw Could you please check if all your comments have been addressed when you have the time, thanks a lot!

@sachinprasadhs sachinprasadhs moved this to In Progress in KerasHub Jul 16, 2025
@harshaljanjani
Copy link
Collaborator Author

@mattdangerw @divyashreepathihalli
Good day, just a gentle reminder for a timely review of this PR, thank you!

Copy link
Collaborator

@sachinprasadhs sachinprasadhs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!
I have added few comments, mainly focusing on our standard design process.

@harshaljanjani
Copy link
Collaborator Author

@sachinprasadhs
Thanks for taking the time to review. I've addressed the concerns to the best of my ability.

Copy link
Collaborator

@sachinprasadhs sachinprasadhs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing all the comments, this looks better now. Just one place you might have missed to make change, added comment.

@sachinprasadhs sachinprasadhs added the kokoro:force-run Runs Tests on GPU label Jul 25, 2025
@kokoro-team kokoro-team removed the kokoro:force-run Runs Tests on GPU label Jul 25, 2025
@harshaljanjani
Copy link
Collaborator Author

@sachinprasadhs Resolved, thanks!

@harshaljanjani harshaljanjani added the kokoro:force-run Runs Tests on GPU label Jul 26, 2025
@kokoro-team kokoro-team removed the kokoro:force-run Runs Tests on GPU label Jul 26, 2025
Copy link
Collaborator

@divyashreepathihalli divyashreepathihalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR Harshal! left a few comments1

@harshaljanjani
Copy link
Collaborator Author

Good day @divyashreepathihalli!
Thanks for taking the time to review, I've left responses in a few comments and made changes to resolve them to the best of my ability!

Copy link
Collaborator

@sachinprasadhs sachinprasadhs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending changes #2318 (comment) as discussed with team.
cc: @divyashreepathihalli

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

5 participants