-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: New TableFormer model weights #1100
Conversation
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
🟢 Require two reviewer for test updatesWonderful, this rule succeeded.When test data is updated, we require two reviewers
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tests don't break with new TF weights, so no need to update them!
(Edit: it's because tests run FAST configuration, not ACCURATE, I'm changing that...)
Switched TF model to Accurate by default (in API and CLI), and re-generated tests |
Signed-off-by: Christoph Auer <[email protected]>
Signed-off-by: Maksym Lysak <[email protected]>
… by default Signed-off-by: Maksym Lysak <[email protected]>
0390576
to
43c53a0
Compare
@maxmnemonic after release, let's push the performance details in the docs. |
Checklist:
Evaluation of FinTabNet
FinTabNet table_structure TEDS_struct-with-text:
mean=0.90 median=0.92 std=0.09
vs mean=0.89 median=0.91 std=0.10
FinTabNet table_structure TEDS_struct-only:
mean=0.90 median=0.92 std=0.08
vs mean=0.90 median=0.92 std=0.09
Evaluation of PubTabNet
PubTabNet table_structure TEDS_struct-with-text:
mean=0.85 median=0.90 std=0.17
vs mean=0.80 median=0.83 std=0.13
PubTabNet table_structure TEDS_struct-only:
mean=0.86 median=0.90 std=0.15
vs mean=0.81 median=0.84 std=0.12
Evaluation of Pub1M
Pub1M table_structure TEDS_struct-with-text:
mean=0.82 median=0.87 std=0.18
vs mean=0.82 median=0.86 std=0.17
Pub1M table_structure TEDS_struct-only:
mean=0.84 median=0.88 std=0.17
vs mean=0.84 median=0.87 std=0.16