Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove check for public and private scores to have same keys when using custom metrics #52

Open
ktrapeznikov opened this issue Feb 19, 2025 · 1 comment
Labels

Comments

@ktrapeznikov
Copy link

We are setting up a competition where public and private metrics will have different names. For example, we may have different sub-splits or categories of the data reported on the leaderboards. The number and the name of these metrics will be different for public and private.

I see this constraint is checked here:

if evaluation["public_score"].keys() != evaluation["private_score"].keys():

@ktrapeznikov ktrapeznikov changed the title Remove constrain for public and private scores have same keys when using custom metrics Remove check for public and private scores to have same keys when using custom metrics Feb 20, 2025
Copy link

github-actions bot commented Mar 7, 2025

This issue is stale because it has been open for 15 days with no activity.

@github-actions github-actions bot added the stale label Mar 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant