Skip to content

Conversation

@mam10eks
Copy link
Contributor

The test queries for the trec tot task are now available, and this pull request adds them in a similar way to how they are added for dev and train (no qrels are available yet, we can also wait to merge this until the qrels are available).

@seanmacavaney
Copy link
Collaborator

Thanks! A few points:

  • Can we rename it to trec-tot/2023/test? This is more aligned with the typical naming convention.
  • Are the sentence_annotations of a standard structure? If so, they should be represented as NamedTuples instead of dicts.
  • Can you add a metadata record and documentation for this dataset? (This is why the tests failed.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants