| title | Crucible Example Generated Reports - Reveal |
|---|---|
| author | Anonymous |
Crucible rates the "Goose" report higher than "Bear".
Bear : Uses only the Crucibe base system (SupportedAnswerExtractorRequest prompt)
Goose : Uses Crucible base along with both the guessed prompt and guessed nugget subversion probe
One can see how the citation support filter removes sentences where the extracted passage is missing critical entities (e.g. Bayer vs a company). This increases the user's trust in the faithfulness of the report.
Gold nuggets are even-numbered. Automatically generated system nuggets are odd-numbered.
We note that the automatically generated nuggets include a diverse set of questions about numbers, such as not just the number of lawsuits, but differentiating which are lost, pending, or settled, and at which court level they are currently pending.