Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update examples/FinanceBench with LM-assisted Evaluation #213

Merged
merged 26 commits into from
May 8, 2024
Merged

update examples/FinanceBench with LM-assisted Evaluation #213

merged 26 commits into from
May 8, 2024

Conversation

TheVinhLuong102
Copy link
Collaborator

No description provided.

@TheVinhLuong102 TheVinhLuong102 self-assigned this May 8, 2024
@TheVinhLuong102
refactor examples/FinanceBench/eval module
a14c8c7
@TheVinhLuong102
rename examples/FinanceBench/util.log_qa_and_eval_correctness_and_upd…
0a2c9e6 
…ate_output_file(...) decorator
@TheVinhLuong102
refactor examples/FinanceBench/eval.eval_correctness(...) and make it…
63dcb0f 
… return bool
@TheVinhLuong102
refactor examples/FinanceBench/eval script
b471bf5
@TheVinhLuong102
enhance examples/FinanceBench/util
6ca2d97
@TheVinhLuong102
mark financebench_id_00283 with evaluator-unreliable
60d2cd9
@TheVinhLuong102
Merge branch 'main' of https://GitHub.com/Aitomatic/OpenSSA into exam…
8884030 
…ples/FinanceBench
@TheVinhLuong102
enhance examples/FinanceBench/eval script with human-evaluation optio…
22a680e 
…n for difficult cases
@TheVinhLuong102
minor edit in examples/FinanceBench/eval.eval_correctness(...)
84b969d
@TheVinhLuong102
update financebench_id_03473 ground truth
56bdf42
@TheVinhLuong102
enhance examples/FinanceBench/eval.eval_correctness(...) human-evalua…
fa6f904 
…tion option
@TheVinhLuong102
enhance reported output formatting in examples/FinanceBench/eval script
6cdc342
@TheVinhLuong102
refactor examples/FinanceBench/eval script
569de35
@TheVinhLuong102
refactor examples/FinanceBench/eval script
224a3a2
@TheVinhLuong102
factor out examples/FinanceBench/eval.eval_all()
c67ac62
@TheVinhLuong102
Revert "enhance examples/FinanceBench/util"
1d58a45 
This reverts commit 6ca2d97.
@TheVinhLuong102
Revert "rename examples/FinanceBench/util.log_qa_and_eval_correctness…
80e5d9d 
…_and_update_output_file(...) decorator"

This reverts commit 0a2c9e6.
@TheVinhLuong102
enhance examples/FinanceBench/util.enable_batch_qa_and_eval(...) deco…
53678f9 
…rator
@TheVinhLuong102
refactor examples/FinanceBench/eval module
fa362ff
@TheVinhLuong102 TheVinhLuong102 changed the title update examples/FinanceBench with various Plan Creation & Plan Execution combinations update examples/FinanceBench May 8, 2024
@TheVinhLuong102 TheVinhLuong102 changed the title update examples/FinanceBench update examples/FinanceBench with LM-assisted Evaluation May 8, 2024
@TheVinhLuong102 TheVinhLuong102 merged commit 01939e0 into main May 8, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@TheVinhLuong102 TheVinhLuong102

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@TheVinhLuong102