update examples/FinanceBench with LM-assisted Evaluation#213

Merged

TheVinhLuong102 merged 26 commits intomainfrom examples/FinanceBench

May 8, 2024

Commits on May 6, 2024

update examples/Planning-and-Reasoning notebook to overcome current-working-directory bug in running Jupyter Notebooks in VSCode
TheVinhLuong102
committed
Merge branch 'main' of https://GitHub.com/Aitomatic/OpenSSA into examples/FinanceBench
TheVinhLuong102
committed

Commits on May 8, 2024

Merge branch 'main' of https://GitHub.com/Aitomatic/OpenSSA into examples/FinanceBench
TheVinhLuong102
committed
refactor examples/FinanceBench/eval module
TheVinhLuong102
committed
refactor examples/FinanceBench/eval module
TheVinhLuong102
committed
rename examples/FinanceBench/util.log_qa_and_eval_correctness_and_update_output_file(...) decorator
TheVinhLuong102
committed
refactor examples/FinanceBench/eval.eval_correctness(...) and make it return bool
TheVinhLuong102
committed
refactor examples/FinanceBench/eval script
TheVinhLuong102
committed
enhance examples/FinanceBench/util
TheVinhLuong102
committed
mark financebench_id_00283 with evaluator-unreliable
TheVinhLuong102
committed
Merge branch 'main' of https://GitHub.com/Aitomatic/OpenSSA into examples/FinanceBench
TheVinhLuong102
committed
enhance examples/FinanceBench/eval script with human-evaluation option for difficult cases
TheVinhLuong102
committed
minor edit in examples/FinanceBench/eval.eval_correctness(...)
TheVinhLuong102
committed
update financebench_id_03473 ground truth
TheVinhLuong102
committed
enhance examples/FinanceBench/eval.eval_correctness(...) human-evaluation option
TheVinhLuong102
committed
enhance reported output formatting in examples/FinanceBench/eval script
TheVinhLuong102
committed
refactor examples/FinanceBench/eval script
TheVinhLuong102
committed
refactor examples/FinanceBench/eval script
TheVinhLuong102
committed
factor out examples/FinanceBench/eval.eval_all()
TheVinhLuong102
committed
Revert "enhance examples/FinanceBench/util"
TheVinhLuong102
committed
Revert "rename examples/FinanceBench/util.log_qa_and_eval_correctness_and_update_output_file(...) decorator"
TheVinhLuong102
committed
enhance examples/FinanceBench/util.enable_batch_qa_and_eval(...) decorator
TheVinhLuong102
committed
refactor examples/FinanceBench/eval module
TheVinhLuong102
committed
refactor examples/FinanceBench/util.enable_batch_qa_and_eval(...) decorator
TheVinhLuong102
committed
refactor examples/FinanceBench/eval.eval_all(...)
TheVinhLuong102
committed
enhance examples/FinanceBench/util.enable_batch_qa_and_eval(...) decorator
TheVinhLuong102
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update examples/FinanceBench with LM-assisted Evaluation#213

Commits on May 6, 2024

update examples/Planning-and-Reasoning notebook to overcome current-working-directory bug in running Jupyter Notebooks in VSCode

Merge branch 'main' of https://GitHub.com/Aitomatic/OpenSSA into examples/FinanceBench

Commits on May 8, 2024

Merge branch 'main' of https://GitHub.com/Aitomatic/OpenSSA into examples/FinanceBench

refactor examples/FinanceBench/eval module

refactor examples/FinanceBench/eval module

rename examples/FinanceBench/util.log_qa_and_eval_correctness_and_update_output_file(...) decorator

refactor examples/FinanceBench/eval.eval_correctness(...) and make it return bool

refactor examples/FinanceBench/eval script

enhance examples/FinanceBench/util

mark financebench_id_00283 with evaluator-unreliable

Merge branch 'main' of https://GitHub.com/Aitomatic/OpenSSA into examples/FinanceBench

enhance examples/FinanceBench/eval script with human-evaluation option for difficult cases

minor edit in examples/FinanceBench/eval.eval_correctness(...)

update financebench_id_03473 ground truth

enhance examples/FinanceBench/eval.eval_correctness(...) human-evaluation option

enhance reported output formatting in examples/FinanceBench/eval script

refactor examples/FinanceBench/eval script

refactor examples/FinanceBench/eval script

factor out examples/FinanceBench/eval.eval_all()

Revert "enhance examples/FinanceBench/util"

Revert "rename examples/FinanceBench/util.log_qa_and_eval_correctness_and_update_output_file(...) decorator"

enhance examples/FinanceBench/util.enable_batch_qa_and_eval(...) decorator

refactor examples/FinanceBench/eval module

refactor examples/FinanceBench/util.enable_batch_qa_and_eval(...) decorator

refactor examples/FinanceBench/eval.eval_all(...)

enhance examples/FinanceBench/util.enable_batch_qa_and_eval(...) decorator