graph LR
Configuration_Management["Configuration Management"]
LLM_Interaction_Execution["LLM Interaction & Execution"]
Core_Evaluation_Engine["Core Evaluation Engine"]
Data_Processing_Grading["Data Processing & Grading"]
Data_Reporting_Tools["Data & Reporting Tools"]
Core_Evaluation_Engine -- "loads settings from" --> Configuration_Management
Core_Evaluation_Engine -- "initiates queries to" --> LLM_Interaction_Execution
Core_Evaluation_Engine -- "exchanges data with" --> Data_Processing_Grading
Data_Reporting_Tools -- "prepares input for" --> Core_Evaluation_Engine
Data_Reporting_Tools -- "consumes evaluation results from" --> Core_Evaluation_Engine
Data_Reporting_Tools -- "consumes grading results from" --> Data_Processing_Grading
click LLM_Interaction_Execution href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/matharena/LLM_Interaction_Execution.md" "Details"
click Core_Evaluation_Engine href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/matharena/Core_Evaluation_Engine.md" "Details"
click Data_Processing_Grading href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/matharena/Data_Processing_Grading.md" "Details"
click Data_Reporting_Tools href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/matharena/Data_Reporting_Tools.md" "Details"
The MathArena project is designed to evaluate Large Language Models (LLMs) on mathematical problems. The core flow involves loading configurations, running evaluation tasks, interacting with LLMs, processing their outputs, and generating reports.
Manages the loading, parsing, and validation of all system configurations, including LLM model parameters, competition details, and evaluation settings.
Related Classes/Methods:
LLM Interaction & Execution [Expand]
Provides a unified interface for interacting with various Large Language Model APIs and offers a secure sandbox for executing LLM-generated code.
Related Classes/Methods:
Core Evaluation Engine [Expand]
The central orchestrator of the evaluation workflow, managing problem loading, initiating LLM queries, coordinating output processing, and implementing advanced prompting strategies like Chain-of-Thought.
Related Classes/Methods:
Data Processing & Grading [Expand]
Extracts, normalizes, and validates answers from diverse LLM output formats, then performs automated scoring against ground truth or predefined criteria.
Related Classes/Methods:
Data & Reporting Tools [Expand]
A suite of scripts and utilities for preparing and managing competition data, as well as for post-evaluation analysis, statistical reporting, and visualization of results.
Related Classes/Methods: