Add evaluation framework for comparing model outputs across providers

[coderleeon]

Problem

Currently, Thunderbolt allows users to plug in multiple model providers, but there is no standardized way to evaluate output quality across them.

Why this matters

In enterprise settings, choosing between models requires measurable signals like:

• correctness
• latency
• hallucination rate
• cost efficiency

Proposed Improvement

Introduce an evaluation layer that:

• runs predefined prompts across models
• logs structured outputs
• computes metrics (accuracy, reasoning quality, latency)
• enables side-by-side comparison

Impact

This would significantly improve decision-making for model selection and align Thunderbolt with production AI workflows.

[cjroth]

Take a look at https://github.com/thunderbird/thunderbolt/tree/main/src/ai/eval