Skip to content
@prometheus-eval

prometheus-eval

Codebase to inference and train foundation models specialized on evaluating other foundation models
README.md

We train language models specialized in evaluating other language models and optimize evaluation pipelines!

Repositories

Below are our key projects, with links to their repositories and related publications:

Repository Description Paper
prometheus-eval A repository for evaluating LLMs in generation tasks. Supports Prometheus 2, GPT-4, and others. Link
prometheus An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Link
prometheus-vision An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Link

Popular repositories Loading

  1. prometheus-eval prometheus-eval Public

    Evaluate your LLM's response with Prometheus and GPT4 đź’Ż

    Python 949 54

  2. prometheus prometheus Public

    [ICLR 2024 & NeurIPS 2023 WS] An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on a customized score ru…

    Python 301 18

  3. prometheus-vision prometheus-vision Public

    [ACL 2024 Findings & ICLR 2024 WS] An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on customized scor…

    Python 71 7

  4. scaling-evaluation-compute scaling-evaluation-compute Public

    Repository for "Scaling Evaluation-time Compute with Reasoning Models as Process Evaluators"

    12

  5. .github .github Public

    Organization README for prometheus-eval

  6. prometheus-eval.github.io prometheus-eval.github.io Public

    Documentation and blogposts for Prometheus

    1

Repositories

Showing 7 of 7 repositories

Top languages

Loading…

Most used topics

Loading…