outlines-func-call-gorilla-leaderboard-experiment

Small models can be guided using library like outlines.

This make them MUCH MUCH more reliable for function calling. This unlocks a new world of possibilities.

This repo tries to evaluate the improvement provided by outlines for small models on the gorilla leaderboard.

---

install

pip install -r requirements.txt

then open the main notebook main.ipynb

---

If you use this repo to improve our knowledge about the function calling performances of models, please report your results publicly so we can progress together.

For citations

Connesson, Rémi. (Apr 2024). Outlines Function Call Gorilla Leaderboard Experiment. GitHub. https://github.com/remiconnesson/outlines-func-call-gorilla-leaderboard-experiment/tree/main.

or

@software{connesson_2024_outlines_gorilla,
  author       = {Connesson, Rémi},
  title        = {Outlines Function Call Gorilla Leaderboard Experiment},
  month        = apr,
  year         = 2024,
  publisher    = {GitHub},
  version      = {1.0.0},
  url          = {https://github.com/remiconnesson/outlines-func-call-gorilla-leaderboard-experiment/tree/main}
}