Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dispatch prompt template variations on the model name #1

Merged
merged 5 commits into from
Jul 29, 2024

Conversation

rlouf
Copy link
Member

@rlouf rlouf commented Jul 26, 2024

When using large language models for a given task, the output structure is fixed while the details of the prompt are often model-dependent. This is especially true for chat formats and tool use. Switching models in Outlines thus requires two changes:

  1. The model name in outlines.models.provider
  2. The name of the prompt template

We can mitigate the potential errors (it's easy to forget to change the name of the function!) by allowing users to register prompt variations based on model names. This feature works similarly to functools.singledispatch except dispatch happens on the model name and not the type:

import prompts


@prompts.template
def simple_prompt(query: str):
    """{{ query }}"""


@simple_prompt.register("microsoft/Phi-3-medium-128k-instruct")
def simple_prompt_name(query: str):
    """Phi-3: {{ query }}"""

So that

prompt = simple_prompt("question")
print(prompt)
# question

print(simple_prompt["gpt2"]("question"))
# question

print(simple_prompt["microsoft/Phi-3-medium-128k-instruct"]("question"))
# Phi-3: question

And when calling outlines.generate.text(model)(prompt) the prompt that is specific to model is used.

@rlouf rlouf added the enhancement New feature or request label Jul 26, 2024
@rlouf rlouf force-pushed the dispatch-model-name branch from c8e9d36 to 43a9682 Compare July 26, 2024 12:32
@rlouf rlouf force-pushed the dispatch-model-name branch from 7d51e29 to 466831e Compare July 26, 2024 12:57
@rlouf rlouf force-pushed the dispatch-model-name branch from 466831e to b7694f1 Compare July 29, 2024 10:10
@rlouf rlouf force-pushed the dispatch-model-name branch from ee27f2e to 1f12d66 Compare July 29, 2024 10:40
@rlouf rlouf force-pushed the dispatch-model-name branch from 3a9a926 to be22a05 Compare July 29, 2024 10:59
@rlouf rlouf merged commit 0e5a56c into dottxt-ai:master Jul 29, 2024
5 checks passed
@rlouf rlouf deleted the dispatch-model-name branch July 29, 2024 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant