Use Better Models, Better Prompts in Docs (#619)

Some of the results in the docs are difficult to reproduce. Prompt
engineering and better model recommendations are applied here to
alleviate that.

fixes #586

- Update docs: never recommend `mistralai/Mistral-7B-v0.1`, always
prefer `WizardLM/WizardMath-7B-V1.1` for math above kindergarten level
and `mistralai/Mistral-7B-Instruct-v0.2` for everything else.
- Smoke tested all `README.md` and `quickstart.md` examples. Replaced
model with `mistralai/Mistral-7B-Instruct-v0.2`, generation quality was
equal or better in all cases.
- fixes #612 (see
#612 (comment))
- don't force `device="cuda"` in any `outlines.models.transformers`
- fix arithmetic grammar to not have recursion issue (fixes
#580)
---------

Co-authored-by: Andrew Lapp <[email protected]>

Author: lapp0

README.md

@@ -53,7 +53,7 @@ You can reduce the completion to a choice between multiple possibilities:
 ``` python
 import outlines
 
-model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 
 prompt = """You are a sentiment-labelling assistant.
 Is the following review positive or negative?
@@ -73,15 +73,18 @@ You can instruct the model to only return integers or floats:
 ``` python
 import outlines
 
-model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+model = outlines.models.transformers("WizardLM/WizardMath-7B-V1.1")
 
-prompt = "1+1="
+prompt = "<s>result of 9 + 9 = 18</s><s>result of 1 + 2 = "
 answer = outlines.generate.format(model, int)(prompt)
+print(answer)
+# 3
 
 prompt = "sqrt(2)="
-
 generator = outlines.generate.format(model, float)
-answer = generator(prompt)
+answer = generator(prompt, max_tokens=10)
+print(answer)
+# 1.41421356
 ```
 
 ### Efficient regex-guided generation
@@ -93,7 +96,7 @@ hood:
 ``` python
 import outlines
 
-model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 
 prompt = "What is the IP address of the Google DNS servers? "
 
@@ -156,7 +159,7 @@ class Character(BaseModel):
     strength: int
 
 
-model = outlines.models.transformers("mistralai/Mistral-7B-v0.1", device="cuda")
+model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 
 # Construct guided sequence generator
 generator = outlines.generate.json(model, Character, max_tokens=100)
@@ -165,25 +168,15 @@ generator = outlines.generate.json(model, Character, max_tokens=100)
 rng = torch.Generator(device="cuda")
 rng.manual_seed(789001)
 
-sequence = generator("Give me a character description", rng=rng)
-print(sequence)
-# {
-#   "name": "clerame",
-#   "age": 7,
-#   "armor": "plate",
-#   "weapon": "mace",
-#   "strength": 4171
-# }
-
-sequence = generator("Give me an interesting character description", rng=rng)
-print(sequence)
-# {
-#   "name": "piggyback",
-#   "age": 23,
-#   "armor": "chainmail",
-#   "weapon": "sword",
-#   "strength": 0
-# }
+character = generator("Give me a character description", rng=rng)
+
+print(repr(character))
+# Character(name='Anderson', age=28, armor=<Armor.chainmail: 'chainmail'>, weapon=<Weapon.sword: 'sword'>, strength=8)
+
+character = generator("Give me an interesting character description", rng=rng)
+
+print(repr(character))
+# Character(name='Vivian Thr', age=44, armor=<Armor.plate: 'plate'>, weapon=<Weapon.crossbow: 'crossbow'>, strength=125)
 ```
 
 The method works with union types, optional types, arrays, nested schemas, etc. Some field constraints are [not supported yet](https://github.com/outlines-dev/outlines/issues/215), but everything else should work.
@@ -232,9 +225,9 @@ schema = '''{
     }
 }'''
 
-model = outlines.models.transformers("mistralai/Mistral-7B-v0.1", device="cuda")
+model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 generator = outlines.generate.json(model, schema)
-sequence = generator("Give me a character description")
+character = generator("Give me a character description")
 ```
 
 ### Using context-free grammars to guide generation
@@ -245,34 +238,25 @@ Formal grammars rule the world, and Outlines makes them rule LLMs too. You can p
 import outlines
 
 arithmetic_grammar = """
-    ?start: sum
+    ?start: expression
 
-    ?sum: product
-        | sum "+" product   -> add
-        | sum "-" product   -> sub
+    ?expression: term (("+" | "-") term)*
 
-    ?product: atom
-        | product "*" atom  -> mul
-        | product "/" atom  -> div
+    ?term: factor (("*" | "/") factor)*
 
-    ?atom: NUMBER           -> number
-         | "-" atom         -> neg
-         | "(" sum ")"
+    ?factor: NUMBER
+           | "-" factor
+           | "(" expression ")"
 
     %import common.NUMBER
-    %import common.WS_INLINE
-
-    %ignore WS_INLINE
 """
 
-model = outlines.models.transformers("mistralai/Mistral-7B-v0.1", device="cuda")
+model = outlines.models.transformers("WizardLM/WizardMath-7B-V1.1")
 generator = outlines.generate.cfg(model, arithmetic_grammar)
-sequence = generator("Write a formula that returns 5 using only additions and subtractions.")
-
-# It looks like Mistral is not very good at arithmetics :)
+sequence = generator("Alice had 4 apples and Bob ate 2. Write an expression for Alice's apples:")
 
 print(sequence)
-# 1+3-2-4+5-7+8-6+9-6+4-2+3+5-1+1
+# (8-2)
 ```
 
 This was a very simple grammar, and you can use `outlines.generate.cfg` to generate syntactically valid Python, SQL, and much more than this. Any kind of structured text, really. All you have to do is search for "X EBNF grammar" on the web, and take a look at the [Outlines Grammars repository](https://github.com/outlines-dev/grammars).
@@ -288,9 +272,9 @@ import outlines
 def add(a: int, b: int):
     return a + b
 
-model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+model = outlines.models.transformers("WizardLM/WizardMath-7B-V1.1")
 generator = outlines.generate.json(model, add)
-result = generator("Return two integers named a and b respectively. a is odd and b even.")
+result = generator("Return json with two integers named a and b respectively. a is odd and b even.")
 
 print(add(**result))
 # 3
@@ -329,7 +313,7 @@ def labelling(to_label, examples):
     {{ to_label }} //
     """
 
-model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 prompt = labelling("Just awesome", examples)
 answer = outlines.generate.text(model)(prompt, max_tokens=100)
 ```

docs/quickstart.md

@@ -15,7 +15,7 @@ The first step when writing a program with Outlines is to initialize a model. We
 ```python
 import outlines
 
-model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 ```
 
 ### Text generator
@@ -27,7 +27,7 @@ Once the model is initialized you can build a text generator. This generator can
     ```python
     import outlines
 
-    model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+    model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
     generator = outlines.generate.text(model, max_tokens=100)
 
     result = generator("What's 2+2?")
@@ -43,7 +43,7 @@ Once the model is initialized you can build a text generator. This generator can
     ```python
     import outlines
 
-    model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+    model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
     generator = outlines.generate.text(model, max_tokens=100)
 
     stream = generator.stream("What's 2+2?")
@@ -64,7 +64,7 @@ Outlines allows you to do multi-label classification by guiding the model so it
 ```python
 import outlines
 
-model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 generator = outlines.generate.choice(model, ["Blue", "Red", "Yellow"])
 
 color = generator("What is the closest color to Indigo? ")
@@ -96,7 +96,7 @@ Outlines can guide models so that they output valid JSON **100%** of the time. Y
         armor: Armor
         strength: conint(gt=1, lt=100)
 
-    model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+    model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
     generator = outlines.generate.json(model, Character)
 
     character = generator(
@@ -131,7 +131,7 @@ Outlines can guide models so that they output valid JSON **100%** of the time. Y
         "type": "object"
     }"""
 
-    model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")
+    model = outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
     generator = outlines.generate.json(model, schema)
     character = generator(
         "Generate a new character for my awesome game: "
@@ -175,12 +175,12 @@ arithmetic_grammar = """
     %ignore WS_INLINE
 """
 
-model = models.transformers("mistralai/Mistral-7B-v0.1")
+model = models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 generator = generate.cfg(model, arithmetic_grammar, max_tokens=100)
 
-result = generator("Write a series of operations on integers that return the number 5")
+result = generator("Question: How can you write 5*5 using addition?\nAnswer:")
 print(result)
-# 4*5*3*2*1/6*4*3*2*1/2*1*1*1/4*1*1*1/2*1*1*1/2*1*1/2*1*1*5*1/2*2*1*1/2*1*1*6*1*1/2*1*1*1*1*2*1*1*1*1
+# 5+5+5+5+5
 ```
 
 
@@ -189,12 +189,12 @@ EBNF grammars can be cumbersome to write. This is why Outlines provides grammar
 ```python
 from outlines import models, generate, grammars
 
-model = models.transformers("mistralai/Mistral-7B-v0.1")
+model = models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 generator = generate.cfg(model, grammars.arithmetic, max_tokens=100)
 
-result = generator("Write a series of operations on integers that return the number 5 ")
+result = generator("Question: How can you write 5*5 using addition?\nAnswer:")
 print(result)
-# 100-2-75+50-18+27-501.
+# 5+5+5+5+5
 ```
 
 The available grammars are listed [here](https://github.com/outlines-dev/outlines/tree/main/outlines/grammars).
@@ -207,14 +207,14 @@ Slightly simpler, but no less useful, Outlines can generate text that is in the
 ```python
 from outlines import models, generate
 
-model = models.transformers("mistralai/Mistral-7B-v0.1")
+model = models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 
 regex_str = r"((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)"
 generator = generate.regex(model, regex_str)
 
-result = generator("What is the IP address of Google's DNS sever?")
+result = generator("What is the IP address of localhost?\nIP: ")
 print(result)
-# 0.0.0.0
+# 127.0.0.100
 ```
 
 ### Generate a given Python type
@@ -224,7 +224,7 @@ We provide a shortcut to regex-guided generation for simple use cases. Pass a Py
 ```python
 from outlines import models, generate
 
-model = models.transformers("mistralai/Mistral-7B-v0.1")
+model = models.transformers("mistralai/Mistral-7B-Instruct-v0.2")
 generator = generate.format(model, int)
 
 result = generator("What is 2+2?")
@@ -245,15 +245,15 @@ python -m outlines.serve.serve
 This will by default start a server at `http://127.0.0.1:8000` (check what the console says, though)  with the OPT-125M model. If you want to specify another model:
 
 ```python
-python -m outlines.serve.serve --model="mistralai/Mistral-7B-v0.1"
+python -m outlines.serve.serve --model="mistralai/Mistral-7B-Instruct-v0.2"
 ```
 
 You can then query the model in shell by passing a prompt and a [JSON Schema][jsonschema]{:target="_blank"} specification for the structure of the output:
 
 ```bash
-curl http://0.0.0.1:8000 \
+curl http://127.0.0.1:8000/generate \
     -d '{
-        "prompt": "What is the capital of France?",
+        "prompt": "Question: What is a language model? Answer:",
         "schema": {"type": "string"}
         }'
 ```
@@ -329,25 +329,28 @@ Once you are done experimenting with a prompt and an output structure, it is use
 
     @outlines.prompt
     def tell_a_joke(topic):
-        """Tell me a joke about {{ topic }}."
+        """Tell me a joke about {{ topic }}."""
 
     class Joke(BaseModel):
         setup: str
         punchline: str
 
-    fn = outlines.Function(
+    generate_joke = outlines.Function(
         tell_a_joke,
         Joke,
-        "mistralai/Mistral-7B-v0.1"
+        "mistralai/Mistral-7B-Instruct-v0.2"
     )
     ```
 
 === "Call a function"
 
     ```python
-    from .function import fn as joke
+    from .function import generate_joke
 
-    response = joke("baseball")
+    response = generate_joke("baseball")
+
+    # haha
+    # Joke(setup='Why was the baseball in a bad mood?', punchline='Because it got hit around a lot.')
     ```
 
 === "Call a function stored on GitHub"