Enforce working examples in docstrings and documentation

README.md

@@ -49,10 +49,11 @@ logfire auth
 
 Here's a simple manual tracing (aka logging) example:
 
-```python
-import logfire
+```python skip-run="true" skip-reason="blocking"
 from datetime import date
 
+import logfire
+
 logfire.configure()
 logfire.info('Hello, {name}!', name='world')
 
@@ -68,21 +69,24 @@ with logfire.span('Asking the user their {question}', question='age'):
 
 Or you can also avoid manual instrumentation and instead integrate with [lots of popular packages](https://logfire.pydantic.dev/docs/integrations/), here's an example of integrating with FastAPI:
 
-```py
-import logfire
-from pydantic import BaseModel
+```py skip-run="true" skip-reason="global-instrumentation"
 from fastapi import FastAPI
+from pydantic import BaseModel
+
+import logfire
 
 app = FastAPI()
 
 logfire.configure()
 logfire.instrument_fastapi(app)
 # next, instrument your database connector, http library etc. and add the logging handler
 
+
 class User(BaseModel):
     name: str
     country_code: str
 
+
 @app.post('/')
 async def add_user(user: User):
     # we would store the user here

docs/concepts.md

@@ -43,6 +43,7 @@ In this example:
 
 ```py
 from pathlib import Path
+
 import logfire
 
 logfire.configure()
@@ -71,7 +72,7 @@ In this example:
 2. The user input is captured in the terminal
 3. `dob` (date of birth) is displayed in the span. Logfire calculates the age from the `dob` and displays age in the debug message
 
-```py
+```py skip-run="true" skip-reason="non-deterministic"
 from datetime import date
 
 import logfire
@@ -116,6 +117,7 @@ low‑overhead signal about the overall health and performance of your services.
 
 ```python
 import time
+
 import logfire
 
 logfire.configure()

docs/guides/onboarding-checklist/add-auto-tracing.md

@@ -10,7 +10,7 @@ For example, suppose all your code lives in the `app` package, e.g. `app.main`,
 Instead of starting your application with `python app/main.py`,
 you could create another file outside of the `app` package, e.g:
 
-```py title="main.py"
+```py title="main.py" skip="true" skip-reason="intentional-error"
 import logfire
 
 logfire.configure()
@@ -52,7 +52,7 @@ This function will be called with an [`AutoTraceModule`][logfire.AutoTraceModule
 `filename` attributes. For example, this should trace all modules that aren't part of the standard library or
 third-party packages in a typical Python installation:
 
-```py
+```py skip="true" skip-reason="incomplete"
 import pathlib
 
 import logfire
@@ -72,7 +72,7 @@ logfire.install_auto_tracing(should_trace, min_duration=0)
 Once you've selected which modules to trace, you probably don't want to trace *every* function in those modules.
 To exclude a function from auto-tracing, add the [`no_auto_trace`][logfire.no_auto_trace] decorator to it:
 
-```py
+```py skip="true" skip-reason="incomplete"
 import logfire
 
 @logfire.no_auto_trace

docs/guides/onboarding-checklist/add-manual-tracing.md

@@ -2,7 +2,7 @@
 
 Here's a simple example of using Logfire:
 
-```python
+```python skip-run="true" skip-reason="blocking"
 import time
 
 import logfire
@@ -53,7 +53,7 @@ A _trace_ is a tree of spans/logs sharing the same root. Whenever you create a n
 
 Spans and logs can have structured data attached to them, e.g:
 
-```python
+```python skip="true" skip-reason="incomplete"
 logfire.info('Hello', name='world')
 ```
 
@@ -67,7 +67,7 @@ Both spans and logs can have attributes containing arbitrary values which will b
 
 Sometimes it's useful to attach an attribute to a span after it's been created but before it's finished. You can do this by calling the `span.set_attribute` method:
 
-```python
+```python skip="true" skip-reason="incomplete"
 with logfire.span('Calculating...') as span:
     result = 1 + 2
     span.set_attribute('result', result)
@@ -95,7 +95,7 @@ Here you can see that:
 
 You can also set `span.message` after a span is started but before it's finished, e.g:
 
-```python
+```python skip="true" skip-reason="incomplete"
 with logfire.span('Calculating...') as span:
     result = 1 + 2
     span.message = f'Calculated: {result}'
@@ -105,14 +105,14 @@ You could use `message` to filter for related records, e.g. `message like 'Hello
 
 To allow efficiently filtering for related records, span names should be _low cardinality_, meaning they shouldn't vary too much. For example, this would be bad:
 
-```python
+```python skip="true" skip-reason="incomplete"
 name = get_username()
 logfire.info('Hello ' + name, name=name)
 ```
 
 because now the `span_name` column will have a different value for every username. But this would be fine:
 
-```python
+```python skip="true" skip-reason="incomplete"
 word = 'Goodbye' if leaving else 'Hello'
 logfire.info(word + ' {name}', name=name)
 ```
@@ -121,7 +121,7 @@ because now the `span_name` column will only have two values (`'Goodbye {name}'`
 
 You can use the `_span_name` argument when you want the span name to be different from the message template, e.g:
 
-```python
+```python skip="true" skip-reason="incomplete"
 logfire.info('Hello {name}', name='world', _span_name='Hello')
 ```
 
@@ -131,13 +131,13 @@ This will set the `span_name` to `'Hello'` and the `message` to `'Hello world'`.
 
 Instead of this:
 
-```python
+```python skip="true" skip-reason="incomplete"
 logfire.info('Hello {name}', name=name)
 ```
 
 it's much more convenient to use an f-string to avoid repeating `name` three times:
 
-```python
+```python skip="true" skip-reason="incomplete"
 logfire.info(f'Hello {name}')
 ```
 
@@ -156,7 +156,7 @@ Contrary to the previous section, this _will_ work well in Python 3.11+ because
 
 The `logfire.span` context manager will automatically record any exceptions that cause it to exit, e.g:
 
-```python
+```python skip-run="true" skip-reason="exception-demo"
 import logfire
 
 logfire.configure()
@@ -171,7 +171,7 @@ If you click on the span in the Live view, the panel on the right will have an '
 
 Exceptions which are caught and not re-raised will not be recorded, e.g:
 
-```python
+```python skip="true" skip-reason="incomplete"
 with logfire.span('This is a span'):
     try:
         raise ValueError('This is an acceptable error not worth recording')
@@ -181,7 +181,7 @@ with logfire.span('This is a span'):
 
 If you want to record a handled exception, use the [`span.record_exception`][logfire.LogfireSpan.record_exception] method:
 
-```python
+```python skip="true" skip-reason="incomplete"
 with logfire.span('This is a span') as span:
     try:
         raise ValueError('Catch this error, but record it')
@@ -191,7 +191,7 @@ with logfire.span('This is a span') as span:
 
 Alternatively, if you only want to log exceptions without creating a span for the normal case, you can use [`logfire.exception`][logfire.Logfire.exception]:
 
-```python
+```python skip="true" skip-reason="incomplete"
 try:
     raise ValueError('This is an error')
 except ValueError:
@@ -204,15 +204,15 @@ except ValueError:
 
 Often you want to wrap a whole function in a span. Instead of doing this:
 
-```python
+```python skip="true" skip-reason="incomplete"
 def my_function(x, y):
     with logfire.span('my_function', x=x, y=y):
         ...
 ```
 
 you can use the [`@logfire.instrument`][logfire.Logfire.instrument] decorator:
 
-```python
+```python skip="true" skip-reason="incomplete"
 @logfire.instrument()
 def my_function(x, y):
     ...
@@ -225,7 +225,7 @@ The default span name will be something like `Calling module_name.my_function`.
 You can pass an alternative span name as the first argument to `instrument`, and it can even be a template
 into which arguments will be formatted, e.g:
 
-```python
+```python skip="true" skip-reason="incomplete"
 @logfire.instrument('Applying my_function to {x=} and {y=}')
 def my_function(x, y):
     ...
@@ -255,7 +255,7 @@ To log a message with a variable level you can use `logfire.log`, e.g. `logfire.
 
 Spans are level `info` by default. You can change this with the `_level` argument, e.g. `with logfire.span('This is a debug span', _level='debug'):`. You can also change the level after the span has started but before it's finished with [`span.set_level`][logfire.LogfireSpan.set_level], e.g:
 
-```python
+```python skip="true" skip-reason="incomplete"
 with logfire.span('Doing a thing') as span:
     success = do_thing()
     if not success:

docs/guides/onboarding-checklist/add-metrics.md

@@ -18,6 +18,7 @@ import logfire
 # Create a counter metric
 messages_sent = logfire.metric_counter('messages_sent')
 
+
 # Increment the counter
 def send_message():
     messages_sent.add(1)
@@ -39,11 +40,7 @@ To create a counter metric, use the [`logfire.metric_counter`][logfire.Logfire.m
 ```py
 import logfire
 
-counter = logfire.metric_counter(
-    'exceptions',
-    unit='1',  # (1)!
-    description='Number of exceptions caught'
-)
+counter = logfire.metric_counter('exceptions', unit='1', description='Number of exceptions caught')  # (1)!
 
 try:
     raise Exception('oops')
@@ -72,11 +69,7 @@ To create a histogram metric, use the [`logfire.metric_histogram`][logfire.Logfi
 ```py
 import logfire
 
-histogram = logfire.metric_histogram(
-    'request_duration',
-    unit='ms',  # (1)!
-    description='Duration of requests'
-)
+histogram = logfire.metric_histogram('request_duration', unit='ms', description='Duration of requests')  # (1)!
 
 for duration in [10, 20, 30, 40, 50]:
     histogram.record(duration)
@@ -104,15 +97,13 @@ To create an up-down counter metric, use the [`logfire.metric_up_down_counter`][
 ```py
 import logfire
 
-active_users = logfire.metric_up_down_counter(
-    'active_users',
-    unit='1',  # (1)!
-    description='Number of active users'
-)
+active_users = logfire.metric_up_down_counter('active_users', unit='1', description='Number of active users')  # (1)!
+
 
 def user_logged_in():
     active_users.add(1)
 
+
 def user_logged_out():
     active_users.add(-1)
 ```
@@ -140,11 +131,8 @@ To create a gauge metric, use the [`logfire.metric_gauge`][logfire.Logfire.metri
 ```py
 import logfire
 
-temperature = logfire.metric_gauge(
-    'temperature',
-    unit='°C',
-    description='Temperature'
-)
+temperature = logfire.metric_gauge('temperature', unit='C', description='Temperature')
+
 
 def set_temperature(value: float):
     temperature.set(value)
@@ -164,29 +152,25 @@ To create a counter callback metric, use the [`logfire.metric_counter_callback`]
 ```py
 from typing import Iterable
 
-import logfire
 from opentelemetry.metrics import CallbackOptions, Observation
 
+import logfire
+
 
 def cpu_time_callback(options: CallbackOptions) -> Iterable[Observation]:
     observations = []
-    with open("/proc/stat") as procstat: # (1)!
+    with open('/proc/stat') as procstat:  # (1)!
         procstat.readline()  # skip the first line
         for line in procstat:
-            if not line.startswith("cpu"):
+            if not line.startswith('cpu'):
                 break
             cpu, user_time, nice_time, system_time = line.split()
-            observations.append(
-                Observation(int(user_time) // 100, {"cpu": cpu, "state": "user"})
-            )
-            observations.append(
-                Observation(int(nice_time) // 100, {"cpu": cpu, "state": "nice"})
-            )
-            observations.append(
-                Observation(int(system_time) // 100, {"cpu": cpu, "state": "system"})
-            )
+            observations.append(Observation(int(user_time) // 100, {'cpu': cpu, 'state': 'user'}))
+            observations.append(Observation(int(nice_time) // 100, {'cpu': cpu, 'state': 'nice'}))
+            observations.append(Observation(int(system_time) // 100, {'cpu': cpu, 'state': 'system'}))
     return observations
 
+
 logfire.metric_counter_callback(
     'system.cpu.time',
     unit='s',
@@ -209,23 +193,24 @@ To create a gauge callback metric, use the [`logfire.metric_gauge_callback`][log
 ```py
 from typing import Iterable
 
-import logfire
 from opentelemetry.metrics import CallbackOptions, Observation
 
+import logfire
+
 
 def get_temperature(room: str) -> float:
-    ...
+    return 22.0
 
 
 def temperature_callback(options: CallbackOptions) -> Iterable[Observation]:
-    for room in ["kitchen", "living_room", "bedroom"]:
+    for room in ['kitchen', 'living_room', 'bedroom']:
         temperature = get_temperature(room)
-        yield Observation(temperature, {"room": room})
+        yield Observation(temperature, {'room': room})
 
 
 logfire.metric_gauge_callback(
     'temperature',
-    unit='°C',
+    unit='C',
     callbacks=[temperature_callback],
     description='Temperature',
 )
@@ -243,12 +228,13 @@ To create an up-down counter callback metric, use the
 ```py
 from typing import Iterable
 
-import logfire
 from opentelemetry.metrics import CallbackOptions, Observation
 
+import logfire
+
 
 def get_active_users() -> int:
-    ...
+    return 5
 
 
 def active_users_callback(options: CallbackOptions) -> Iterable[Observation]: