|
| 1 | +# Nebo |
| 2 | + |
| 3 | +Lightweight observability for Python programs. Decorate your functions with `@nb.fn()`, and nebo automatically infers a DAG from your call graph, captures logs, metrics, inspections, and errors -- all queryable in real time via CLI, MCP tools, or a Rich terminal dashboard. |
| 4 | + |
| 5 | +## Installation |
| 6 | + |
| 7 | +```bash |
| 8 | +pip install nebo |
| 9 | +``` |
| 10 | + |
| 11 | +The CLI entry point is `nb`: |
| 12 | + |
| 13 | +```bash |
| 14 | +nb --help |
| 15 | +``` |
| 16 | + |
| 17 | +## Quick Start |
| 18 | + |
| 19 | +```python |
| 20 | +import nebo as nb |
| 21 | + |
| 22 | +@nb.fn() |
| 23 | +def load_data(path: str = "data.csv") -> list[dict]: |
| 24 | + """Load records from a file.""" |
| 25 | + records = [{"id": i, "value": i * 0.5} for i in range(100)] |
| 26 | + nb.log(f"Loaded {len(records)} records from {path}") |
| 27 | + return records |
| 28 | + |
| 29 | +@nb.fn() |
| 30 | +def transform(records: list[dict]) -> list[dict]: |
| 31 | + """Normalize values.""" |
| 32 | + out = [] |
| 33 | + for r in nb.track(records, name="transforming"): |
| 34 | + out.append({**r, "value": r["value"] / 50.0}) |
| 35 | + nb.log(f"Transformed {len(out)} records") |
| 36 | + nb.log_metric("record_count", float(len(out))) |
| 37 | + return out |
| 38 | + |
| 39 | +@nb.fn() |
| 40 | +def run(): |
| 41 | + """Main pipeline entry point.""" |
| 42 | + records = load_data() |
| 43 | + result = transform(records) |
| 44 | + nb.log(f"Pipeline complete: {len(result)} records") |
| 45 | + return result |
| 46 | + |
| 47 | +if __name__ == "__main__": |
| 48 | + run() |
| 49 | +``` |
| 50 | + |
| 51 | +Running this produces a Rich terminal display showing the DAG, node execution counts, logs, and progress bars. The DAG edges (`run -> load_data`, `load_data -> transform`) are inferred automatically from data flow -- no manual wiring required. |
| 52 | + |
| 53 | +## Core Concepts |
| 54 | + |
| 55 | +### `@nb.fn()` -- Register a function as a DAG node |
| 56 | + |
| 57 | +Every function decorated with `@nb.fn()` becomes a node in the pipeline DAG. Edges are inferred from **data flow**: when a node's return value is passed as an argument to another node, an edge is created from the producer to the consumer. |
| 58 | + |
| 59 | +```python |
| 60 | +@nb.fn() |
| 61 | +def load_data(): |
| 62 | + return [1, 2, 3] |
| 63 | + |
| 64 | +@nb.fn() |
| 65 | +def transform(data): |
| 66 | + return [x * 2 for x in data] |
| 67 | + |
| 68 | +@nb.fn() |
| 69 | +def run(): |
| 70 | + records = load_data() # edge: run -> load_data (no data dependency) |
| 71 | + result = transform(records) # edge: load_data -> transform (data flows from load_data) |
| 72 | + return result |
| 73 | +``` |
| 74 | + |
| 75 | +When a child node receives no node-produced arguments, the edge falls back to the calling parent node. |
| 76 | + |
| 77 | +You can use it in several ways: |
| 78 | + |
| 79 | +```python |
| 80 | +@nb.fn # bare decorator |
| 81 | +@nb.fn() # with parentheses |
| 82 | +@nb.fn(depends_on=[other_fn]) # with explicit dependencies |
| 83 | +@nb.fn(ui={"collapsed": True}) # with per-node UI hints |
| 84 | +``` |
| 85 | + |
| 86 | +### Class Decoration |
| 87 | + |
| 88 | +`@nb.fn()` can be applied to classes. All methods are wrapped with scope tracking, and the class name becomes a visual group in the DAG: |
| 89 | + |
| 90 | +```python |
| 91 | +@nb.fn() |
| 92 | +class Agent: |
| 93 | + def think(self, query): |
| 94 | + nb.log(f"Thinking about: {query}") |
| 95 | + return {"plan": "respond"} |
| 96 | + |
| 97 | + def act(self, plan): |
| 98 | + nb.log(f"Acting on: {plan}") |
| 99 | + return "result" |
| 100 | + |
| 101 | +agent = Agent() |
| 102 | +agent.think("hello") |
| 103 | +agent.act({"plan": "respond"}) |
| 104 | +``` |
| 105 | + |
| 106 | +Methods appear as `Agent.think` and `Agent.act` in the DAG, grouped under `Agent`. |
| 107 | + |
| 108 | +### Lazy Materialization |
| 109 | + |
| 110 | +Nodes only become visible in the DAG when they produce observable output (via `nb.log()`, `nb.log_metric()`, etc.). Functions that run silently are registered internally but don't clutter the visualization. |
| 111 | + |
| 112 | +### `depends_on` -- Explicit dependency declaration |
| 113 | + |
| 114 | +Some dependencies cannot be detected automatically (shared mutable state, class attributes, global variables). Use `depends_on` to declare these explicitly: |
| 115 | + |
| 116 | +```python |
| 117 | +@nb.fn() |
| 118 | +def setup(): |
| 119 | + """Initialize shared resources.""" |
| 120 | + ... |
| 121 | + |
| 122 | +@nb.fn(depends_on=[setup]) |
| 123 | +def process(): |
| 124 | + """Uses resources initialized by setup.""" |
| 125 | + ... |
| 126 | +``` |
| 127 | + |
| 128 | +### `nb.log(message)` -- Text logging |
| 129 | + |
| 130 | +Log a message to the current node. Messages appear in the terminal dashboard and are queryable via MCP tools. |
| 131 | + |
| 132 | +```python |
| 133 | +@nb.fn() |
| 134 | +def train(data): |
| 135 | + nb.log(f"Training on {len(data)} samples") |
| 136 | + for epoch in range(10): |
| 137 | + loss = do_train(data) |
| 138 | + nb.log(f"Epoch {epoch}: loss={loss:.4f}") |
| 139 | +``` |
| 140 | + |
| 141 | +### `nb.log_metric(name, value, step=None)` -- Scalar metrics |
| 142 | + |
| 143 | +Log scalar metrics with automatic step counting. |
| 144 | + |
| 145 | +```python |
| 146 | +@nb.fn() |
| 147 | +def train(model, data): |
| 148 | + for epoch in range(100): |
| 149 | + loss = train_one_epoch(model, data) |
| 150 | + nb.log_metric("loss", loss) |
| 151 | + nb.log_metric("lr", optimizer.param_groups[0]["lr"]) |
| 152 | +``` |
| 153 | + |
| 154 | +### `nb.log_cfg(cfg)` -- Configuration logging |
| 155 | + |
| 156 | +Log configuration for the current node. |
| 157 | + |
| 158 | +```python |
| 159 | +@nb.fn() |
| 160 | +def train(lr=0.001, epochs=50): |
| 161 | + nb.log_cfg({"lr": lr, "epochs": epochs}) |
| 162 | + ... |
| 163 | +``` |
| 164 | + |
| 165 | +### `nb.track(iterable, name=None, total=None)` -- Progress tracking |
| 166 | + |
| 167 | +Wrap any iterable for tqdm-like progress tracking. |
| 168 | + |
| 169 | +```python |
| 170 | +@nb.fn() |
| 171 | +def process(items): |
| 172 | + for item in nb.track(items, name="processing"): |
| 173 | + transform(item) |
| 174 | +``` |
| 175 | + |
| 176 | +### `nb.log_image(image, name=None, step=None)` -- Image logging |
| 177 | + |
| 178 | +Log images (PIL, NumPy arrays, or PyTorch tensors) for visual inspection. |
| 179 | + |
| 180 | +### `nb.log_audio(audio, sr=16000, name=None, step=None)` -- Audio logging |
| 181 | + |
| 182 | +Log audio data for playback and analysis. |
| 183 | + |
| 184 | +### `nb.log_text(name, text)` -- Rich text / Markdown logging |
| 185 | + |
| 186 | +Log formatted text or Markdown content. |
| 187 | + |
| 188 | +### `nb.md(description)` -- Workflow description |
| 189 | + |
| 190 | +Set a workflow-level description (Markdown supported). Visible in MCP tools and the dashboard. |
| 191 | + |
| 192 | +```python |
| 193 | +nb.md("A pipeline that loads images, runs inference, and exports predictions.") |
| 194 | +``` |
| 195 | + |
| 196 | +### `nb.ui()` -- Run-level UI defaults |
| 197 | + |
| 198 | +Set default layout and display options for the web UI: |
| 199 | + |
| 200 | +```python |
| 201 | +nb.ui(layout="horizontal", view="dag", minimap=True, theme="dark") |
| 202 | +``` |
| 203 | + |
| 204 | +### `nb.ask(question, options=None, timeout=None)` -- Human-in-the-loop |
| 205 | + |
| 206 | +Pause the pipeline and ask the user a question via MCP or the terminal. |
| 207 | + |
| 208 | +```python |
| 209 | +@nb.fn() |
| 210 | +def review(predictions): |
| 211 | + answer = nb.ask( |
| 212 | + "Model accuracy is 73%. Continue training?", |
| 213 | + options=["yes", "no", "retrain with more data"] |
| 214 | + ) |
| 215 | + if answer == "no": |
| 216 | + return predictions |
| 217 | + ... |
| 218 | +``` |
| 219 | + |
| 220 | +## CLI Reference |
| 221 | + |
| 222 | +### Start the daemon server |
| 223 | + |
| 224 | +```bash |
| 225 | +nb serve # foreground |
| 226 | +nb serve -d # background (daemon mode) |
| 227 | +nb serve --port 3000 # custom port |
| 228 | +nb serve --no-store # disable .nebo file storage |
| 229 | +``` |
| 230 | + |
| 231 | +### Run a pipeline |
| 232 | + |
| 233 | +```bash |
| 234 | +nb run my_pipeline.py |
| 235 | +nb run my_pipeline.py --name "experiment-1" |
| 236 | +``` |
| 237 | + |
| 238 | +### Load a .nebo file |
| 239 | + |
| 240 | +```bash |
| 241 | +nb load .nebo/2026-04-06_143000_run-1.nebo |
| 242 | +``` |
| 243 | + |
| 244 | +### Check status, logs, errors |
| 245 | + |
| 246 | +```bash |
| 247 | +nb status |
| 248 | +nb logs |
| 249 | +nb logs --run experiment-1 --node train --limit 50 |
| 250 | +nb errors |
| 251 | +nb errors --run experiment-1 |
| 252 | +``` |
| 253 | + |
| 254 | +### Stop the daemon |
| 255 | + |
| 256 | +```bash |
| 257 | +nb stop |
| 258 | +``` |
| 259 | + |
| 260 | +### MCP integration |
| 261 | + |
| 262 | +```bash |
| 263 | +nb mcp # print Claude Code MCP config |
| 264 | +``` |
| 265 | + |
| 266 | +## MCP Tools for AI Agents |
| 267 | + |
| 268 | +Nebo exposes 15 MCP tools for querying and controlling pipelines from an AI agent (e.g., Claude). The daemon server must be running. |
| 269 | + |
| 270 | +### Observation Tools |
| 271 | + |
| 272 | +| Tool | Description | |
| 273 | +|------|-------------| |
| 274 | +| `nebo_get_graph` | Full DAG structure: nodes, edges, execution counts | |
| 275 | +| `nebo_get_node_status` | Detailed status for one node: logs, metrics, errors, params | |
| 276 | +| `nebo_get_logs` | Recent log entries, filterable by node and run | |
| 277 | +| `nebo_get_metrics` | Metric time series for a node | |
| 278 | +| `nebo_get_errors` | All errors with full tracebacks and node context | |
| 279 | +| `nebo_get_description` | Workflow description and all node docstrings | |
| 280 | + |
| 281 | +### Action Tools |
| 282 | + |
| 283 | +| Tool | Description | |
| 284 | +|------|-------------| |
| 285 | +| `nebo_run_pipeline` | Start a pipeline script, returns a run ID | |
| 286 | +| `nebo_stop_pipeline` | Stop a running pipeline by run ID | |
| 287 | +| `nebo_restart_pipeline` | Stop and re-run a pipeline with same args | |
| 288 | +| `nebo_get_run_status` | Status of a specific run (running/completed/crashed) | |
| 289 | +| `nebo_get_run_history` | List all runs with outcomes and timestamps | |
| 290 | +| `nebo_get_source_code` | Read a pipeline source file | |
| 291 | +| `nebo_write_source_code` | Write or patch a pipeline source file | |
| 292 | +| `nebo_ask_user` | Send a question to the user via the terminal | |
| 293 | +| `nebo_wait_for_event` | Block until a pipeline event occurs or timeout elapses | |
| 294 | + |
| 295 | +## .nebo File Format |
| 296 | + |
| 297 | +Runs are persisted as `.nebo` binary files using MessagePack serialization. Each file contains a header (magic, version, metadata) followed by append-only event entries. Use `nb load` to replay a file into the daemon. |
| 298 | + |
| 299 | +## Architecture |
| 300 | + |
| 301 | +``` |
| 302 | ++----------------+ +------------------+ +------------------+ |
| 303 | +| Your Python |---->| Nebo SDK |---->| Daemon Server | |
| 304 | +| Pipeline | | (@fn, log, | | (FastAPI, | |
| 305 | +| | | track, ...) | | port 2048) | |
| 306 | ++----------------+ +--------+---------+ +--------+---------+ |
| 307 | + | | |
| 308 | + +-------v-------+ +--------------+---------------+ |
| 309 | + | Terminal | | | | |
| 310 | + | Dashboard | | +------v------+ +------v------+ |
| 311 | + | (Rich) | | | MCP Tools | | Web UI | |
| 312 | + +--------------+ | | (Claude) | | | |
| 313 | + | +-------------+ +-------------+ |
| 314 | + +-----v-----+ |
| 315 | + | CLI | |
| 316 | + | nb | |
| 317 | + +-----------+ |
| 318 | +``` |
| 319 | + |
| 320 | +Two execution modes: |
| 321 | + |
| 322 | +- **Local mode** (default): In-process only. No daemon needed. |
| 323 | +- **Server mode**: Events stream to a persistent daemon via HTTP. Use `nb serve` to start the daemon, then `nb run` to execute pipelines. |
| 324 | + |
| 325 | +## API Reference |
| 326 | + |
| 327 | +### Module: `nebo` |
| 328 | + |
| 329 | +| Function | Signature | Description | |
| 330 | +|----------|-----------|-------------| |
| 331 | +| `fn` | `@fn()`, `@fn(depends_on=[...])`, `@fn(ui={...})` | Register a function/class as a DAG node | |
| 332 | +| `log` | `log(message: str)` | Log a text message | |
| 333 | +| `log_metric` | `log_metric(name, value, step=None)` | Log a scalar metric | |
| 334 | +| `log_cfg` | `log_cfg(cfg: dict)` | Log node configuration | |
| 335 | +| `log_image` | `log_image(image, name=None, step=None)` | Log an image | |
| 336 | +| `log_audio` | `log_audio(audio, sr=16000, name=None, step=None)` | Log audio data | |
| 337 | +| `log_text` | `log_text(name, text)` | Log rich text / Markdown | |
| 338 | +| `track` | `track(iterable, name=None, total=None)` | Progress tracking | |
| 339 | +| `md` | `md(description: str)` | Set workflow description | |
| 340 | +| `ui` | `ui(layout, view, collapsed, minimap, theme)` | Set run-level UI defaults | |
| 341 | +| `init` | `init(port, host, mode, backends, terminal, dag_strategy, flush_interval, store)` | Manual initialization | |
| 342 | +| `ask` | `ask(question, options=None, timeout=None)` | Human-in-the-loop prompt | |
| 343 | +| `get_state` | `get_state() -> SessionState` | Access the global state singleton | |
| 344 | + |
| 345 | +### Logging Backends |
| 346 | + |
| 347 | +Implement the `LoggingBackend` protocol to send events to external systems: |
| 348 | + |
| 349 | +```python |
| 350 | +from nebo import LoggingBackend |
| 351 | + |
| 352 | +class MyBackend: |
| 353 | + def on_log(self, node: str, message: str, timestamp: float) -> None: ... |
| 354 | + def on_metric(self, node: str, name: str, value: float, step: int) -> None: ... |
| 355 | + def on_image(self, node: str, name: str, image_bytes: bytes, step: int) -> None: ... |
| 356 | + def on_audio(self, node: str, name: str, audio_bytes: bytes, sr: int) -> None: ... |
| 357 | + def on_node_start(self, node: str, params: dict) -> None: ... |
| 358 | + def on_node_end(self, node: str, duration: float) -> None: ... |
| 359 | + def flush(self) -> None: ... |
| 360 | + def close(self) -> None: ... |
| 361 | + |
| 362 | +nb.init(backends=[MyBackend()]) |
| 363 | +``` |
0 commit comments