Course Motivation: When AI Decides What's Next

[jrbrowning]

A Maybe Machine

The allure of tool calling (or "function calling") in Large Language Models (LLMs) lies in the shift from "here's what I can tell you" to "here's what I can do for you." Tool calling enables action. Where Retrieval Augmented Generation (RAG) gave LLMs context control, tool calling offers a path to act on that context.

Current models can already browse the web, write code, and analyze images. Incredible, but on-demand today. When products chain these capabilities to find patterns I missed or never thought of, that's when the magic returns.

But unlike RAG today, where we can trace back to source documents, tool-generated actions create a new but familiar grounding problem. When an AI decides to call an API, update a database, or trigger a workflow, the “truth” of that decision is less clear than retrieving a specific document.

Orchestration Question

The core tension I see as a developer is that technically elegant implementations aren't how I think the winners will be decided. These are foundational coding decisions (OpenAI-compatible or MCP or ???), but it's more about who decides what happens next in a multi-step AI interaction. Whoever controls that decision point controls the economics and technical direction of the entire stack.

When you need multiple AI calls, tool use, or data lookups to complete a task, something has to decide the sequence. That “something”, the orchestrator, will naturally become the funnel for both the user experience and revenue flow.

Landscape Today (Aug, 2025)

Manual orchestration: You explicitly direct the AI through each step via conversational prompts. You tell it to browse the web, then ask it to analyze what it found, then request code to visualize the data. The platform handles the execution, but you’re making the strategic decisions about task flow.

Developer-controlled orchestration: Your application code decides the flow. The AI is just one component among many that the developer stitches together. The developer writes the logic for when to call which models, how to handle errors, and how to route between different services.

Model-driven orchestration: The AI model itself decides what to do next. It calls tools, routes to other models, or accesses data based on its own judgment. This ranges from tactical decisions (which specific tool to use) to strategic ones (how to break down a complex task).

Hybrid orchestration: Some combination where humans make strategic decisions while AI handles tactical sequencing, or where orchestration shifts between layers depending on the task complexity or confidence thresholds.

Implementation Gap

From this landscape view, the current tool calling implementation space seems fractured between two extremes: "complete solutions" that abstract away the orchestration decisions I want insights into, and foundational tutorials that leave me wondering how to keep going. This makes sense as we are still in the sorting-this-whole-thing-out phase. For me, the bridge between these extremes was building an application solution as a learning tutorial.

User-facing applications need real-time streaming, responsive design, interactivity, and state management across the stack. Most tool-calling tutorials today are rooted in siloed, language-specific solutions. And as professionals, we're building toward production implementations. We need production infrastructure thinking along the way.

In Python, that means Pydantic validation and structured error handling. In JavaScript, that means TypeScript frontends that handle streaming responses with responsive designs and state management. Each layer should be containerized—frontend, API, and model services, which allows running multiple LLM instances as needed. Without this separation as infrastructure, I know I'm going to be putting out fires before I know it.

“Only you can prevent fires” Thinking

My approach to development focuses on building frameworks that prevent fires before they start. (Personal philosophy: Fire can't be trusted in all aspects of life.)

Successful fire prevention centers on three forms of thinking designed around constraint management along with feature delivery:

1. Solve today's problem - Build working tool calling systems with proper typing, validation, and error handling.
2. Watch for what makes #1 irrelevant tomorrow - Design for the orchestration patterns that are emerging, not just today's simple function calls. To me today, this means we start by encapsulating what a tool needs into a central class component that can be tested independently, then adding it to a registry. Week 2 digs into this in depth.
3. Build infrastructures that can pivot, not just iterate - We need 2 git repositories for this application: Backend (handle the different API requests and interaction with the LLMs) and a Frontend (user interaction with the LLMs). You can "combine them later" easier than "pull them apart later" in my experience. So we start with 2 repos.

Course Goal

My goal in open-sourcing each chapter and publishing through GitHub discussions is to harness the algorithmic power of today's world to find others on this journey. To connect with individuals ahead of me who can provide feedback, and others interested in this domain but feeling overwhelmed by the gap.

I started with OpenAI-compatible tool calling because the spec is open and many models are readily available locally via Ollama. Everything can be built for no cost per token, then redirected to stronger cloud-based models as needed (which carry a cost, but you get better precision in return).

Developer-controlled orchestration works well for exploratory tasks like this because we can utilize open-source software (OSS) everywhere. Building this has helped me think about when developer orchestration makes sense (predictable workflows, validated operations) versus when model-driven orchestration might be worth the added complexity.

I hope it will for you as well. Look forward to meeting you in the discussion boards.

Parting Thought:

The interesting part to me now isn't which orchestration approach is "best", they're all solving different problems and serving different use cases. I think there will be several winners.

But whichever layer ends up handling the most complex orchestration decisions tends to capture the most value. History suggests this from the operating system wars of the 80s-90s. Wherever the orchestration happens, that's where pricing power accumulates.

Today, we're seeing this play out across all these layers simultaneously, with different winners emerging for different types of tasks.

How cool is that.

If you find this course useful, please give it a star. It really helps the machines know it's useful.