Chapter 1: Local LLM Environment - Poetry, Docker & 4 Models

[jrbrowning]

Building Your Model Workshop

This course experiments with local AI-assisted learning.

The code for each chapter is in a branch you can reference later (chapter 1). The most recent code will always be in main

Ask the AI code-specific questions. Ask it architecture questions. Ask it about my reasoning logic behind the way I've broken up the chapters.

On that note...

My Reasoning Logic on how I've broken up this chapter:

You need a place to work with models. Not a lab, not a playground - a workshop where you can see how these different AI "tools" perform.

For our workshop, we need some Prerequisites & Setup

Python - It's what most AI developers already speak, so we start here.
Poetry - A developer I respect recommended it. It works. UV might be next, but Poetry hasn't given me a reason to switch.
Ollama - The model library. Nearly every open-source model I'm interested in today lives there.
Docker - Infrastructure separation of concerns and reproducibility.

Model Math

We need 4 models to start.

Most models today fall into two categories: Traditional or Reasoning. We'll want one of each later.
We need 2 models.

To compare within each category, we need two of each.
We actually need 4 models to start.

We use Docker to map the ports to allow us to have >1 model locally.

Course Prompts

For this chapter, try using these prompts.

Option 1: Copilot - Explain something in a file
ex: #file:docker-compose.yml How does this enable multiple Ollama instances?

Option 2: Local Models - RAG Experiment (You can read more about the idea Course Goal)

(use # in ollama to include file in context)
ex: #(select the file) docker-compose.yml How does this enable multiple Ollama instances?

---

Questions? → Questions board