I originally created Physical AI as an independent project and repository. As the curriculum and textbook developed, I decided to merge it directly into the Machine Learning Systems textbook series as Volume IV: Physical AI Systems to provide a single, unified curriculum spanning from foundation models and scale to agents and physical embodiment.
📘 Read Online: https://mlsysbook.ai
💻 Active GitHub Repository: github.com/harvard-edge/cs249r_book (seebooks/vol4)
🧰 Hardware Kits & Labs: mlsysbook.ai/kits · mlsysbook.ai/labsPlease star and watch harvard-edge/cs249r_book for all active development, chapters, and kit updates.
Student-facing course syllabus. This Markdown file is the syllabus—view it on GitHub or any Markdown preview. ETH packaging notes live in syllabus.md. The textbook is the separate Quarto project in ../book/ and now consolidated into harvard-edge/cs249r_book.
Project seminar & hardware studio · Harvard University · ETH Zurich / open follow-along
| Credits | 6 ECTS (proposed) |
| Format | Weekly seminar + kit studio · no written exam |
| Language | English |
| Level | Advanced Bachelor & Master |
| Instructor | Prof. Vijay Janapa Reddi · vj@eecs.harvard.edu · ETZ F 83 |
| Portal | mlsysbook.ai |
| Book | Physical AI: Machine Learning Systems That Sense and Act (harvard-edge/cs249r_book) |
Machine learning usually ends behind glass. A wrong label is a retry. Physical AI begins when software moves mass and spends energy—the world permanently changes. You cannot ctrl+z kinetic energy.
This is the systems course for that transition: not a kinematics survey, not TinyML “fit a model on a chip,” and not an LLM-agents lab. You learn the nuggets that matter when a learned model may act in the physical world—
know · measure · enforce · preserve · prove — then deploy, condition, or refuse.
Who this is for. Students from ML / AI, embedded / ECE, or robotics / control who want to build and measure systems where learned software can command actuators. Studio teams usually mix software-heavy and hardware-heavy backgrounds.
Prerequisites. Intro ML systems (models as components with latency, memory, energy) · comfortable Python and/or C/C++ · willingness to work in a small team on hardware. Helpful: embedded / TinyML. Not required: LLM-agents or a full robotics sequence. Baseline (quantize, prune, serve) → mlsysbook.ai.
What you will do. Build a physical agent across the semester · measure end-to-end paths · put permission on the MCU · keep a short engineering notebook · defend deploy / condition / refuse under a seeded fault.
Seventeen chapters organized into three parts + capstone. Same spine as the book. Labs track the chapters; formal specifications and code live in labs/ and harvard-edge/cs249r_book.
| # | Chapter & Core Focus |
|---|---|
| 1 | Boundary — When ML becomes Physical AI: closed-loop causal dynamics & the loop charter |
| 2 |
Body — Reflected inertia ( |
| 3 | Brain — What a learned component gives you (VLM / VLA) and the proposal–permission boundary |
| 4 | Nervous System — Multi-rate execution hierarchy, zero-allocation SRAM, and real-time IPC |
| 5 | Data — Demonstrations, compounding covariate shift, and teleoperation physics |
| 6 | Training — Multimodal policies (Diffusion / ACT), contact mechanics, and sim-to-real transfer |
| 7 | Evaluation — The astronomical exposure wall, non-asymptotic bounds, and offline disconnects |
| Book | Physical AI: Machine Learning Systems That Sense and Act (Volume IV of MLSysBook) |
| Labs | Kit bring-up → measure both brains → VLM intent → 1 kHz MCU enforcer → ship gate → capstone |
| Notebook | Short chapter checkpoints frozen in the engineering notebook (no classical written exam) |
| Baseline | mlsysbook.ai for quantize / prune / serve (not re-taught here) |
| # | Chapter & Core Focus |
|---|---|
| 8 | Perception — Sensor topologies, temporal alignment, and multi-sensor fusion |
| 9 | Memory — Stateful context, spatial maps, and dynamic world representations |
| 10 | Intent — Structured goal specification, affordance extraction, and safety constraints |
| 11 | Planning — Trajectory optimization, motion generation, and contact-aware reasoning |
| 12 | Enforcement — Real-time safety filters, control barrier functions, and fallback envelopes |
| # | Chapter & Core Focus |
|---|---|
| 13 | Placement — Heterogeneous compute partitioning (Cloud vs Edge vs MCU) |
| 14 | Intervention — Human-in-the-loop teleoperation, shared autonomy, and takeovers |
| 15 | Verification — Formal bounds, metamorphic testing, and hardware-in-the-loop |
| 16 | Release — Safety dossiers, regulatory compliance, and operational lifecycle |
| 17 | Frontier — Open research challenges in embodied and physical intelligence |