Fast Food CV — Camera-Based Productivity & Wait Time System

A real-time computer vision system for fast food kitchens. Uses your laptop camera + YOLOv8 to detect workers, classify activities, track food prep stages, and display a live web dashboard with wait time estimates and productivity metrics.

---

Quick Start

1. Install dependencies

cd fastfood_cv
pip install -r requirements.txt

2. Run the dashboard

python run.py

Then open http://localhost:5000 in your browser.

On first run, yolov8n.pt (the base COCO model) downloads automatically (~6 MB). It already detects people out of the box — kitchen-specific detection requires fine-tuning (see below).

---

What It Does Right Now (Base Model)

Feature	Status
Detect workers in camera frame	✅ Works immediately
Track individuals across frames	✅ Works immediately
Heuristic activity classification	✅ Works immediately
Food prep stage tracking (simulated)	✅ Demo mode
Wait time estimation	✅ Demo mode
Productivity metrics (throughput, utilisation)	✅ Works immediately
Accurate food-stage detection	🔧 Needs fine-tuning

---

Fine-Tuning for Your Kitchen (How to Train)

Step 1 — Collect training frames

From a video file of your kitchen:

python training/collect_frames.py video --video kitchen_clip.mp4 --output data/images/train --fps 2

Or capture live from the camera right now:

python training/collect_frames.py live --output data/images/train --duration 120

Step 2 — Label your data

Upload the extracted frames to Roboflow Annotate (free).

Draw bounding boxes around:

• Workers and their activity state (person_idle, person_cooking, person_assembling, etc.)
• Food items at each prep stage (food_raw, food_cooking, food_assembled, food_ready)

You need roughly 500–2000 labeled frames per class for good results.

Step 3 — Create your dataset config

python training/train.py template --output data/dataset.yaml

Edit data/dataset.yaml to match your class names.

Step 4 — Train

python training/train.py train --data data/dataset.yaml --epochs 50 --model yolov8s.pt

Or download a public food/kitchen dataset from Roboflow Universe first:

# Set your API key (free at roboflow.com)
export ROBOFLOW_API_KEY=your_key_here

python training/train.py roboflow \
  --workspace roboflow-universe-projects \
  --project food-detection-eoh3x \
  --version 1 \
  --then-train

Step 5 — Use your trained model

python run.py --model runs/fastfood/train/weights/best.pt

---

Public Datasets to Bootstrap Training

Search Roboflow Universe for:

Search term	What you get
kitchen activity	Kitchen action recognition
food preparation	Food item detection at various stages
restaurant worker	Person detection in restaurant context
fast food	Fast food specific items
cooking detection	Cooking activity scenes

---

Project Structure

fastfood_cv/
├── run.py                  ← Start here
├── app.py                  ← Flask web server + MJPEG stream + SSE
├── detector.py             ← YOLOv8 inference + tracking + state machine
├── requirements.txt
├── templates/
│   └── dashboard.html      ← Web dashboard UI
├── training/
│   ├── train.py            ← Fine-tuning script
│   └── collect_frames.py   ← Frame extraction from video/camera
├── data/                   ← Your training images and labels go here
└── models/                 ← Save custom .pt files here

---

API Endpoints

Method	Endpoint	Description
GET	/	Dashboard UI
GET	/video_feed	MJPEG camera stream
GET	/stream	SSE stream of live detection data
GET	/api/snapshot	Current state as JSON
GET	/api/metrics	Productivity metrics
POST	/api/orders/new	Start a new order
POST	/api/orders/<id>/advance	Manually advance order stage

---

Tuning Wait Time Estimates

Edit STAGE_DURATIONS in detector.py to match your restaurant's actual prep times:

STAGE_DURATIONS = {
    "idle":         0,
    "prep_started": 15,   # seconds
    "cooking":      90,   # adjust for your grill/fryer time
    "assembly":     30,
    "packaging":    15,
    "ready":        0,
}

After collecting real timing data, these can be replaced with a regression model trained on observed completion times.

---

Hardware Notes

• Works with any USB or built-in webcam
• For better coverage, use a wide-angle camera mounted overhead
• A GPU (NVIDIA) dramatically improves FPS — on CPU expect ~5–15 FPS with yolov8n
• Use yolov8n.pt (nano) for CPU, yolov8s.pt or yolov8m.pt with GPU