Skip to content

getting-started.md

Latest commit

 

History

History
323 lines (235 loc) · 7.91 KB

getting-started.md

File metadata and controls

323 lines (235 loc) · 7.91 KB
Raw

Getting Started

Complete guide to get PDF Content Search running in 5 minutes.

Prerequisites

  • Docker & Docker Compose installed
  • Make (pre-installed on Linux/macOS)
  • 8GB RAM minimum (16GB recommended)
  • Ollama installed natively on the host — see Ollama Setup below

Ollama Setup

Ollama runs natively on the host (not in Docker). The PHP container reaches it via host.docker.internal.

1. Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

2. Configure the systemd service

sudo systemctl edit ollama

Add the following and save:

[Service]
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_NUM_PARALLEL=4"

Then reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Verify the variables are active:

systemctl show ollama --property=Environment

Why these settings?

  • OLLAMA_HOST=0.0.0.0 — Ollama listens on all interfaces so Docker containers can reach it
  • OLLAMA_NUM_PARALLEL=4 — allows 4 simultaneous LLM generation requests (improves translation throughput when multiple workers are active)

3. Pull the required models

ollama pull qwen2.5:3b        # Translation model (~1.9 GB)
ollama pull nomic-embed-text  # Embedding model (~274 MB)

Note: qwen2.5:3b translates a full PDF page in ~52s on a Core Ultra 7 155U (CPU-only). nomic-embed-text generates 768-dimensional embeddings in ~190ms per batch on the same hardware.

Development Setup

Quick Start

# 1. Clone repository
git clone https://github.com/josego85/pdf-content-search.git
cd pdf-content-search

# 2. Start everything (Ollama must already be running natively)
make dev

Access: http://localhost

The make dev command automatically:

  • ✅ Builds Docker images
  • ✅ Installs dependencies (Composer + NPM)
  • ✅ Runs database migrations
  • ✅ Creates Elasticsearch index
  • ✅ Builds frontend assets (Webpack)
  • ✅ Starts 3 Messenger workers for async jobs

What's Running

Service URL/Port Description
Web App http://localhost Main application
Analytics http://localhost/analytics Search metrics dashboard
Elasticsearch http://localhost:9200 Search engine
PostgreSQL localhost:5432 Database
Ollama http://localhost:11434 AI models (native host)

Adding PDFs

# 1. Copy PDFs to public directory
cp /path/to/files/*.pdf public/pdfs/

# 2. Index them (with semantic embeddings)
docker compose -p pdf-content-search exec php php bin/console app:index-pdfs

# Skip embeddings for faster indexing (no AI search)
docker compose -p pdf-content-search exec php php bin/console app:index-pdfs --skip-embeddings

Production Setup

1. Generate Secrets

./bin/init-prod-secrets.sh

This creates .env.prod.local with auto-generated:

  • APP_SECRET (32-char hex)
  • POSTGRES_PASSWORD (32-char hex)
  • ELASTIC_PASSWORD (32-char hex)

Important: .env.prod.local is NOT committed to Git (contains secrets).

2. Start Production

make prod

Access: http://localhost:8080

Differences from dev:

  • Port: 8080 (instead of 80)
  • Debug: OFF
  • Optimized PHP-FPM settings (512M memory, opcache)
  • Multi-stage Docker build (smaller images)
  • Elasticsearch authentication enabled
  • Apache Brotli compression (quality 11)

Common Commands

# Development
make dev           # Start development
make down          # Stop development
make restart       # Restart development
make logs          # View all logs
make logs SERVICE=php  # View specific service
make shell         # Open PHP container shell
make test          # Run PHPUnit tests

# Production
make prod          # Start production
make down-prod     # Stop production
make logs-prod     # View production logs
make shell-prod    # Open production shell

# Utilities
make status        # Show all environments status
make clean-dev     # Remove dev volumes (DESTRUCTIVE)
make clean-prod    # Remove prod volumes (DESTRUCTIVE)
make help          # Show all commands

Environment Configuration

Development (.env)

The .env file is committed to Git with safe defaults (Symfony standard):

APP_ENV=dev
APP_DEBUG=1
APP_SECRET=change-me-in-env-local  # OK for dev

POSTGRES_PASSWORD=dev_password      # OK for dev
ELASTICSEARCH_HOST=http://elasticsearch:9200  # No auth in dev
OLLAMA_HOST=http://host.docker.internal:11434
OLLAMA_MODEL=qwen2.5:3b
OLLAMA_EMBEDDING_MODEL=nomic-embed-text

For local overrides: Create .env.local (not committed).

Production (.env.prod + .env.prod.local)

File hierarchy:

  1. .env (base config, committed)
  2. .env.prod (production overrides, committed)
  3. .env.prod.local (secrets, NOT committed, auto-generated)

Production defaults (.env.prod):

APP_ENV=prod
APP_DEBUG=0
APACHE_PORT=8080
POSTGRES_DB=pdf_search_prod
ELASTICSEARCH_HOST=http://elastic:${ELASTIC_PASSWORD}@elasticsearch:9200

Docker Architecture

File Structure

docker-compose.yml          # Base (shared by dev & prod)
docker-compose.dev.yml      # Development overrides
docker-compose.prod.yml     # Production overrides

.docker/
├── dev/
│   ├── app/Dockerfile      # Dev PHP image
│   ├── apache/             # Apache config (Brotli quality 6)
│   └── supervisor/         # Messenger workers
└── prod/
    ├── app/Dockerfile      # Prod multi-stage build
    ├── apache/             # Apache config (Brotli quality 11)
    └── supervisor/         # Optimized workers

Composition Pattern

Development:

docker compose -f docker-compose.yml -f docker-compose.dev.yml -p pdf-content-search up -d

Production:

docker compose -f docker-compose.yml -f docker-compose.prod.yml -p pdf-content-search-prod up -d

Why separate project names?

  • Prevents volume conflicts
  • Can run dev + prod simultaneously (different ports)

Troubleshooting

Port 80 Already in Use

# Check what's using port 80
sudo lsof -i :80

# Change port in .env
echo "APACHE_PORT=8080" >> .env.local
make restart

Elasticsearch Won't Start (Linux)

# Increase vm.max_map_count
sudo sysctl -w vm.max_map_count=262144

# Make permanent
echo 'vm.max_map_count=262144' | sudo tee -a /etc/sysctl.conf

Docker Daemon Not Running

sudo systemctl start docker

Ollama Not Reachable from Docker

# Verify Ollama is running on the host
systemctl status ollama

# Verify it listens on 0.0.0.0 (not just 127.0.0.1)
systemctl show ollama --property=Environment

# Test from inside the PHP container
docker compose exec php curl http://host.docker.internal:11434/api/tags

# If models are missing, pull them on the host
ollama pull qwen2.5:3b
ollama pull nomic-embed-text

Messenger Workers Not Processing

# Check worker status
docker compose -p pdf-content-search exec php php bin/console messenger:stats

# Check Supervisor logs
make logs SERVICE=php | grep messenger

# Restart workers
docker compose -p pdf-content-search restart php

Reset Everything (Nuclear Option)

# Stop and remove all data (DESTRUCTIVE)
make clean-dev

# Start fresh
make dev

More Issues?

See troubleshooting.md for complete guide.

Next Steps

Learn more:

Development: