VFXB-Studio-Plan

Plan: VFXB Full-Stack AI Video Platform Build

Context

• Existing frontend: Complete Vite+React+TS frontend at e:\VFXB\VFXB - Studio with 50+ components, full design system, all UI screens — NO backend wired up yet
• User decisions: Keep Vite/React frontend | New separate monorepo e:\vfxb\ | FastAPI heavy + simple BFF | MVP = NLP Edit Engine

TL;DR

Build the full backend infrastructure (FastAPI + Worker + DB + Storage + AI) in a new Turborepo monorepo at e:\vfxb\. Wire the existing Vite/React frontend to it. MVP priority is the NLP edit engine — user types English, AI cuts the video.

---

Architecture Decision Record

• Frontend: Existing Vite+React stays at e:\VFXB\VFXB - Studio → becomes apps/web in new monorepo (symlink or copy)
• API Layer: FastAPI Python handles all endpoints. "Both" interpreted as: simple endpoints in FastAPI routers (auth, CRUD) + heavy async processing in Redis worker. No Next.js app needed.
• Auth: Supabase Auth (JWT) — frontend uses @supabase/supabase-js SDK, backend validates tokens
• State: Zustand added to existing Vite frontend (replace prop-drilling + custom events pattern)

---

Phase 0 — Monorepo Bootstrap (Day 1)

Steps

1. Run npx create-turbo@latest vfxb --package-manager pnpm at e:\
2. Move/copy existing VFXB - Studio into vfxb/apps/web (keep Vite config intact)
3. Create apps/api/ (FastAPI) and apps/worker/ (RQ processor) directories
4. Create packages/types/ for shared TS types consumed by frontend
5. Create root .env.example with all required keys (Supabase, OpenAI, Anthropic, Google AI, AssemblyAI, Cloudflare R2, Redis, Stripe)
6. Create docker-compose.yml for local dev: web:5173, api:8000, worker, redis:6379
7. Create turbo.json with build/dev/test/lint pipelines

Files

• e:\vfxb\turbo.json
• e:\vfxb\pnpm-workspace.yaml
• e:\vfxb\.env.example
• e:\vfxb\docker-compose.yml
• e:\vfxb\apps/web/ (existing Studio codebase)
• e:\vfxb\apps/api/main.py (FastAPI entry)
• e:\vfxb\apps/worker/worker.py (RQ entry)

---

Phase 1 — Database & Auth (Days 2-4)

Steps

1. Create Supabase project → enable Email + Google + GitHub OAuth
2. Run SQL schema in Supabase editor: tables users, videos, edits, chat_messages, exports, subscriptions
3. Enable RLS on all tables with per-user policies
4. Create apps/api/db/schema.sql with full DDL
5. Create apps/api/db/client.py: Supabase Python client with service key
6. Implement FastAPI auth middleware: apps/api/middleware/auth.py — validates Supabase JWT, extracts user_id, injects into request state
7. Create FastAPI routers: apps/api/routers/users.py (GET /me, PATCH /me)
8. Wire existing AuthScreen.tsx to real Supabase Auth: install @supabase/supabase-js, replace mock handleAuthenticate with Supabase signInWithPassword + signInWithOAuth
9. Create apps/web/src/lib/supabase.ts: singleton Supabase client
10. Create apps/web/src/store/auth.ts: Zustand auth store with user, session, signIn, signOut
11. Protect routes in App.tsx using real session state

Key code patterns

• FastAPI dependency: async def get_current_user(token: str = Depends(oauth2_scheme)) → decode Supabase JWT
• Frontend: supabase.auth.onAuthStateChange() → update Zustand store
• RLS policy pattern: CREATE POLICY "owner" ON videos FOR ALL USING (auth.uid() = user_id)

Files

• apps/api/db/schema.sql
• apps/api/db/client.py
• apps/api/middleware/auth.py
• apps/api/routers/users.py
• apps/web/src/lib/supabase.ts
• apps/web/src/store/auth.ts (new Zustand store)

---

Phase 2 — File Storage & Upload (Days 5-6)

Steps

1. Create Cloudflare R2 buckets: vfxb-videos + vfxb-exports
2. Configure R2 CORS for browser uploads, set lifecycle rules
3. Implement apps/api/routers/upload.py:
   • POST /api/upload/presigned: validates user quota, creates video DB record (status=uploading), returns S3-presigned URL + video_id
   • POST /api/upload/complete: sets status=processing, enqueues analyze_video job
   • POST /api/upload/from-url: yt-dlp download → R2 upload → video record
4. Implement quota check: free=3 videos, pro=unlimited (read from users.plan)
5. Wire VFXBUploadScreen.tsx to real upload:
   • Replace mock progress with real TUS upload (tus-js-client)
   • On complete: call /api/upload/complete, navigate to studio view with real video_id
6. Create apps/web/src/store/video.ts: Zustand video store (currentVideo, uploadProgress, isProcessing)

Files

• apps/api/routers/upload.py
• apps/api/services/storage.py (R2 client wrapper)
• apps/web/src/store/video.ts
• apps/web/src/hooks/useVideoUpload.ts

---

Phase 3 — Video Analysis Engine (Days 7-10) [AI Core]

Steps (all in apps/worker/)

1. Set up Redis Queue: apps/worker/queue.py — job types: analyze_video, apply_edit, export_video
2. Step A — Transcription (steps/transcribe.py):
   • Submit audio to AssemblyAI with word timestamps + speaker diarization + filler detection
   • Poll until complete, store in videos.transcript + videos.analysis.words
3. Step B — Visual Analysis (steps/visual_analysis.py):
   • ffmpeg -vf fps=1 to extract frames
   • Sample every 5th frame → GPT-4o Vision → {has_face, motion_level, engagement_potential...}
   • Store in videos.analysis.visual_data
4. Step C — Audio Analysis (steps/audio_analysis.py):
   • librosa: detect silences >1.5s, dead zones, SNR, BPM
   • Store {silences: [...], avg_energy, background_noise_db} in videos.analysis
5. Step D — Virality Scoring (steps/virality_score.py):
   • Send all analysis to Claude 3.5 Sonnet with structured system prompt
   • Returns {overall_score, grade, factors, top_issues, predicted_retention_curve}
   • Store in videos.score_breakdown, videos.virality_score, set status=analyzed
6. Wire Dashboard to real data: GET /api/videos?user_id=me → replace mock stats with DB query
7. Wire VideoIntelligenceCard + AIDirectorPanel to real analysis data from video store

Files

• apps/worker/queue.py
• apps/worker/steps/transcribe.py
• apps/worker/steps/visual_analysis.py
• apps/worker/steps/audio_analysis.py
• apps/worker/steps/virality_score.py
• apps/api/routers/videos.py (CRUD + analysis results)

---

Phase 4 — NLP Edit Engine [MVP Priority] (Days 11-15)

Steps

1. NLP Parser (apps/api/services/nlp_parser.py):
   • Takes user message + full video context (transcript, silences, analysis, score)
   • Sends to Claude 3.5 Sonnet with VFXB system prompt
   • Returns structured {understood_intent, edits[], response_message, needs_confirmation}
2. FFmpeg Executor (apps/worker/services/ffmpeg_executor.py):
   • Implement: cut_silence, trim_range, change_speed, add_captions, platform_export, color_grade, extract_best_clip
   • Each func: download from R2 → run FFmpeg → upload output to R2 → return URL
   • Safety: validate all paths, sanitize filenames, enforce max duration
3. Chat API (apps/api/routers/chat.py):
   • POST /api/chat/message: load video context → load last 10 messages → route to correct AI agent → parse edit intent → stream SSE response
   • SSE events: {type: "text", content}, {type: "edit_plan", plan}, {type: "done"}
   • POST /api/chat/confirm-edit: if confirmed → queue apply_edit job
   • GET /api/chat/history/:video_id: last 50 messages
4. Edit Status WebSocket (apps/api/routers/websocket.py):
   • WS /ws/edit-status/:job_id → polls Redis job status → pushes {status, progress, output_url}
5. Wire Frontend (existing BeautifulAIChat.tsx / ChatInput.tsx):
   • Add apps/web/src/hooks/useChatStream.ts: EventSource wrapper for SSE
   • Add apps/web/src/store/chat.ts: Zustand chat store (messages, isStreaming, pendingEdit)
   • Replace mock responses in BeautifulAIChat with real SSE stream
   • Show edit_plan cards with Confirm/Cancel in ChatThread
   • Add progress bar message type during edit job
   • Connect WebSocket for real-time edit progress

Key Security Points

• Validate video ownership before any edit (user must own video)
• Sanitize all FFmpeg path arguments (no shell injection)
• Rate limit chat endpoint: 20 req/min per user (Redis sliding window)

Files

• apps/api/services/nlp_parser.py
• apps/api/routers/chat.py
• apps/api/routers/websocket.py
• apps/worker/services/ffmpeg_executor.py
• apps/web/src/store/chat.ts
• apps/web/src/hooks/useChatStream.ts
• apps/web/src/hooks/useEditStatus.ts

---

Phase 5 — Frontend Wiring (Days 16-18)

Steps

1. Install Zustand in apps/web: replace prop-drilling and window.dispatchEvent patterns
2. Create apps/web/src/store/ui.ts (activeFeaturePanel, comparisonMode, beforeUrl, afterUrl)
3. Wire VideoPreview.tsx to real R2 URLs from video store
4. Wire Dashboard.tsx to real API data (stats, recent videos)
5. Wire ExportShareModal.tsx to real export API + platform OAuth
6. Wire EditHistoryPage.tsx to GET /api/edits?video_id=...
7. Add skeleton loading states (already use shadcn/ui Skeleton)
8. Add VITE_API_URL env var to Vite config, update all API calls

Files

• apps/web/src/store/ui.ts
• apps/web/vite.config.ts (proxy to backend in dev)
• apps/web/.env.example

---

Phase 6 — Payments (Days 19-20)

Steps

1. Create Stripe products: Free ($0), Pro ($29/mo or $19/mo yearly)
2. apps/api/routers/stripe.py:
   • POST /api/stripe/create-checkout: creates Stripe Checkout session
   • POST /api/stripe/webhook: handles checkout.session.completed, subscription.deleted, payment_failed
   • GET /api/stripe/portal: opens Customer Portal
3. Enforce plan limits in FastAPI dependency middleware
4. Wire UpgradePage.tsx to real Stripe checkout

Files

• apps/api/routers/stripe.py
• apps/api/middleware/plan_check.py

---

Phase 7 — Power Features (Days 21-28)

1. Autonomous Publishing Agent (apps/api/services/autonomous_agent.py): multi-step job chain (analyze → fix → caption → export → YouTube API upload)
2. Creator DNA (apps/api/services/creator_dna.py): extract style patterns after 3+ videos, inject into all future Claude prompts
3. Audience Simulation (apps/api/services/simulation.py): Claude retention curve prediction, SVG chart in frontend
4. Platform Optimizer (apps/worker/services/platform_optimizer.py): per-platform resize, pacing, captions
5. Real-time Collaboration (apps/api/routers/collab.py): Supabase Realtime, invite system, presence indicators
6. Enterprise API (apps/api/routers/enterprise.py): API key auth, rate limits per tier

---

Phase 8 — Deploy (Days 29-30)

1. apps/api/Dockerfile (python:3.12-slim + ffmpeg)
2. apps/worker/Dockerfile (python:3.12-slim + ffmpeg + yt-dlp)
3. Deploy frontend to Vercel (or keep Vite → deploy to Cloudflare Pages)
4. Deploy API + Worker to Railway
5. GitHub Actions CI/CD: test → build → deploy

---

Verification Steps

1. Upload a real MP4 → verifying R2 storage + DB record created
2. Analysis pipeline completes → virality score appears in UI with real data
3. Type "remove silences" in chat → edit plan shown → confirm → FFmpeg job runs → new video appears with before/after comparison
4. Auth flow: sign up → email verification → dashboard loads user's videos
5. Stripe checkout: upgrade to Pro → plan updates in DB → video limit removed
6. pytest apps/api/tests/ (FastAPI unit tests)
7. Rate limit test: 21 chat messages/min returns 429

---

Decisions

• No Next.js needed for frontend (user keeps Vite). "Both" back-end choice = simple FastAPI routers + async workers
• New monorepo at e:\vfxb\ — existing VFXB Studio files move to apps/web
• MVP = NLP edit engine (Phase 4) — Phases 0-3 are blockers and must run sequentially first
• FFmpeg runs only in worker container, never in the main API process
• All AI calls use streaming where possible (SSE for chat responses)