An AI-powered theft detection and security monitoring system with real-time video processing
- Overview
- Features
- Architecture
- Tech Stack
- Prerequisites
- Installation
- Configuration
- Running the Application
- Project Structure
- API Documentation
- Contributing
VisionGuard.ai is a comprehensive security monitoring solution that combines cutting-edge AI technology with real-time video processing to detect suspicious activities and potential theft. The system features a modern web interface for monitoring multiple camera feeds, reviewing alerts, and managing security across multiple shop locations.
- Real-time AI Detection: Process video feeds in real-time using advanced AI models for anomaly detection
- WebRTC Streaming: Low-latency video streaming directly to web browsers
- Multi-shop Management: Support for multiple locations with role-based access control
- Telegram Integration: Instant notifications and remote monitoring via Telegram bot
- Persistent WebSocket Connections: Real-time alerts with automatic reconnection and heartbeat monitoring
- Authentication & Authorization: Secure JWT-based authentication with owner and manager roles
- 📊 Real-time Dashboard - Monitor system metrics, active cameras, and detection accuracy
- 📹 Live Feed - WebRTC camera streaming with AI processing
- 🚨 Suspicious Activity - Review, filter, and manage detected security events
- 🏪 Shop Management - Multi-location support with shop-specific configurations
- 🔔 Real-time Notifications - Instant alerts via WebSocket connections
- 🎨 Modern UI - Dark/light theme with gradient accents and smooth animations
- 📱 Responsive Design - Seamless experience across desktop, tablet, and mobile
- 🔐 Secure Authentication - JWT-based auth with role-based access control
- ✅ AI Video Processing - Real-time frame processing with configurable AI models
- ✅ WebRTC Signaling - SDP offer/answer exchange for peer connections
- ✅ WebSocket Support - Persistent connections for real-time notifications
- ✅ Telegram Bot - Bi-directional communication and remote control
- ✅ Database Management - PostgreSQL with Alembic migrations
- ✅ RESTful API - OpenAPI 3.0 compliant endpoints
- ✅ CORS Support - Configurable cross-origin resource sharing
- ✅ Health Monitoring - Connection tracking and health check endpoints
- ✅ Training Data Collection - Reinforcement learning data storage
┌─────────────────────────────────────────────────────────────────────┐
│ VisionGuard.ai System │
└─────────────────────────────────────────────────────────────────────┘
┌──────────────────┐ ┌──────────────────┐
│ Next.js Client │ │ FastAPI Backend │
│ (Frontend) │ │ (Python) │
│ │ WebRTC Video │ │
│ ┌────────────┐ │◄────────(Encrypted)────────────┤ ┌────────────┐ │
│ │ Live Feed │ │ │ │ Signaling │ │
│ │ (WebRTC) │ │ │ │ Router │ │
│ └────────────┘ │ WebSocket │ └────────────┘ │
│ │◄──────(Notifications)───────────┤ │
│ ┌────────────┐ │ │ ┌────────────┐ │
│ │ Dashboard │ │ REST API │ │ AI Stream │ │
│ │ & Alerts │ │◄──────(JSON Data)──────────────┤ │ Track │ │
│ └────────────┘ │ │ └──────┬─────┘ │
│ │ │ │ │
│ ┌────────────┐ │ │ ┌──────▼─────┐ │
│ │ Auth & │ │ │ │ AI │ │
│ │ Shops │ │ │ │ Processor │ │
│ └────────────┘ │ │ └──────┬─────┘ │
└──────────────────┘ │ │ │
│ ┌──────▼─────┐ │
┌──────────────────┐ │ │ PostgreSQL │ │
│ Telegram Bot │◄────────(Polling)───────────────┤ │ Database │ │
│ (Notifications) │ │ └────────────┘ │
└──────────────────┘ └──────────────────┘
- Framework: Next.js 14 (App Router)
- Language: TypeScript
- Styling: Tailwind CSS
- Charts: Recharts
- Icons: Lucide React
- Animations: Framer Motion
- AI Models: TensorFlow.js with COCO-SSD
- HTTP Client: Axios
- State Management: React Context API
- Framework: FastAPI 0.121.0
- Language: Python 3.8+
- WebRTC: aiortc 1.14.0
- Video Processing: OpenCV, PyAV
- AI/ML: TensorFlow 2.x, Keras
- Database: PostgreSQL with Alembic migrations
- Authentication: JWT (PyJWT)
- Real-time: WebSockets
- Telegram: python-telegram-bot
- CORS: FastAPI CORS middleware
- Node.js: 18+
- Python: 3.8 or higher
- PostgreSQL: 13+
- FFmpeg: Required for video processing
- Git: For version control
- Telegram Bot Token: For notification features
- CUDA: For GPU-accelerated AI processing (optional)
git clone <repository-url>
cd visionguardFor detailed installation instructions:
- Backend Setup: See backend-visionguard-ai/README.md
- Frontend Setup: See visionguardai-frontend/README.md
For detailed configuration instructions:
- Backend Configuration: See backend-visionguard-ai/README.md for environment variables, database setup, and Telegram bot configuration
- Frontend Configuration: See visionguardai-frontend/README.md for Next.js environment setup
Backend: cd backend-visionguard-ai && python main.py → http://localhost:8000
Frontend: cd visionguardai-frontend && npm run dev → http://localhost:3000
For detailed instructions including Telegram bot setup and production deployment:
- Backend: See backend-visionguard-ai/README.md
- Frontend: See visionguardai-frontend/README.md
visionguard/
├── backend-visionguard-ai/ # Python FastAPI Backend
│ ├── app/ # Application code
│ │ ├── ai/ # AI models and processors
│ │ ├── api/ # API endpoints
│ │ ├── core/ # Core utilities
│ │ ├── db/ # Database configuration
│ │ ├── models/ # SQLAlchemy models
│ │ └── schemas/ # Pydantic schemas
│ ├── docs/ # Backend documentation
│ ├── main.py # Application entry point
│ └── README.md # Backend documentation
│
├── visionguardai-frontend/ # Next.js Frontend
│ ├── app/ # Next.js app directory
│ ├── components/ # React components
│ ├── context/ # React contexts
│ ├── hooks/ # Custom hooks
│ ├── lib/ # API and services
│ └── README.md # Frontend documentation
│
└── README.md # This file
For detailed project structure:
Once the backend is running, visit:
- Swagger UI: http://localhost:8000/docs
- ReDoc: http://localhost:8000/redoc
For detailed API endpoint documentation, see:
VisionGuard uses JWT-based authentication with role-based access control:
- OWNER: Full access to all features, can manage shops and users
- MANAGER: Limited access to shop management and monitoring
- User logs in with credentials
- Server returns access token (30 min) and refresh token
- Access token is sent with each API request
- Refresh token is used to get new access token when expired
- Persistent connection with exponential backoff reconnection
- Heartbeat mechanism (30s ping/pong)
- Automatic stale connection cleanup
- Real-time anomaly alerts
- Low-latency peer-to-peer video streaming
- STUN server configuration for NAT traversal
- Automatic connection management
- Frame-by-frame AI processing
The system includes a Telegram bot for remote monitoring:
- Receive instant alerts for detected anomalies
- View camera snapshots
- Check system status
- Remote shop management
For testing instructions, see:
For more detailed information, see: