Real-Time AR Text Scanner

A lightweight, real-time Optical Character Recognition (OCR) pipeline that acts as a local, offline alternative to tools like Google Lens. This application utilizes a YOLO11-Nano model for high-speed text detection and EasyOCR for accurate character recognition, all wrapped in an interactive Streamlit web interface.

Demo and Working:

• App Working for Image Text Recognition!

• App Working for Live Video Text Recognition!

You can find the video of the working of the app and its functions on my LinkedIn profile: https://www.linkedin.com/in/syed-muhammad-hassan-aa112928b/

🚀 Features

• Live Augmented Reality (AR) Overlay: Processes webcam feeds in real-time, snapping bounding boxes around physical text and overlaying the digital translation directly on the screen.
• Static Image Extraction: Upload any image to instantly extract its text into a clean, copy-pasteable document format.
• Edge-Optimized: Built specifically to run locally on standard hardware without requiring massive cloud GPUs.
• Frame Caching: Implements custom frame-skipping and caching logic to maintain smooth FPS during live video processing.

🧠 The Architecture

This pipeline solves the computational bottleneck of standard OCR by splitting the task into two highly optimized phases:

1. Detection (The Eyes): Ultralytics YOLO11 (Nano variant) is fine-tuned to detect text regions at blazing speeds. Thanks to https://huggingface.co/RoyRud1902 for the pre-trained model for text boundbox, I really appricate your work!
2. Recognition (The Brain): EasyOCR (a PyTorch-based CRNN architecture) crops the bounding boxes and decodes the pixels into string data.

💻 Installation & Setup

1. Clone the repository

git clone [https://github.com/your-username/realtime-ar-text-scanner.git](https://github.com/your-username/realtime-ar-text-scanner.git)
cd realtime-ar-text-scanner

2. Install dependencies Ensure you have Python installed, then run:

pip install -r requirements.txt

3. Model Weights

• The EasyOCR weights will download automatically on the first run.
• Download a YOLO11 text-detection weights file (.pt) and place it in the Models/ directory. Update the path in app.py if necessary.

4. Run the Streamlit App

streamlit run app.py

The web interface will automatically open in your default browser at http://localhost:8501.

🗺️ Project Roadmap

• [x] Phase 1: End-to-end pipeline using YOLO11 + EasyOCR with a Streamlit GUI.
• [ ] Phase 2: Train and integrate a custom to replace the EasyOCR backend for an even lighter computational footprint. (It would might not be as intelligent to seperate a word after detection, but will see🙂)

🤝 Author & Acknowledgments

"Syed Muhammad Hassan" - AI Engineer/Computer Vision Developer

Developed as part of the engineering initiatives at AgenticEra Systems.