i was not able to read my doctors handwriting so i made something that can (still under development).

Medical Prescription OCR

Overview

This project aims to convert doctors' handwritten medical prescriptions into digital text using a hybrid OCR system. It combines Google Cloud Vision API with a custom-trained transformer model optimized for medical text recognition.

Features

• Hybrid OCR system for higher accuracy
• Recognizes 50+ common medications and dosage instructions
• Structured output of medicines, dosages, and frequencies
• Synthetic data generator for training
• Streamlit-based user-friendly web interface

Setup

Prerequisites

• Python 3.8+
• Google Cloud account with Vision API enabled
• Service account JSON credentials for Google Cloud
• Optional GPU for faster model inference

Installatio dependencies

pip install -r requirements.txt

Prepare synthetic training data

Set environment variables

Create .env file with GOOGLE_APPLICATION_CREDENTIALS=./path/to/your/service-account-key.json GOOGLE_CLOUD_PROJECT=your-project-id CUSTOM_MODEL_PATH=./medical-ocr-mode

Setup base model

python setup_hybrid_model.py

Train custom model (optional)

python custom_medical_model.py

Run the web app

streamlit run app.py

Usage

• Upload prescription images via the web UI
• Choose to run hybrid OCR, Google Vision only, or custom model only
• View results with medicines, dosages, instructions, and confidence

Project Structure

medical-prescription-ocr/ ├── app.py # Main Streamlit app ├── hybrid_medical_ocr.py # Hybrid OCR combining Google Vision + custom model ├── custom_medical_model.py # Custom transformer training and dataset generation ├── data_preparation.py # Synthetic data generator script ├── train_medical_model.py # Custom model training script ├── setup_hybrid_model.py # Setup script for dependencies and base model ├── test_hybrid_system.py # Test suite for hybrid OCR system ├── requirements.txt # Python package dependencies ├── .env # Environment variables for credentials ├── data/ │ ├── medical_images/ # Training images folder │ └── annotations/ # CSV annotations folder └── medical-ocr-model/ # Saved custom model files

Contributing

Feel free to fork, improve, and send pull requests.

License

Licensed under Apache-2.0 License.

Contact

Open issues on GitHub for questions or feature requests.