Asymptote OCR with Enhanced Image Processing

This repository combines computer vision-based image preprocessing with a RAG (Retrieval-Augmented Generation) system to generate high-quality Asymptote code from geometric images.

New Features

Enhanced Image Processing Pipeline

• Computer Vision Preprocessing: Uses OpenCV-based algorithms to extract geometric features
• Geometric Feature Detection: Automatically detects lines, curves, and junction points
• Enhanced RAG Input: Provides precise geometric data to improve LLM analysis
• Fallback System: Gracefully falls back to original processing if CV fails

Easy Algorithm Switching

• One-line change: Switch between different image processing algorithms
• Configurable: Adjust parameters via environment variables
• Extensible: Easy to add new algorithms

Architecture

User Upload → Computer Vision Processing → Enhanced Gemini Analysis → RAG System → Final Asymptote Code

1. Image Upload: User uploads geometric image
2. CV Processing: ffcolor.py extracts lines, curves, junctions
3. Enhanced Analysis: Gemini gets both image + geometric data
4. RAG Processing: Vector search with enhanced descriptions
5. Code Generation: Final asymptote code output

Setup

Prerequisites

• Python 3.8+
• Node.js 18+ (for frontend)
• MongoDB with vector search index

Backend Setup

cd asy/server

# Install dependencies
pip install -r requirements.txt

# Set environment variables
export VOYAGE_API_KEY="your_voyage_key"
export GEMINI_API_KEY="your_gemini_key"
export MONGODB_URI="your_mongodb_uri"

# Run server
python init.py

Frontend Setup

cd asy/client

# Install dependencies
npm install

# Run development server
npm run dev

Configuration

Switching Image Processing Algorithms

# Use ffcolor algorithm (default)
export IMAGE_PROCESSOR_ALGORITHM=ffcolor

# Use alternative algorithm (when implemented)
export IMAGE_PROCESSOR_ALGORITHM=cx

Adjusting ffcolor Parameters

export FFCOLOR_AREA_THRESH=200        # Minimum area for components
export FFCOLOR_DEV_THRESH=3.0         # Line deviation threshold
export FFCOLOR_MIN_LINE_LEN=15        # Minimum line length
export FFCOLOR_MIN_SEG_LEN=15         # Minimum segment length
export FFCOLOR_ANGLE_THRESH=5.0       # Angle threshold for merging
export FFCOLOR_JUNCTION_MERGE_DIST=15 # Junction clustering distance

Adding New Algorithms

1. Extend the ImageProcessor Class

# In pic-asy/image_processor.py
def _process_with_new_algorithm(self, image_path: str, **kwargs):
    # Your algorithm implementation
    pass

2. Update Configuration

# In asy/server/config.py
SUPPORTED_ALGORITHMS = [
    'ffcolor',
    'new_algorithm',  # Add your algorithm
]

3. Switch Algorithm

export IMAGE_PROCESSOR_ALGORITHM=new_algorithm

Testing

Test Image Processing

cd pic-asy
python test_processor.py

Test Full Pipeline

1. Start backend: cd asy/server && python init.py
2. Start frontend: cd asy/client && npm run dev
3. Upload an image through the web interface
4. Check processing information and generated code

What You'll See

Processing Information Display

• Algorithm Used: Which CV algorithm processed the image
• Features Detected: Count of lines, curves, junctions found
• Enhanced Description: Gemini's analysis using CV data
• Generated Code: Final asymptote code

Example Output

Processing Information:
Algorithm: ffcolor
Features Detected:
  - Lines: 4
  - Curves: 0
  - Junctions: 4

Enhanced Description: [Click to expand]
Generated Asymptote Code: [In textarea]

How It Works

1. Image Processing (ffcolor.py)

• Skeletonization: Converts image to skeleton representation
• Feature Extraction: Finds lines, curves, and junctions
• Coordinate Mapping: Maps features to image coordinates
• Visualization: Creates debug image showing detected features

2. Enhanced Gemini Analysis

• Dual Input: Original image + processed visualization
• Structured Data: Precise geometric coordinates and classifications
• Better Understanding: Gemini can focus on relationships rather than basic detection

3. Improved RAG

• Better Queries: Enhanced descriptions lead to better vector search
• Relevant Context: More accurate retrieval of asymptote examples
• Quality Output: Higher quality final code generation

Troubleshooting

Common Issues

1. Import Errors: Ensure pic-asy is in Python path
2. CV Failures: Check image format and size
3. Fallback Mode: System automatically uses original method if CV fails

Debug Mode

export DEBUG=True
# Check server logs for detailed processing information

Future Enhancements

• Multiple Algorithm Support: Easy switching between CV approaches
• Parameter Tuning: Web interface for adjusting CV parameters
• Batch Processing: Process multiple images simultaneously
• Performance Metrics: Track accuracy and processing time

License

This project is licensed under the MIT License - see the LICENSE file for details.