Fibonacci's Lens

An intelligent photography composition assistant using machine learning and computer vision

Fibonacci's Lens is an iOS camera application that uses advanced image segmentation and face detection to provide real-time composition suggestions for portrait photography. The app leverages Apple's Vision framework and Core ML to analyze facial features and guide photographers to achieve better composition using classical photography rules.

🎯 Features

Real-Time Composition Analysis

• Live Camera Preview: Full-featured camera interface with zoom controls and orientation support
• Smart Composition Guidelines: Supports two classic photography composition techniques:
  • Rule of Thirds: Traditional grid-based composition guidance
  • Golden Ratio: Mathematical composition based on the golden ratio (φ = 1.618)
• Dynamic Grid Overlay: Visual guides that adapt to the selected composition style

Intelligent Subject Detection

• Face Detection: Uses Vision framework's VNDetectFaceRectanglesRequest for accurate face detection
• Facial Feature Segmentation: Employs a custom Core ML model for detailed facial parsing with 19 different segments:
  • Background, skin, eyebrows (left/right), eyes (left/right), eye glasses
  • Ears (left/right), ear rings, nose, mouth, lips (upper/lower)
  • Neck areas, clothing, hair, and hat detection
• Subject Orientation Analysis: Determines face direction by analyzing skin and nose pixel distribution
• Distance Assessment: Calculates optimal shooting distance based on face size in frame

Smart Composition Feedback

• Real-Time Position Indicator: Visual dot that shows optimal subject placement
• Adaptive Recommendations: Position suggestions that change based on:
  • Face orientation (left, right, or centered)
  • Subject distance from camera
  • Selected composition rule
• Visual and Haptic Feedback:
  • Color-coded indicators (red for incorrect positioning)
  • Vibration feedback when composition improves
  • Smooth animations and transitions

Advanced Camera Features

• Multi-Orientation Support: Portrait and landscape modes with automatic adaptation
• High-Quality Capture: HEIF format support with high-resolution photo output
• Live Photo Support: Optional Live Photo capture capability
• Professional Controls: Manual zoom with slider interface
• Photo Library Integration: Direct saving to user's photo library

🛠 Technical Implementation

Core Technologies

Vision Framework

• Face Detection: VNDetectFaceRectanglesRequest for robust face boundary detection
• Core ML Integration: VNCoreMLRequest for running custom facial segmentation models
• Image Processing: Real-time processing of camera feed with VNImageRequestHandler

Core ML & Machine Learning

• Custom FaceParsing Model: iOS 14+ compatible model for detailed facial feature segmentation
• Multi-Array Processing: Custom SegmentationResultMLMultiArray wrapper for efficient ML output handling
• Real-Time Inference: Optimized processing pipeline to maintain smooth camera performance

Metal & Graphics

• Custom Metal Shaders: Efficient GPU-based image processing and rendering
• Texture Management: Metal-based texture generation and manipulation for camera feed
• Performance Optimization: Hardware-accelerated image processing pipeline

AVFoundation

• Camera Capture: Complete camera session management with AVCaptureSession
• Photo Processing: High-quality photo capture with AVCapturePhotoOutput
• Video Preview: Real-time camera preview with AVCaptureVideoPreviewLayer
• Format Selection: Dynamic camera format selection for optimal quality

Architecture & Design Patterns

MVC Architecture

• Separation of Concerns: Clear separation between UI, business logic, and data models
• Delegate Patterns: Extensive use of delegation for camera and ML processing callbacks
• Observer Pattern: Combine framework integration for reactive programming

Memory Management

• Efficient Processing: Careful memory management for real-time ML inference
• Texture Recycling: Metal texture reuse to minimize GPU memory allocation
• Background Processing: Proper queue management for camera and ML operations

User Experience

• Permission Management: Comprehensive privacy permission handling using SPPermissions
• Onboarding Flow: Welcome screen with privacy explanations
• Accessibility: Proper orientation handling and device adaptation
• Error Handling: Graceful fallbacks for unsupported devices or permission denial

Key Algorithms

Composition Calculation

// Rule of Thirds positioning
points.leftUp = CGPoint(x: width/3, y: height/3)
points.rightUp = CGPoint(x: 2*width/3, y: height/3)

// Golden Ratio positioning  
points.leftUp = CGPoint(x: Double(width)/2.618, y: Double(height)/2.618)
points.rightUp = CGPoint(x: Double(width)*1.618/2.618, y: Double(height)/2.618)

Face Direction Analysis

The app analyzes pixel distribution to determine face orientation:

• Counts skin pixels on left and right sides of the nose
• Uses a threshold-based system to determine optimal positioning
• Adapts composition recommendations based on subject's gaze direction

Averaging & Stabilization

• Bounding Box History: Maintains a rolling average of face detection results for stable positioning
• Confidence Scoring: On/off counting system to ensure stable recommendations
• Temporal Smoothing: Prevents jittery recommendations through frame averaging

📱 System Requirements

• iOS 14.0+ (for Core ML facial segmentation)
• iPhone with A12 Bionic or later (recommended for optimal ML performance)
• Camera permissions required
• Photo Library access (for saving captured photos)

🎨 Photography Composition Rules

Rule of Thirds

The classic photography guideline that divides the frame into nine equal sections. Key subject elements should be placed along these lines or at their intersections for more balanced and visually appealing compositions.

Golden Ratio

Based on the mathematical ratio φ (phi) ≈ 1.618, this composition technique creates more natural and aesthetically pleasing arrangements. The app calculates golden ratio points dynamically based on the camera frame dimensions.

🔧 Development Notes

Performance Optimizations

• Frame Rate Management: Processes every 4th frame to balance accuracy with performance
• Metal Integration: GPU-accelerated processing for real-time performance
• Memory Efficiency: Careful management of ML model memory usage
• Background Threading: Proper queue management to keep UI responsive

Device Compatibility

• Adaptive UI: Dynamic layout adjustments for different screen sizes
• Orientation Handling: Full support for portrait and landscape photography
• Performance Scaling: Adjusts processing intensity based on device capabilities

Privacy & Security

• Local Processing: All ML inference happens on-device
• Permission Transparency: Clear explanations for camera and photo library access
• No Data Collection: No user data or photos are transmitted externally

📚 Third-Party Dependencies

• SPPermissions: Elegant permission request handling
• CardSlider: Interactive composition selection interface

🤝 Contributing

This project demonstrates advanced iOS development techniques including:

• Real-time computer vision and machine learning
• Custom Metal shader programming
• Professional camera application development
• Advanced UI/UX design patterns

---

Created with ❤️ for photographers who want to improve their composition skills through intelligent assistance