OVMobileBench - OpenVINO™ Mobile Benchmarking Pipeline

OVMobileBench is an end-to-end automation pipeline for benchmarking OpenVINO inference performance on mobile devices. It handles the complete workflow from building OpenVINO runtime, packaging models, deploying to devices, executing benchmarks, and generating comprehensive reports.

🚀 Quick Start

# Install from source
git clone https://github.com/embedded-dev-research/OVMobileBench.git
cd OVMobileBench
pip install -e .

# Run complete benchmark pipeline
ovmobilebench all -c experiments/android_example.yaml

# View results
cat experiments/results/*.csv

OpenVINO Distribution Modes

OVMobileBench supports three flexible ways to obtain OpenVINO:

1. Build Mode - Build OpenVINO from source
2. Install Mode - Use pre-built OpenVINO installation
3. Link Mode - Download OpenVINO archive (supports "latest" for auto-detection)

See Configuration Reference for details.

📚 Documentation

• Getting Started Guide - Installation and first benchmark
• OpenVINO Modes Guide - Three ways to obtain OpenVINO runtime
• User Guide - Complete usage documentation
• Configuration Reference - YAML configuration schema
• Device Setup - Android/Linux device preparation
• Android Installer Module - Automated Android SDK/NDK setup
• Build Guide - Building OpenVINO for mobile
• Benchmarking Guide - Running and interpreting benchmarks
• Testing Guide - Running and writing tests
• CI/CD Integration - GitHub Actions and automation
• API Reference - Python API documentation
• Troubleshooting - Common issues and solutions

✨ Key Features

• 🔨 Flexible OpenVINO Distribution - Three modes: build from source, use existing install, or download archives
• 📦 Smart Packaging - Bundle runtime, libraries, and models
• 🚀 Multi-Device - Deploy via ADB (Android) or SSH (Linux using paramiko)
• ⚡ Matrix Testing - Test multiple configurations automatically
• 📊 Rich Reports - JSON/CSV output with detailed metrics
• 🌡️ Device Control - Temperature monitoring, performance tuning
• 🔄 CI/CD Ready - GitHub Actions integration included
• 📈 Reproducible - Full provenance tracking of builds and runs
• 🤖 Android SDK/NDK Installer - Automated setup of Android development tools
• 🔗 Auto-Download - Fetch latest OpenVINO builds for your platform

🔧 Supported Platforms

Host OS	Host Arch	Device OS	Device Arch	Transport	Library	Status
Linux	x86_64/ARM64	Android	x86_64/ARM64	ADB	adbutils	✅ Stable
macOS	x86_64/ARM64	Android	x86_64/ARM64	ADB	adbutils	✅ Stable
Windows	x86_64/ARM64	Android	x86_64/ARM64	ADB	adbutils	✅ Stable
Linux	x86_64	Linux	ARM64/ARM32	SSH	paramiko	✅ Stable
macOS	x86_64/ARM64	Linux	ARM64/ARM32	SSH	paramiko	✅ Stable
Windows	x86_64/ARM64	Linux	ARM64/ARM32	SSH	paramiko	✅ Stable
Any	Any	iOS	ARM64	USB	-	🚧 Planned

📋 Requirements

• Python: 3.11+
• For Android targets:
  • Android NDK r26d+
  • CMake 3.24+
  • Ninja 1.11+
  • Android device with USB debugging enabled
• For Linux ARM targets:
  • SSH access to device
  • Cross-compilation toolchain

🎯 Use Cases

• Performance Testing - Measure inference speed across devices
• Regression Detection - Track performance changes over time
• Hardware Evaluation - Compare different SoCs and configurations
• Model Optimization - Find optimal runtime parameters
• CI/CD Integration - Automated testing in development pipelines

📖 Learn More

• Getting Started Guide
• Android SDK/NDK Setup
• Architecture Overview
• API Reference
• Contributing Guide
• Security Policy
• Changelog

📄 License

Apache License 2.0 - See LICENSE for details.

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

💬 Support

• 📝 GitHub Issues - Bug reports and feature requests
• 💡 Discussions - Questions and ideas
• 📧 Contact: [email protected]