A comprehensive leaderboard tracking the state-of-the-art (SOTA) performance of:
- Vision-Language-Action (VLA) models across multiple robotics benchmarks
- Dexterous Manipulation models across hand manipulation benchmarks
🌐 Live Demo: https://sota.evomind-tech.com
| Benchmark | Description | Primary Metric |
|---|---|---|
| LIBERO | Lifelong robot learning with 130 language-conditioned manipulation tasks | Average Success Rate (%) |
| LIBERO Plus | Extended LIBERO with 6 robustness categories (camera, robot, language, light, background, layout) | Average Success Rate (%) |
| Meta-World | Multi-task learning with 50 distinct robotic manipulation tasks | Average Success Rate (%) |
| CALVIN | Long-horizon language-conditioned tasks (ABC→D, ABCD→D, D→D settings) | Average Completed Tasks (0-5) |
| RoboChallenge | Real-world robotic manipulation benchmark with diverse household tasks | Score |
| RoboCasa-GR1-Tabletop | Household tabletop manipulation tasks in realistic environments using GR1 robot | Average Success Rate (%) |
| RoboCasa365 | Generalist robot policy evaluation on atomic and composite household manipulation tasks | Average Success Rate (%) |
| RoboTwin 2.0 | Scalable data generator and benchmark with strong domain randomization for bimanual manipulation | Hard Success Rate (%) |
| Benchmark | Description | Primary Metric |
|---|---|---|
| Adroit | In-hand manipulation benchmarks featuring the Adroit hand and diverse object tasks | Mean Success Rate (%) |
| DexArt | Articulated object manipulation and tool use tasks with dexterous hands | Mean Success Rate (%) |
| Bi-DexHands | Bimanual dexterous manipulation tasks requiring coordinated two-hand control | Mean Success Rate (%) |
- 📈 Interactive Leaderboards - Sortable tables with expandable details for each model
- Dual Track Coverage - VLA models and dexterous manipulation models in separate sections
- 📉 Progress Visualization - Scatter plots showing development over time for both tracks
- 🔍 Smart Filtering - Filter by model type (SFT/RL), benchmark settings, and open-source status
- 🔎 Model Search - Dedicated
/modelspage with fuzzy search across all benchmarks; quickly find any model's scores, ranks, and leaderboard links from a single interface; also accessible via the search bar on the homepage - 🌍 Bilingual Support - English and Chinese (中文) interface
- Framework: Next.js 14 (App Router)
- Styling: Tailwind CSS
- Charts: Recharts
- Deployment: GitHub Pages (Static Export)
- Language: TypeScript
- Node.js 18+
- npm or pnpm
# Clone the repository
git clone https://github.com/MINT-SJTU/Evo-SOTA.io.git
cd Evo-SOTA.io
# Install dependencies
npm install
# Start development server
npm run devOpen http://localhost:3000 in your browser.
npm run buildThe static files will be generated in the out/ directory.
Evo-SOTA.io/
├── app/ # Next.js App Router pages
│ ├── page.tsx # VLA Homepage
│ ├── models/ # Model search page (fuzzy search across all benchmarks)
│ ├── methodology/ # VLA Methodology page
│ ├── benchmarks/ # VLA Benchmark leaderboard pages
│ │ ├── libero/
│ │ ├── liberoplus/
│ │ ├── calvin/
│ │ ├── metaworld/
│ │ ├── robochallenge/
│ │ ├── robocasa_gr1_tabletop/
│ │ ├── robocasa365/
│ │ └── robotwin2/
│ └── dex/ # Dexterous Manipulation section
│ ├── page.tsx # Dex Homepage
│ ├── methodology/ # Dex Methodology page
│ ├── leaderboard/ # Dex unified leaderboard
│ └── benchmarks/ # Dex benchmark detail pages
├── components/ # React components
│ ├── [VLA components]
│ └── dex/ # Dex-specific components
├── data/ # VLA JSON data files & processing scripts
│ ├── libero.json
│ ├── liberoPlus.json
│ ├── calvin.json
│ ├── metaworld.json
│ ├── robochallenge.json
│ ├── robocasa_gr1_tabletop.json
│ ├── robocasa365.json
│ ├── robotwin2.json
│ └── DataProcess.py # CSV to JSON converter
├── lib/ # Utilities & i18n
│ ├── i18n.ts # Translations for both VLA and Dex
│ └── dex/ # Dex-specific utilities
├── public/ # Static assets
│ ├── data/ # VLA data files
│ └── dex/ # Dex data and assets
│ └── data/
└── types/ # TypeScript type definitions
Found errors or want to submit your model? Reach out to us:
- GitHub Issues: Create an issue
- Email: business@evomind-tech.com
- Wechat group: Join our Wechat Group
Contributions are welcome! If you'd like to:
- Add a new VLA model: Please provide the paper link and benchmark scores
- Add a new Dexterous Manipulation model: Please provide the paper link and performance metrics
- Report an error: Open an issue with details
- Suggest improvements: PRs are appreciated
- All benchmark results are collected from original papers or reproduced by third parties
- Results may vary due to different evaluation protocols, random seeds, or implementation details
- This leaderboard is for research reference only and does not represent official rankings
- Please verify results with original papers before citation
- We categorize entries by specific training methodologies (e.g., SFT, RL) to keep comparisons as fair and objective as possible
- For dexterous manipulation benchmarks, we track performance across different hand configurations and task settings
- Ye Zewei: Website development and updates, Data collection
- Li Yiqin: Website updates, Data collection
- Mao Yiran: Website updates, Data collection
- Ye Kai: Dexterous hand pages development and data collection
- Lin Tao (@EvoMind): Project Lead
- Thanks to all researchers who contributed to the VLA and dexterous manipulation fields
- VLA Benchmark creators: LIBERO, LIBERO Plus, CALVIN, Meta-World, RoboChallenge, RoboCasa-GR1-Tabletop, RoboCasa365, RoboTwin 2.0 teams
- Dexterous Manipulation benchmark creators: Adroit, DexArt, Bi-DexHands teams
- https://github.com/EvanNotFound/vercount for visitor statistics
Note: This is a community-maintained project. For official benchmark results, please refer to the original papers and repositories. The leaderboard includes both VLA models for language-conditioned manipulation and dexterous manipulation models for hand-based control tasks.