็ฎไฝไธญๆ ยท English ยท ะ ัััะบะธะน ยท ํ๊ตญ์ด ยท ๆฅๆฌ่ช
In a nutshell: An enhanced spatial computing tool based on ORB-SLAM2 for Android, providing stable 6DoF spatial positioning for mobile devices.
ใSolemn Thanksใ Original ORB-SLAM2 Core๏ผ
https://github.com/raulmur/ORB_SLAM2
ใSolemn Thanksใ Original Android ORB-SLAM2 Project๏ผ
https://github.com/Martin20150405/SLAM_AR_Android.git
The lowercase 's' in ORB-SLAM2s stands for Smart ยท Swift ยท Small, emphasizing enhanced intelligence, faster performance, and lighter weight compared to the original ORB-SLAM2.
This project uses lowercase 's'. Another paper with uppercase 'S' named ORB-SLAM2S:
(Y. Diao, R. Cen, F. Xue and X. Su, "ORB-SLAM2S: A Fast ORB-SLAM2 System
with Sparse Optical Flow Tracking," 2021 13th International Conference
on Advanced Computational Intelligence (ICACI), Wanzhou, China, 2021,
pp. 160-165, doi: 10.1109/ICACI52617.2021.9435915.)
It shares similar performance optimization concepts with this project, though its methods have not yet been integrated. This paper can serve as a valuable reference for future optimization directions. Sincere gratitude to the scholars for their contributions and sharing.
ORB-SLAM2s is an enhanced spatial computing tool based on ORB-SLAM2 for Android. It supports real-time sparse point cloud mapping, map saving/loading, and relocation matching. By combining computer vision, inertial navigation (IMU), and Augmented Reality (AR) technologies, this project provides stable 6DoF (Six Degrees of Freedom) spatial positioning for mobile devices.
- Lightweight Design: Fewer dependencies and streamlined code, making it easier to compile and deploy on mobile platforms.
- About IMU & VIO Support:
- High Barrier of ORB-SLAM3: The VIO in the official implementation is tightly coupled, requiring high-quality IMU and strict Camera-IMU timestamp synchronization. This is common in robotics but difficult to achieve "out of the box" on Android phones via standard APIs.
- Android Reality: Most existing Android devices use asynchronous acquisition for Camera and IMU, lacking a unified hardware synchronization framework, and consumer-grade IMUs have significant noise/drift. Without strict calibration and synchronization, VIO is more prone to divergence than pure visual SLAM. The SENSOR_TIMESTAMP method is only available in Android 11 and later.
- Low Resource Consumption: Lower CPU and RAM requirements compared to ORB-SLAM3.
- Practical Efficiency: Performance in monocular-only scenarios is comparable to ORB-SLAM3 for standard mobile use cases.
- ๐บ๏ธ Point Cloud SLAM Mapping โโ Real-time sparse mapping based on monocular camera input
- ๐พ Map Persistence โโ Save maps to local storage and reload them for future sessions
- ๐ฏ Relocalization & Matching โโ Pose estimation and feature matching upon loading existing maps
- ๐ Confidence Visualization โโ Visual tracking of keypoints and matching statistics between current frames and the loaded map
- ๐ Plane Detection โโ Intelligent floor/surface detection based on current pose and point cloud data
- ๐จ Native AR Rendering โโ High-performance 3D rendering via cgltf and OpenGL ES, supporting fast loading of GLB/glTF 3D models and interactive placement on detected planes
- ๐ Dark Frame Detection โโ Automatically skips dark or low-quality frames to prevent SLAM thread blocking
- ๐ฑ๏ธ AR Object Management โโ Place, scale (pinch gesture), and interact with 3D objects
- ๐งญ 3DOF Orientation Tracking โโ Three-degrees-of-freedom orientation tracking using onboard device sensors (Rotation Vector / Accelerometer + Magnetometer)
- ๐ Multi-Map Support โโ Simultaneous loading and matching of multiple map files
Currently tested primarily on Qualcomm Snapdragon platform CPUs:
| SoC | Device | Performance |
|---|---|---|
| Snapdragon 8 Elite | Xiaomi 15 | 30 FPS |
| Snapdragon 8+ Gen1 | Redmi K60 | 30 FPS |
| Snapdragon 870 | Xiaomi 10S | 30 FPS |
| Snapdragon 7s Gen 2 | Redmi Pad Pro | 30 FPS |
| Snapdragon 835 | Xiaomi 6 | 30 FPS |
| Snapdragon AR1 Gen 1 | Rokid Glass3 | 20-30 FPS |
To ensure a smooth user experience, the system monitors exposure levels. When the environment is too dark, SLAM tracking is paused to avoid computational lag and "lost" states.
- Sparse Point Cloud SLAM Mapping
- Map Save / Load functionality
- Relocalization Matching
- Basic AR Rendering Engine
- Dark Frame Skip Logic
- 3D AR Object Management
- Simultaneous loading and matching of multiple map files
- Improve mapping speed and initialization
- Enhance AR stability and 6DoF robustness
- Optimize rendering pipeline for higher frame rates
- Deepen sensor fusion (VIO โโ Visual Inertial Odometry)
- SLAM frequency downsampling and adaptive noise handling
- Refined sensor gating logic
- Integration with Unity3D
git clone --recursive https://github.com/Olsc/Android_ORB-SLAM2s.git
This project is built upon the following excellent open-source libraries:
Authors: Raul Mur-Artal, Juan D. Tardos, J. M. M. Montiel and Dorian Galvez-Lopez (DBoW2)
ORB-SLAM2 is a real-time SLAM library for Monocular, Stereo and RGB-D cameras that computes the camera trajectory and a sparse 3D reconstruction (in the stereo and RGB-D case with true scale). It is able to detect loops and relocalize the camera in real time. We provide examples to run the SLAM system in the KITTI dataset as stereo or monocular, in the TUM dataset as RGB-D or monocular, and in the EuRoC dataset as stereo or monocular.
[Monocular] Raรบl Mur-Artal, J. M. M. Montiel and Juan D. Tardรณs. ORB-SLAM: A Versatile and Accurate Monocular SLAM System. IEEE Transactions on Robotics, vol. 31, no. 5, pp. 1147โ1163, 2015. (2015 IEEE Transactions on Robotics Best Paper Award). PDF
[Stereo and RGB-D] Raรบl Mur-Artal and Juan D. Tardรณs. ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras. IEEE Transactions on Robotics, vol. 33, no. 5, pp. 1255โ1262, 2017. PDF
[DBoW2 Place Recognizer] Dorian Gรกlvez-Lรณpez and Juan D. Tardรณs. Bags of Binary Words for Fast Place Recognition in Image Sequences. IEEE Transactions on Robotics, vol. 28, no. 5, pp. 1188โ1197, 2012. PDF
The ORB-SLAM2 core library is released under a GPLv3 license. For a list of all code/library dependencies (and associated licenses), please see Dependencies.md.
For a closed-source version of ORB-SLAM2 for commercial purposes, please contact the authors: orbslam (at) unizar (dot) es.
This project adopts a modular license architecture based on Android IPC process isolation:
app/Module: Licensed under the Apache License, Version 2.0 (Apache-2.0). Contains UI, camera preview, native cgltf/OpenGL ES 3D rendering, and sensor components.MenthaAR/Engine Module: Derived from ORB-SLAM2 and released under the GNU General Public License v3.0 (GPLv3). Contains native C++ SLAM algorithms and processing.- Inter-Process Communication (IPC): The
appmodule andMenthaARengine run in separate Android OS processes and communicate strictly via Binder IPC and SharedMemory (Ashmem/memfd), preserving license boundaries between modules.
See LICENSE.txt, app/LICENSE, and MenthaAR/LICENSE for details.
For project collaboration or other field cooperation inquiries, please contact: OlscStudio@outlook.com
This project relies on several excellent open-source third-party libraries. We strictly adhere to their respective open-source licenses:
- OpenCV โโ Apache 2.0 License
- srrg_hbst โโ BSD 3-Clause License. Used for fast and scalable image matching.
- DBoW2 โโ BSD License. Used for basic vocabulary vectors and feature matching structures.
- g2o โโ BSD License (core components). Used for non-linear optimization.
- Eigen3 โโ MPL2 (Mozilla Public License v2.0). Used for matrix operations and algebraic calculations.
- cgltf โโ MIT License. Single-file C glTF 2.0 / GLB parser used for native, high-performance 3D model loading in the app module.
For any issues related to third-party licenses, please refer to their respective official repositories.
If you use ORB-SLAM2 (Monocular) in an academic work, please cite:
@article{murTRO2015,
title={{ORB-SLAM}: a Versatile and Accurate Monocular {SLAM} System},
author={Mur-Artal, Ra\'ul, Montiel, J. M. M. and Tard\'os, Juan D.},
journal={IEEE Transactions on Robotics},
volume={31},
number={5},
pages={1147--1163},
doi = {10.1109/TRO.2015.2463671},
year={2015}
}If you use ORB-SLAM2 (Stereo or RGB-D) in an academic work, please cite:
@article{murORB2,
title={{ORB-SLAM2}: an Open-Source {SLAM} System for Monocular,
Stereo and {RGB-D} Cameras},
author={Mur-Artal, Ra\'ul and Tard\'os, Juan D.},
journal={IEEE Transactions on Robotics},
volume={33},
number={5},
pages={1255--1262},
doi = {10.1109/TRO.2017.2705103},
year={2017}
}If you use this Android adaptation (ORB-SLAM2s) in an academic work, please acknowledge this work appropriately.
We use OpenCV to manipulate images and features.
We have integrated srrg_hbst for fast and scalable image matching. It significantly improves loop closing and relocalization performance compared to DBoW2.
Required by g2o (see below). Download and install instructions can be found at: http://eigen.tuxfamily.org.
We use modified versions of the DBoW2 library to perform place recognition and g2o library to perform non-linear optimizations. Both modified libraries (which are BSD) are included in the Thirdparty folder.
Reference: https://github.com/Olsc/Android_3dof
This project has not yet completed research and integration.
Character: Mentha