Skip to content

Repository files navigation

Blend-CamMoCap

Point your phone. Hit record. Done.
Turn an Android phone into a virtual camera for Blender. Move the phone and Blender's camera moves with it over Wi-Fi,
and one button bakes the motion into ordinary, editable keyframes.

Android Kotlin Jetpack Compose Blender 6DOF License

demo-v0.1.mp4

News  ·  Features  ·  Get started  ·  Download  ·  How it works  ·  License


News

  • 29 Jul · A viewfinder on the phone. Blender's camera streams live to the handset, so you can frame the shot without looking at the monitor.
  • 26 Jul · The interface, rebuilt. Turn the phone and every control turns with it, while the dock stays exactly where your thumb left it.
  • 25 Jul · Lens control. A zoom dial drives Blender's focal length, and a zoom you perform mid-take is recorded with the move.
  • 21 Jul · Takes you can trust. A searchable library on the phone, and every take uploaded losslessly after the cut rather than assembled from the live stream.

What it does

No account. No cloud. No subscription. No twelve-toggle setup screen. One screen, one button. The phone and Blender talk directly on your Wi-Fi, and nothing ever leaves the room.

Feature What you get
Live rotation tracking Pan, tilt and roll the phone; Blender's camera follows instantly, with latest-wins UDP so Wi-Fi jitter never rubber-bands the shot.
6DOF (position) via ARCore Opt-in: walk, and the camera walks with you for real handheld parallax, not just rotation. When tracking degrades the phone says why ("point at a detailed surface", "add more light") and holds position instead of lurching.
Blender's camera, on the phone A live viewfinder of the actual shot, so you can frame without looking at the monitor. It follows the camera whatever you are doing on the PC — another editor, another window, even minimised — because it draws the camera itself rather than mirroring your screen. Tell it how much effort to spend and Blender picks the detail to match your machine.
Lens control on the phone A zoom dial under the viewfinder drives Blender's focal length, and a zoom you perform during a take is recorded with it. Taking the lens adopts the one you already set, so it never reframes your shot; hand it back with a tap.
Record to editable keyframes One button bakes the move, resampled at scene FPS onto normal F-curves you can trim and tweak like any animation.
Takes are lossless, not "whatever arrived" The phone records the take locally at full rate and uploads it after the cut, so Wi-Fi glitches only ever hit the live preview — never the take you keep.
Shoot with no PC in sight Record straight to the phone, walk back, and apply the take to Blender later. The phone keeps a searchable take library you can rename, re-apply and delete.
Shoot over your animation The timeline plays while you record, so you can hand-track a moving subject. Every take is kept; nothing is overwritten.
Live-synced controls Smoothing, movement-scale and lens mirror between the phone and the Blender panel in real time.
Pairing with no ritual Scan a QR code and you are paired. Tracking starts when you say Go live, so a fresh connection never yanks the camera away from what you were doing. Reopen the app and it silently reconnects.

Get started (5 minutes)

You need an Android phone (8.0+), Blender 4.2+, and both devices on the same Wi-Fi.

  1. Install the add-on. Grab the zip from Download (or build it with ./tools/dev-install.ps1), then in Blender open Edit › Preferences › Get Extensions › Install from Disk and pick the zip.
  2. Install the app. Use the APK from Download (or run cd android && ./gradlew.bat :app:assembleDebug and adb install it).
  3. Connect. In Blender's 3D viewport press N and open the add-on's side panel (its tab is labeled pocketrig, the in-app name is still a placeholder). Click Connect Phone and allow the Windows Firewall prompt. A QR code appears.
  4. Pair. On the phone tap Scan to Connect. (No scanner? Tap "enter address manually" and type the ip:port and code from the panel.)
  5. Go live. The camera does not start moving on pairing — that would yank the viewport away from whatever you were doing. Tap Go live on the phone (or Start tracking in the panel) when you are ready; wherever the camera is now becomes the neutral pose.
  6. Record. Hit the big red Record, move for about 10 seconds, then hit it again to stop. The phone shows "Baked N frames". Scrub the Blender timeline and edit the keyframes like any others.

Everything else is optional: tap 6DOF to add positional tracking, turn on Camera view in Adjust to see the shot on the phone, drag the zoom dial (or pinch the frame) to change the lens, and Center whenever you want the current framing to be the new neutral.

Trouble connecting? It is almost always router AP-isolation (join the phone's hotspot with the PC instead) or the firewall. See the troubleshooting table.


Download

Both halves are attached to every tagged release:

Latest release

Take Blend-CamMoCap-<version>-app.apk for the phone and Blend-CamMoCap-<version>-blender-addon.zip for Blender. The APK is not signed for the Play Store, so sideload it; the add-on installs on Blender 4.2+ (tested on 4.5, 5.0 and 5.1). Free for noncommercial use; see License.

Prefer to build both from source? QUICKSTART.md is a two-line job.


How it works

The phone converts its sensor pose into Blender's exact camera convention and streams it as small UDP packets: state, not events, so only the freshest pose ever matters and a dropped packet is a brief freeze rather than a rubber-band. Blender applies it at 120 Hz, and recording resamples the raw, timestamped stream at scene FPS.

The take you keep does not come from that stream. While you record, the phone also writes every sample to its own storage at full rate and uploads that file over TCP after the cut, so the lossy, latest-wins path is only ever the preview — a Wi-Fi glitch mid-take costs you nothing.

Frames travel the other way too. Blender draws the scene camera into an offscreen buffer and sends it to the phone as JPEG over its own TCP connection, so the viewfinder shows the shot rather than a mirror of your screen — it keeps working while you are in the shader editor, in another window, or with Blender minimised. What that costs the PC depends on the scene and the machine, so the host measures both and picks the resolution itself (spec/viewport-channel.md).

flowchart LR
    S["Phone sensors<br/>game-rotation-vector, ARCore VIO"] --> C["Convert to Blender<br/>camera frame"]
    C --> N["UDP, latest-wins<br/>100 Hz POSE"]
    C --> F["Full-rate recording<br/>on the phone"]
    N --> H["Blender add-on<br/>120 Hz apply"]
    H --> V["Live camera<br/>in the viewport"]
    H --> P["Camera drawn offscreen<br/>JPEG over TCP"]
    P --> W["Viewfinder<br/>on the phone"]
    F --> U["TCP upload<br/>after the cut"]
    U --> R["Bake<br/>slerp-resample at scene FPS"]
    R --> K["Editable<br/>keyframes"]

    style S fill:#132a17,color:#c9f7d4,stroke:#3ddc84
    style N fill:#1e3a5f,color:#cfe3ff,stroke:#3b82f6
    style F fill:#132a17,color:#c9f7d4,stroke:#3ddc84
    style U fill:#1e3a5f,color:#cfe3ff,stroke:#3b82f6
    style H fill:#3a1e12,color:#ffe0cf,stroke:#f5792a
    style K fill:#3b0764,color:#efe1ff,stroke:#a855f7
    style P fill:#1e3a5f,color:#cfe3ff,stroke:#3b82f6
    style W fill:#132a17,color:#c9f7d4,stroke:#3ddc84
Loading

The wire protocol (spec/protocol.md) is small, versioned, and pinned by shared cross-platform test vectors: the same fixtures the Android (Kotlin) and Blender (Python) sides both prove themselves against, so any future client such as an iOS port can be verified for free. Live feel is not left to guesswork either. A Motion Lab measures filter lag, jitter attenuation and bake fidelity, and gates them against committed baselines on every test run.


Repository layout

Path What License
android/ Android app: Kotlin + Jetpack Compose, ARCore 6DOF PolyForm Noncommercial 1.0.0
blender/ Blender add-on: extension format, Blender 4.2+ GPL-3.0-or-later
spec/ The wire protocol and coordinate conventions (normative) CC BY-NC-4.0
fixtures/ Cross-platform test vectors every implementation must pass CC BY-NC-4.0
tools/ Dev harness: fake phone, Motion Lab, Viewport Lab, install and test scripts GPL-3.0-or-later
dev/ Developer notes: architecture, platform constraints, testing, measurement informational

Develop and test without a phone at all:

python -m pytest blender/tests            # Blender core, desktop Python (no Blender needed)
python tools/fake_phone.py --mode walk    # drive the add-on with a synthetic 6DOF phone
./tools/dev-install.ps1                   # build + install the add-on (then restart Blender)
./tools/test-all-blenders.ps1             # headless matrix across Blender 4.5, 5.0 and 5.1
cd android; ./gradlew.bat :protocol:test  # frame math and codec vs the shared fixtures

See QUICKSTART.md for the full dev guide and CONTRIBUTING.md to get involved. dev/ has the shorter, sharper notes: how the pieces fit (architecture), the platform rules that will bite you (constraints), which suite covers what (testing), and how the measurement labs work (measurement).


License

Free for noncommercial use. Commercial use requires permission; contact bratuka@bratuka.dev. Licenses differ per component (full detail in LICENSES.md):

  • App: PolyForm Noncommercial 1.0.0
  • Spec and fixtures: CC BY-NC-4.0
  • Blender add-on and tools: GPL-3.0-or-later (Blender's bpy is GPL, so the add-on must be GPL-compatible; GPL still forbids closed, proprietary forks)

Designed and built by BrAtUkA

BrAtUkA

About

Android phone as a virtual camera for Blender: move the phone, record, get editable keyframes. Wi-Fi, 6DOF via ARCore.

Topics

Resources

Contributing

Stars

Watchers

Forks

Releases

Contributors

Languages