Skip to content

Upgrade: dep bumps, daemon transport client, model export, CRDT scene sync - #42

Open
cartertbirchall wants to merge 9 commits into
ZiYang-xie:mainfrom
cartertbirchall:upgrade/v2
Open

Upgrade: dep bumps, daemon transport client, model export, CRDT scene sync#42
cartertbirchall wants to merge 9 commits into
ZiYang-xie:mainfrom
cartertbirchall:upgrade/v2

Conversation

@cartertbirchall

Copy link
Copy Markdown

Summary

Four parallel upgrades to the Auroch Syna platform.

  • Dependency bumps across the ML stack (torch, diffusers, transformers, etc.)
  • Daemon transport client — real WebSocket client so callers can talk to a remote daemon instead of loading models in-process
  • Model export tool — ONNX, TorchScript, and CoreML export for DA-2 and LaMa
  • CRDT scene sync — LWW registers + OR-Set for offline-first, conflict-free scene editing

1 — Dependency bumps (pyproject.toml)

Package Before After
torch ≥2.7.0 ≥2.12.0
torchvision ≥0.22.0 ≥0.27.0
diffusers ≥0.33.1 ≥0.38.0
transformers ≥4.48.3 ≥4.57.6 (keeping 4.x; 5.x has breaking API changes)
accelerate ≥1.1.1 ≥1.13.0
peft ≥0.7.1 ≥0.19.1
timm ≥1.0.15 ≥1.0.27
scikit-image ≥0.24.0 ≥0.26.0
einops ≥0.7.0 ≥0.8.2
opencv-python ≥4.12.0 ≥4.13.0
websockets ≥12.0 ≥14.0

New optional extra [convert] (onnx, onnxruntime, coremltools).
Note: nunchaku wheel URL still pins torch 2.7 — update that URL separately when testing CUDA.

2 — Daemon transport client (daemon/client.py)

DaemonClient — async WebSocket client matching server.py's wire protocol.

  • await client.command(kind, params) — sends command, awaits ack/error
  • client.subscribe(kinds) — opts into event stream
  • DaemonClient.run_command(...) — blocking one-shot wrapper
  • DaemonClient.ping(endpoint) — health check

DaemonModelClient — drop-in for ModelClient that routes generate_scene
to a daemon process, keeping the orchestrator process torch-free.

3 — Model export tool (tools/convert_model.py)

python tools/convert_model.py --model da2  --format onnx        -o models/da2.onnx
python tools/convert_model.py --model da2  --format torchscript  -o models/da2.pt
python tools/convert_model.py --model da2  --format coreml       -o models/da2.mlpackage
python tools/convert_model.py --model lama --format onnx         -o models/lama.onnx
python tools/convert_model.py --model lama --format coreml       -o models/lama.mlpackage

Includes ONNX graph validation, dynamic batch axes, CoreML fp16 targeting macOS 14+.

4 — CRDT scene sync (scene/crdt.py)

Primitive Purpose
LWWRegister[T] Last-Write-Wins register with timestamp + author tiebreak
ORSet[T] Observed-Remove Set; concurrent adds survive removes
SceneDelta Compact JSON diff: added/removed/changed objects + scalar fields
diff(a, b) Compute delta from snapshot a to b
merge(base, delta) Apply delta to produce merged snapshot
merge_snapshots(a, b) Three-way merge of two diverged snapshots

19 unit tests in tests/test_crdt.py — all pass, zero ML deps.

Test plan

  • pytest tests/ — 39 tests pass on CPU-only machine (no GPU needed)
  • auroch-syna info CLI starts and prints device info
  • python tools/convert_model.py --model lama --format torchscript -o /tmp/lama.pt succeeds with [ml] installed
  • DaemonClient.ping("ws://127.0.0.1:8765") returns True against a running daemon

🤖 Generated with Claude Code

cartertbirchall and others added 9 commits May 15, 2026 16:28
… sync

1. Dependency upgrades (pyproject.toml)
   torch 2.7→2.12, torchvision 0.22→0.27, diffusers 0.33→0.38,
   transformers 4.48→4.57, accelerate 1.1→1.13, peft 0.7→0.19,
   timm 1.0.15→1.0.27, scikit-image 0.24→0.26, einops→0.8.2,
   opencv-python→4.13, websockets 12→14.
   Add [convert] optional extra (onnx, onnxruntime, coremltools).
   Add mypy to [dev]. Note nunchaku wheel needs update for torch 2.12.

2. Daemon transport client (daemon/client.py)
   DaemonClient — async-first WebSocket client matching server.py wire
   protocol: send commands, await ack/error, receive events via callback.
   DaemonClient.run_command() and .ping() for blocking callers.
   DaemonModelClient — drop-in for ModelClient that routes generate_scene
   commands to a running daemon (keeps orchestrator process ML-free).

3. Model export tool (tools/convert_model.py)
   Exports DA-2 depth and LaMa inpaint models to ONNX, TorchScript, or
   CoreML. Supports dynamic batch axes on ONNX, fixed-shape tracing for
   CoreML, ONNX graph validation. Extends existing convert_to_coreml.py
   skeleton with working implementations.

4. CRDT scene sync (scene/crdt.py + tests/test_crdt.py)
   LWWRegister[T] — last-write-wins register with timestamp + author
   tiebreak. ORSet[T] — observed-remove set; concurrent adds survive
   removes. SceneDelta — compact JSON diff (added/removed/changed objects
   + scalar fields). diff(a,b), merge(base,delta), merge_snapshots(a,b).
   19 unit tests; all pass.

Also: delete 5 leftover .before-* / .broken.* backup files from repo.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rust backend: daemon_start/stop/alive commands, WebSocket daemon_command
  relay, open_viewer, traffic-light window controls, system tray
- React frontend: TitleBar, DaemonStatus, GeneratePanel (t2s/i2s),
  SceneViewer (viser iframe), log panel, useDaemon polling hook
- macOS vibrancy window, custom CSS design system (--syna-accent purple)
- scripts/build_sidecar.py: PyInstaller one-file daemon for Tauri externalBin
- auroch_syna/daemon/__main__.py: enables python -m auroch_syna.daemon

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Ultimate upgrade for optimal rendering quality
- Full Capitalist design system (white/gold/blue palette, typography, spacing)
- 5 product divisions: Terminal, Reserve, Dispatch, Fleet, Core
- CapitalistHeader, CapitalistNav, and all view components
- 200+ lines of CSS tokens and component styles
- Build fixes: add missing deps (tokio-tungstenite, futures-util, uuid),
  ShellExt import, placeholder icons, fix tauri.conf vibrancy
Core bug fixes
--------------
- general_utils: remove device= param from map_image_to_pano; all tensors
  now stay on rgb_src.device, eliminating cross-device crash on MPS/CPU
- general_utils: depth_match now returns a shallow copy instead of
  mutating bg_pred in-place; calling twice gives the same result
- general_utils: convert_rgbd2mesh_panorama device param defaults to
  rgb.device instead of hardcoded 'cuda'
- splat_utils: add named SH_C0 constant (1 / (2*sqrt(pi)))
- splat_utils: fix three torch.cross calls to pass dim=1 explicitly
  (removes DeprecationWarning in PyTorch >=2.0)
- splat_utils: unify pixel-centre theta sampling in convert_rgbd_to_gs
  to match pano_unit_rays (eliminates half-pixel systematic bias)
- splat_utils: fix mask_splat reshape assumption; add pixel_valid_mask
  kwarg + valid_mask_out on convert_rgbd_to_gs so callers can safely
  intersect masks when dis_threshold > 0
- flux_pano_fill_pipeline: fix NameError on latent_image_ids in the
  latents-is-not-None early-return path
- lor- ltils: derive n_single/n_transformer block counts from checkpoint
  instead of hardcoding 29; match zero-tensor dtype to existing tensors
- pano_gen: replace device='mps' defaults with resolve_device()
- worldgen: remove device= argument from map_image_to_pano call

CI improvements
---------------
- smoke-import.yml: add lint (ruff), unit-tests (pytest), and improved
  smoke-import jobs; package is now actually installed in CI

Tests
-----
- tests/test_worldgen_utils.py: 13 new unit tests for pure-Python
  worldgen utilities (no GPU required)

Housekeeping
------------
- .gitmodules: remove dead pytorch3d submodule entry; change ml-sharp
  SSH URL to HTTPS
- pyproject.toml: remove conflicting [tool.setuptools.dynamic] version
  attr (static version already set in [project])
- REPORT.md: updated to reflect current resolution status of all items
- ARCHITECTURE.md: rewritten with accurate current-state diagram and
  updated roadmap
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant