Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
94 lines (79 loc) · 8.17 KB

ARCHITECTURE.md

File metadata and controls

94 lines (79 loc) · 8.17 KB
Raw

Architecture

Cerul is one Rust crate with a reusable library and a CLI. This page maps source files and documentation to their responsibilities. DESIGN.md defines the behavior, storage contracts, and invariants implementations must preserve.

Module map

Area Entry points Responsibility
CLI main.rs, render.rs, guide.rs, credentials.rs Parse arguments, dispatch commands, render results, complete an under-specified command at a terminal, and manage interactive credential setup.
Library interface lib.rs, config.rs, events.rs Expose reusable operations, explicit configuration, and structured events.
Media and time episode.rs, media/, ocr.rs Discover media properties, map episode time, prepare model inputs, and run embedded OCR.
Providers providers/ Call configured endpoints, validate capabilities, and resolve scoped credentials.
Indexing index/pipeline.rs, index/ Discover inputs, resume processing stations, publish sidecars, and build search indexes.
Annotations annotate/pipeline.rs, annotate/hands.rs, annotations/ Generate semantic modules and optional embodied CPU hand keypoints, validate records, and publish annotations.
Retrieval search/ Apply temporal filters, search vectors or text, and export clips.
Dataset writeback lerobot.rs, lerobot/ Read datasets and stage, validate, publish, or recover subtask writeback.
Persistence and maintenance storage.rs, status.rs, clean.rs Atomic writes and locks, status reporting, and explicit data or cache removal.

Processing flow

  1. The CLI resolves configuration and credentials, then calls library operations.
  2. Discovery establishes content identity, streams, and episode-relative time.
  3. Processing stations reuse valid checkpoints or produce OCR, transcripts, semantic annotations, and embedding vectors.
  4. Validated outputs are published atomically to sidecars. These are authoritative; the Lance indexes are projections that can be rebuilt without model calls.
  5. Search applies scope and temporal filters before ranking, then returns structured matches and optional clip artifacts.

All stored intervals are half-open integer-microsecond episode time. Source and model-relative times are converted at their boundaries. A workspace admits one writer at a time; interrupted operations preserve completed work for recovery.

Integration boundaries

Library code receives configuration and emits structured events without assuming a terminal or process lifecycle. It does not parse process arguments, print progress, or terminate the process. CLI subprocess consumers use the JSON/NDJSON and exit-code contract.

Library hosts can scope credentials with providers::with_credentials and supply a lazy resolver with providers::with_credential_resolver. The library does not read CLI credential files; environment values take precedence. Bundled media tools are resolved by media::command, with explicit overrides taking precedence. Processing CLI commands call media::dependencies::prepare before media work and model calls. This validates versions and codecs and, when necessary, installs a pinned, checksum-verified pair under the workspace's runtime/media. The pair is scoped to the operation and explicitly carried into OCR CPU workers. Library operations do not initiate dependency downloads on their own. Failed or interrupted repairs never publish an active installation; system tools and the CLI are untouched.

This crate implements local processing and endpoint clients. It does not implement product UI, hosted inference, or HTTP/MCP serving. See scope and current limits.

Documentation and supporting files

Location Purpose and editing rule
README.md, Simplified Chinese, Traditional Chinese Product introduction, installation, first use, and links to detailed guides; keep the three entry pages aligned.
docs/README.md Navigation by reader and task.
docs/ User installation, agent-assisted setup, video search, action annotation and LeRobot tutorials, configuration, and compatibility reference.
docs/development/ Source builds, validation, releases, and scope boundaries for contributors and maintainers.
ARCHITECTURE.md, DESIGN.md Source/module orientation and normative implementation contracts, respectively.
CONTRIBUTING.md, AGENTS.md Human contribution workflow and repository automation instructions.
SECURITY.md, CODE_OF_CONDUCT.md Vulnerability reporting and community conduct.
LICENSE, THIRD_PARTY_NOTICES.md, packaging/licenses/ Project license, bundled-component notices, and full third-party license texts. Preserve notices required by distributions.
models/README.md, models/LICENSE, models/characters.txt Model provenance, license, and a runtime OCR dictionary. The dictionary is data; whitespace changes can alter class decoding.
tests/fixtures/README.md, docs/assets/README.md Fixture provenance and brand-asset usage notes next to their files.
prompts/ Runtime Markdown embedded in the binary: eight semantic annotation prompts, plus skill.md, the body of the agent skill. Edit and validate them as processing behavior.
skills/ The generated agent skill exactly as cerul skill --install claude writes it. Regenerate with cerul skill --print > skills/cerul/SKILL.md after changing commands or the prompt.
schemas/ JSON contracts generated from Rust types by generate_schemas.rs; do not hand-edit them.
.github/ Issue forms, PR template, ownership, dependency automation, and workflow YAML. GitHub consumes these files at their designated locations.
Cargo.toml, Cargo.lock, dist-workspace.toml, packaging/dist.toml Rust and distribution manifests and dependency resolution. These are executable build inputs, not user documentation.
vercel.json Explicitly disables legacy Vercel Git deployment; it does not define a product website in this repository.
.gitignore, .gitattributes Exclude local artifacts and preserve byte-sensitive asset behavior.
examples/, scripts/, tests/ Runnable library examples, build/verification tools, and tests. Keep executable examples with their Cargo entry points.
docs/design/ and design/cli-interaction/ Concise decision records that link to current contracts and historical proposals. They are not separate command references or implementation backlogs. CLI design notes are checkout-only and are not required by source-package guides.
.workspace/ Ignored local plans, audits, and working artifacts; excluded from public source and packages.

The contribution policy defines publication boundaries. Cargo's explicit include list and CI package checks verify that source-package documentation, linked assets, and runtime data travel together. A source-package inventory differs from the Git checkout: repository hosting configuration and local artifacts are not package contents.

Dependency constraint

LanceDB 0.38.0 currently needs its remote Cargo feature to compile: its job error conversion references a feature-gated HTTP error type. Cerul opens only the configured local workspace directory. This compatibility feature does not enable a cloud connection or implement HTTP or MCP serving.