Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 80 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,64 @@
# recollect

`tiye/cumulo/recollect` is the standalone structural diff/patch package extracted from the larger Cumulo demo.
`tiye/recollect` is a structural JSON diff/patch library for MoonBit.

It is intentionally kept as a leaf package inside this repository so it can be split into its own module later with minimal work.
It computes a minimal sequence of `PatchOp` operations between two `Json` trees
Copy link

Copilot AI May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

README claims the library computes a "minimal" sequence of PatchOps. The implementation falls back to positional array diffing for non-keyed arrays, which can produce non-minimal sequences for insertions/deletions (it may emit multiple Sets instead of a single Insert/Delete). Consider adjusting the wording to avoid promising minimality unless it’s guaranteed.

Suggested change
It computes a minimal sequence of `PatchOp` operations between two `Json` trees
It computes a sequence of `PatchOp` operations between two `Json` trees

Copilot uses AI. Check for mistakes.
and applies them to reconstruct the target — suitable for efficient network
transport in realtime sync architectures.

It focuses on a small JSON-first API:
## Immutable data design

- `diff_json(old_json, new_json)` computes structural patch operations between two `Json` trees.
- `diff_value(old_value, new_value)` computes patches from typed MoonBit values via `ToJson`.
- `apply_patch(root, patch)` applies a single patch operation to a `Json` tree.
- `apply_patches(root, patches)` applies a patch sequence to a `Json` tree.
- `apply_to_value(value, patches)` applies patches to a typed MoonBit value via `ToJson` and `FromJson`.
`recollect` is designed for **immutable-data** workflows, as used in
[Cumulo](https://github.com/Cumulo/moonbit-cumulo) and
[Respo](https://github.com/Respo/respo.mbt).

The patch format is exposed through two public enums:
- `diff_json` / `diff_value` — **read-only**; inputs are never modified.
- `apply_patch` / `apply_patches` / `apply_to_value` — return a **new** `Json`
tree (or typed value); unchanged subtrees are shared structurally with the
original. The input is never mutated.

- `PathSegment`: `Field(String)` and `Index(Int)`
- `PatchOp`: `Set`, `Remove`, `Insert`, `Delete`, `Move`
This means it is safe to hold references to both the old and new state at the
same time, which is required for Respo's render-loop equality check and for
Cumulo's server-side twig/diff logic.

Current package boundary:
```moonbit nocheck
// old_remote and new_remote are independent values — old is never touched
let new_remote = @recollect.apply_to_value(old_remote, patches)
catch { _ => old_remote }
```

- source: `recollect/recollect.mbt`
- tests: `recollect/recollect_test.mbt`
- only dependency: `moonbitlang/core/json`
> ⚠️ Do **not** wrap your state struct in a `mut` field and mutate it in place.
> Doing so breaks structural equality (`==`) and makes the render loop skip
> updates. Always replace the whole value with the newly returned one.

## Import
## API

Add the package to a `moon.pkg` import list:
- `diff_json(old, new)` — compute `Array[PatchOp]` between two `Json` trees
- `diff_value(old, new)` — same, via `ToJson` serialization of typed values
- `apply_patch(root, patch)` — apply one `PatchOp`, return new `Json`
- `apply_patches(root, patches)` — apply a sequence, return new `Json`
- `apply_to_value(value, patches)` — apply patches to a typed value via `ToJson`/`FromJson`

```moonbit
Patch format (both enums are `pub(all)` with `ToJson`/`FromJson`):

- `PathSegment`: `Field(String)` | `Index(Int)`
- `PatchOp`: `Set` | `Remove` | `Insert` | `Delete` | `Move`

`Insert` / `Delete` / `Move` are only emitted for **keyed arrays** — arrays
where every element is a JSON object with a unique string `"id"` field.
Plain arrays fall back to positional `Set` patches.

## Import

```
import {
"tiye/cumulo/recollect" @recollect,
"tiye/recollect" @recollect,
}
```

## Json example

```moonbit
```moonbit nocheck
let before : Json = {
"todos": [{ "id": "t-1", "title": "A", "done": false }],
}
Expand All @@ -47,6 +70,43 @@ let after : Json = {
],
}

let patches = @recollect.diff_json(before, after)
let rebuilt = @recollect.apply_patches(before, patches)
// rebuilt == after
```

## Typed value example
Copy link

Copilot AI May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are now two "## Typed value example" sections in the README (one starts here, and another exists later in the file). Consider removing one to avoid duplicated/conflicting documentation.

Suggested change
## Typed value example
## Typed value example (structs)

Copilot uses AI. Check for mistakes.

```moonbit nocheck
struct Todo {
id : String
title : String
done : Bool
} derive(FromJson, ToJson)

struct AppState {
todos : Array[Todo]
} derive(FromJson, ToJson)

let before = AppState::{ todos: [{ id: "t-1", title: "A", done: false }] }
let after = AppState::{ todos: [
{ id: "t-1", title: "A", done: true },
{ id: "t-2", title: "B", done: false },
] }

let patches = @recollect.diff_value(before, after)
let rebuilt : AppState = @recollect.apply_to_value(before, patches)
catch { _ => before }
// rebuilt == after
```}

let after : Json = {
"todos": [
{ "id": "t-1", "title": "A", "done": true },
{ "id": "t-2", "title": "B", "done": false },
],
}

let patches = @recollect.diff_json(before, after)
let rebuilt = @recollect.apply_patches(before, patches)
Comment on lines +101 to 111
Copy link

Copilot AI May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Typed value example code block is closed with }, which breaks Markdown rendering, and the following let after : Json = { ... } snippet appears to be stray/duplicated content outside any fenced block. Fix the fence terminator to ``` and remove the unrelated duplicated JSON snippet so the README renders and copy/paste examples work.

Suggested change
```}
let after : Json = {
"todos": [
{ "id": "t-1", "title": "A", "done": true },
{ "id": "t-2", "title": "B", "done": false },
],
}
let patches = @recollect.diff_json(before, after)
let rebuilt = @recollect.apply_patches(before, patches)

Copilot uses AI. Check for mistakes.
```
Expand Down
2 changes: 1 addition & 1 deletion moon.mod.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "tiye/recollect",
"version": "0.1.0",
"version": "0.2.0",
"readme": "README.md",
"repository": "https://github.com/Cumulo/recollect.mbt/",
"license": "Apache-2.0",
Expand Down
89 changes: 89 additions & 0 deletions recollect.mbt
Original file line number Diff line number Diff line change
@@ -1,10 +1,33 @@
///|
/// A single segment in a path through a `Json` tree.
///
/// Paths are sequences of segments used by `PatchOp` to address nested
/// locations. `Field("key")` traverses into a JSON object; `Index(n)`
/// traverses into a JSON array.
pub(all) enum PathSegment {
Field(String)
Index(Int)
} derive(Eq, Debug, FromJson, ToJson)

///|
/// A single structural edit operation on a `Json` tree.
///
/// Patch operations are always **non-destructive** with respect to the input:
/// every `apply_*` function returns a new tree rather than mutating the
/// original. This makes the patch format safe to use with immutable-data
/// architectures (e.g. Cumulo server state, Respo stores).
///
/// | Variant | Meaning |
/// |---------|---------|
/// | `Set` | Overwrite a value at `path` (creates intermediate nodes if absent) |
Copy link

Copilot AI May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc table says Set "creates intermediate nodes if absent", but apply_set aborts when it needs to traverse through a missing intermediate node (it substitutes null for a missing child, which then fails the next Object/Array guard). Either adjust the docs to describe the actual behavior (only creates the leaf when the parent exists) or update apply_set to materialize intermediate objects/arrays based on the remaining path segments.

Suggested change
/// | `Set` | Overwrite a value at `path` (creates intermediate nodes if absent) |
/// | `Set` | Overwrite a value at `path` (creates the leaf only when the parent path exists) |

Copilot uses AI. Check for mistakes.
/// | `Remove` | Delete the field or array element addressed by `path` |
/// | `Insert` | Splice a new element into the array at `path` before `index` |
/// | `Delete` | Remove the element at `index` from the array at `path` |
/// | `Move` | Re-order an element within the array at `path` |
///
/// `Insert`/`Delete`/`Move` are only emitted for **keyed arrays** — arrays
/// whose every element is a JSON object with a unique string `"id"` field.
/// Plain arrays fall back to positional `Set` patches.
pub(all) enum PatchOp {
Set(path~ : Array[PathSegment], value~ : Json)
Remove(path~ : Array[PathSegment])
Expand Down Expand Up @@ -262,13 +285,42 @@ fn collect_json_diff(
}

///|
/// Compute a minimal sequence of `PatchOp`s that transforms `old_json` into
/// `new_json`.
///
/// The diff is **structural**: objects are compared field-by-field, arrays are
/// compared element-by-element (or by `"id"` key when all elements carry
/// unique string ids). Unchanged subtrees produce no patches.
Comment on lines +288 to +293
Copy link

Copilot AI May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

diff_json is documented as producing a "minimal" patch sequence, but the current positional array diff emits per-index Set operations for shifts (e.g. inserting at the front of a plain array) and is not minimal in terms of number of ops. Consider relaxing the wording (e.g. "structural" / "deterministic") or defining what "minimal" means here and ensuring the implementation meets it.

Suggested change
/// Compute a minimal sequence of `PatchOp`s that transforms `old_json` into
/// `new_json`.
///
/// The diff is **structural**: objects are compared field-by-field, arrays are
/// compared element-by-element (or by `"id"` key when all elements carry
/// unique string ids). Unchanged subtrees produce no patches.
/// Compute a deterministic structural sequence of `PatchOp`s that transforms
/// `old_json` into `new_json`.
///
/// The diff is **structural**: objects are compared field-by-field, arrays are
/// compared element-by-element (or by `"id"` key when all elements carry
/// unique string ids). Unchanged subtrees produce no patches. For plain
/// arrays, the patch sequence is positional and not guaranteed to be minimal
/// in number of operations.

Copilot uses AI. Check for mistakes.
///
/// The result can be serialized (via `ToJson`/`FromJson` on `PatchOp`) and
/// sent over a network, then applied on the other side with `apply_patches`.
///
/// ```moonbit
/// let patches = diff_json(
/// { "count": 1 },
/// { "count": 2, "label": "hi" },
/// )
/// // patches: [Set(path=[Field("count")], value=2),
/// // Set(path=[Field("label")], value="hi")]
/// ```
pub fn diff_json(old_json : Json, new_json : Json) -> Array[PatchOp] {
let patches = Array::new()
collect_json_diff([], old_json, new_json, patches)
patches
}

///|
/// Compute a minimal patch sequence by first serializing both values to `Json`
/// via `ToJson`, then delegating to `diff_json`.
///
/// This is the typical entry point when working with typed MoonBit structs.
/// The type `T` must derive (or implement) `ToJson`; it does **not** need
/// `FromJson` at the diff stage.
///
/// ```moonbit
/// struct Counter { value : Int } derive(ToJson)
/// let patches = diff_value({ value: 1 }, { value: 2 })
/// ```
pub fn[T : ToJson] diff_value(old_value : T, new_value : T) -> Array[PatchOp] {
diff_json(old_value.to_json(), new_value.to_json())
}
Expand Down Expand Up @@ -443,6 +495,15 @@ fn apply_move(
}

///|
/// Apply a single `PatchOp` to `root` and return the resulting `Json` tree.
///
/// The input `root` is **never mutated** — all intermediate nodes on the
/// affected path are copied, producing a new tree that shares unchanged
/// subtrees with the original. This structural sharing makes the function
/// safe to use in immutable-data architectures.
///
/// Panics (via `abort`) if the path in the patch is structurally inconsistent
/// with the tree (e.g. `Field` segment on an array node).
pub fn apply_patch(root : Json, patch : PatchOp) -> Json {
match patch {
Set(path~, value~) => apply_set(root, path, 0, value)
Expand All @@ -455,6 +516,17 @@ pub fn apply_patch(root : Json, patch : PatchOp) -> Json {
}

///|
/// Apply a sequence of `PatchOp`s to `root`, threading the result of each
/// operation as the input to the next.
///
/// Like `apply_patch`, the original `root` is never mutated.
///
/// An empty `patches` slice returns `root` unchanged.
///
/// ```moonbit
/// let new_json = apply_patches(old_json, diff_json(old_json, new_json))
/// assert_eq(new_json, new_json)
Comment on lines +527 to +528
Copy link

Copilot AI May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The apply_patches doc example is self-referential and will always pass (assert_eq(new_json, new_json)), and it also reuses new_json as both the expected value and the result name. Update the example to compare against an expected/target value so it actually demonstrates correctness.

Suggested change
/// let new_json = apply_patches(old_json, diff_json(old_json, new_json))
/// assert_eq(new_json, new_json)
/// let patched_json = apply_patches(old_json, diff_json(old_json, target_json))
/// assert_eq(patched_json, target_json)

Copilot uses AI. Check for mistakes.
/// ```
pub fn apply_patches(root : Json, patches : Array[PatchOp]) -> Json {
let mut current = root
for patch in patches {
Expand All @@ -464,6 +536,23 @@ pub fn apply_patches(root : Json, patches : Array[PatchOp]) -> Json {
}

///|
/// Apply `patches` to a typed MoonBit value by round-tripping through `Json`.
///
/// The value is serialized to `Json` via `ToJson`, the patches are applied
/// with `apply_patches`, and the result is deserialized back to `T` via
/// `FromJson`. Raises `@json.JsonDecodeError` if the patched JSON cannot be
/// decoded into `T`.
///
/// This is the primary function used by Cumulo-style clients to advance their
/// local state when receiving a `Delta` event from the server:
///
/// ```moonbit nocheck
/// let new_remote = @recollect.apply_to_value(old_remote, patches)
/// catch { err => ... }
/// ```
///
/// Because both serialization and deserialization produce new values, the
/// original `value` is never mutated.
pub fn[T : ToJson + FromJson] apply_to_value(
value : T,
patches : Array[PatchOp],
Expand Down
Loading