A communication system derived from the mathematics of communication.
Replace
YOUR-USERNAMEin this file and inpyproject.tomlwith your GitHub username after you create the repository.
Every mainstream chat app stores a conversation as one straight timeline. Its real shape is a partial order: some messages cause others, many run in parallel. Almost every daily annoyance — replies landing before their cause, "delivered/read" that tells you nothing real, notifications ranked by recency instead of importance, silent merge conflicts — is a symptom of flattening that structure away.
WEFT keeps the structure. This repository is a working, tested implementation of the
buildable core: real cryptographic identities, an append-only Merkle-DAG on disk,
nodes that gossip over TCP and converge, honest epistemic-depth tracking, a typed
message algebra you can query, and information-theoretic attention triage — plus a
real chat application (weft-server) built on top of it.
pip install -e .
python demo.py # base layers, end to end, over real TCP
python demo_frontier.py # ratchet + sublinear sync + live HTTP API
pytest # 27 tests incl. a 3-process network convergence test
WEFT ships as an actual messenger you open in a browser — accounts, direct and group chats, live delivery, reply threading, and read receipts that are real (WEFT epistemic depth), not decorative checkmarks.
weft-server --port 8080 # or: python -m weft.server --port 8080Open http://127.0.0.1:8080 in two browser windows (use a private window for the
second), pick a display name in each, and chat. Messages arrive instantly over
Server-Sent Events; when the other person reads your message, "understood (E¹)"
appears — meaning it is provably common knowledge to depth one, computed from the
DAG. Mark a message as a Question / Decision / Task to use the typed algebra.
Trust model: this is a server-authoritative build — the Telegram / Discord / Teams model, where the hub holds the conversations and signs each message with the sending account's real Ed25519 key. See "How close is this to Signal/WhatsApp?" below for the honest gap and the path to client-side-key E2E.
What runs today is the messaging core those apps share: identities, direct and group conversations, real-time delivery, persistence, typed messages, reply threading, and genuine read-receipt semantics — on a causally-ordered, convergent, content-addressed substrate. What still stands between this and shipping one of those as a product:
| Area | This build | A shipped product |
|---|---|---|
| Clients | one web app (served by the hub) | native iOS/Android + desktop + web |
| Voice/video | — | WebRTC calling, SFU infrastructure |
| Onboarding | pick a display name | phone/email verification, contact discovery |
| Notifications | live only while a tab is open | APNs/FCM push when the app is closed |
| Media | text messages | images, files, voice notes via a CDN |
| Encryption | server signs with real Ed25519 keys (cloud-trust) | client-held keys + ratchet for full E2E |
| Scale/ops | single process, in-memory + optional on-disk log | sharded, replicated, monitored fleets |
The encryption row is the interesting one: the forward-secret Double Ratchet in
ratchet.py is already built and tested; wiring it in with client-held keys
(browser WebCrypto) and sender-keys for groups is what turns the cloud-trust model
into Signal-style E2E. That's the honest next milestone, not a hidden gap.
# terminal 1
weft --dir alice --port 9001
# terminal 2
weft --dir bob --port 9002 --peer 127.0.0.1:9001 --http 8091
# then open web/weft-live.html and point it at http://127.0.0.1:8091Then, in bob's terminal:
weft> decide Ship the release on Friday?
weft> ack <id-prefix> # acknowledge alice's messages
weft> status # open questions, decisions with depth, forks
weft> depth <decision-id> # E^k and who is on the frontier
weft> inbox 45 # attention triage at a 45-bit budget
Both nodes gossip in the background; each is a full offline-first replica with its own signing key and on-disk log.
| § | Layer | Mathematics | Module |
|---|---|---|---|
| 1 | Causal order (poset) | Lamport happens-before, vector clocks | vclock.py, replica.py |
| 2 | Content-addressed substrate | Merkle-DAG, SHA-256, Ed25519 signatures | crypto.py, message.py, store.py |
| 3 | Convergence + gossip | Grow-only-set CRDT (semilattice join), anti-entropy; range-based set reconciliation for sublinear sync | store.py, replica.py, net.py, reconcile.py |
| 4 | Shared understanding | Iterated mutual knowledge Eᵏ; Coordinated Attack theorem |
epistemic.py |
| 5 | Fork detection | Obstruction to gluing local views (sheaf-inspired) | query.py |
| 6 | Attention | Shannon surprisal + rate–distortion under a bit budget | attention.py |
| 7 | Typed algebra + queries | Typing rules over the causal past | message.py, query.py |
| 8 | Content encryption | X25519 + ChaCha20-Poly1305 AEAD; Double Ratchet for forward secrecy | crypto.py, ratchet.py |
| — | Live control surface | JSON/HTTP API over a running node + browser client | httpapi.py, web/weft-live.html |
Three additions, each with real crypto / real sockets and its own tests
(tests/test_frontier.py, demo_frontier.py):
- Double Ratchet (
ratchet.py). A faithful Signal-style construction: an X25519 DH ratchet that turns over on every reply (post-compromise healing) and two HKDF/HMAC symmetric chains that turn over on every message (forward secrecy). Messages are sealed with ChaCha20-Poly1305 with the ratchet header bound as AAD; out-of-order and dropped messages are handled by a bounded skipped-key cache. - Range-based set reconciliation (
reconcile.py). Anti-entropy whose bandwidth tracks the symmetric difference rather than the whole log. Two identical 5,000-id logs reconcile in ~74 bytes (≈1350× less than dumping every id); a two-id difference costs a few KB. Wired into the network layer (node.rsync_with,sync_all(mode="range")) and served alongside the naive protocol by a single dispatcher. - Live HTTP API + browser client (
httpapi.py,web/weft-live.html). A stdlib JSON server over a running node (weft … --http PORT) with the weaving-draft visual polling it live — the demo now drives real, networked nodes instead of a simulation.
- Implemented, tested, and networked: §1, §2, §3 (naive and range reconciliation), §4, §6, §7, a concrete slice of §5 (fork detection and localization), and the Double Ratchet for §8.
- Real but scoped: the Double Ratchet is a genuine forward-secret, self-healing channel, but it is pairwise. Encrypting the group DAG needs sender-keys or MLS on top of it, and asynchronous session setup wants X3DH with signed prekeys — neither is built here. Metadata-minimization goals (mixing, padding, cover traffic) remain future work. The ratchet is a standalone secure channel, not yet fused into the group message layer.
- Research frontier: the full sheaf-cohomological treatment of §5 (computing
H¹obstructions at scale) is a north star, not a finished component. WEFT detects and localizes forks; it does not yet compute cohomology. - Web client caveat:
weft-live.htmlis API-tested (every endpoint it calls is exercised against live nodes in the test suite) but was not run in a browser in this build — treat the rendering as reviewed code, not a shipped screenshot.
Nothing here is oversold: the property tests assert exactly the behavior the math predicts; the network test spawns three real OS processes and checks they converge and agree on epistemic depth; and the frontier tests exercise the ratchet, reconciliation correctness/bandwidth, range sync over sockets, and the HTTP API against live nodes.
weft/
crypto.py Ed25519 identities, signing, hashing, AEAD
vclock.py vector-clock order
message.py signed content-addressed message + typing rules
store.py append-only, content-addressed grow-only set (CRDT)
replica.py CRDT merge, causal delivery, causal queries, timeline projection
epistemic.py Eᵏ depth + frontier
attention.py rate–distortion notification triage
query.py open questions/decisions, retractions, fork detection
net.py anti-entropy gossip over TCP (naive + range protocols)
reconcile.py range-based set reconciliation (sublinear anti-entropy)
ratchet.py Double Ratchet — forward-secret, self-healing pairwise channel
httpapi.py JSON/HTTP API over a live node (stdlib, CORS)
server.py multi-user chat application server (accounts, convs, SSE)
node.py identity + replica + server + gossip, high-level API
cli.py `weft` interactive node (add --http PORT for the API)
web/
weft-messenger.html the chat application (served at / by weft-server)
weft-live.html live browser client that drives a node via the HTTP API
tests/
test_core.py every layer, in-process
test_network.py three OS processes converging over TCP
test_frontier.py ratchet, reconciliation, range sync over sockets, HTTP API
test_app.py the chat application: accounts, DMs, groups, SSE, receipts
demo.py end-to-end showcase (all base layers over TCP)
demo_frontier.py the three v1.1 frontier features
Companion documents (design note, formal protocol spec, and an interactive visual demo) ship alongside this package.