Gossip mesh topology + source-based routing

[callebtc]

gossip + source-based routing

Summary

1. The Problem with Simple Broadcasting

In a basic mesh network, when a device wants to send a message to everyone, it broadcasts the message to all its direct neighbors. Each of those neighbors then broadcasts
it to their neighbors, and so on. This is known as "flooding." While simple, it has major drawbacks:

• Network Congestion: Every message is sent across the entire network, even if it’s only intended for one person. This creates a lot of unnecessary traffic.
• The "Broadcast Storm": In a dense network, devices can receive the same message from multiple neighbors, leading to an exponential increase in redundant transmissions.
• Inefficiency: It’s like shouting in a crowded room and hoping the right person hears you, while everyone else has to stop and listen.

2. The Solution: Source-Based Routing

Source-based routing solves this by having the sender of a message determine the entire path the message should take to its destination. This is like giving a package to a
mail carrier with a specific list of post offices it must visit to reach its final address.

To make this possible, two key components were introduced:

• Mesh Graph Discovery (via Gossip): Every device needs a map of the network.
• Route Planning and Forwarding: Devices need to be able to use that map to create and follow routes.

3. How It Works: A Step-by-Step Guide

Step 1: Building a Map of the Network (Gossip)

1. Announcing Neighbors: When a device (let's call it Node A) sends out its regular "hello, I'm here" message (an IdentityAnnouncement), it now includes extra information: a
   list of its direct neighbors. This is done using a custom GossipTLV (Type-Length-Value) format. So, Node A tells the world, "I'm here, and I'm directly connected to Node B
   and Node C."

2. Collecting Information: Other devices on the network receive this announcement. When Node D hears from Node A, it updates its internal map of the network. It now knows
   that A, B, and C are all connected.

3. The MeshGraphService: This new service is the "keeper of the map." It runs on every device and continuously builds and updates a graph of the entire mesh network based
   on the gossip it hears. It stores:

   • Nodes: All the devices (peers) it knows about.
   • Edges: The direct connections between those devices.

Step 2: Planning the Route

1. Finding the Shortest Path: When Node A wants to send a message to Node Z, it consults its MeshGraphService. It asks the new RoutePlanner utility, "What's the shortest
   path from me to Node Z?"

2. Dijkstra's Algorithm: The RoutePlanner uses a classic and efficient algorithm (Dijkstra's) to find the shortest path across the graph. It might determine the best route
   is A -> C -> F -> Z.

3. Attaching the Route: Node A then creates the message packet and attaches this route to it. The BitchatPacket format was updated to include a new route field to carry this
   list of "hops."

Step 3: Forwarding the Message Along the Route

1. Targeted Sending: Node A doesn't broadcast the message to everyone. It looks at the first hop in the route (Node C) and sends the message directly to Node C.

2. Relaying: When Node C receives the message, it inspects the packet. It sees the route field and notices that it is part of the route. It then does two things:

   • It finds its own ID in the route list.
   • It looks at the next ID in the list (Node F) and forwards the message directly to Node F.

3. Final Delivery: This process repeats. Node F receives the message and forwards it directly to the final destination, Node Z.

4. Fallback to Broadcasting: If at any point a node in the chain can't find the next hop (perhaps that device went offline), it can fall back to the old method of
   broadcasting the message to all its neighbors to ensure it still has a chance of arriving.

5. UI and Debugging

A new Mesh Topology view was added to the app's debug screen. This provides a live, visual graph of the network as the device sees it, showing all the nodes and the
connections between them. This is an invaluable tool for developers to see if the gossip protocol is working correctly and to understand the current shape of the network.

Summary of Changes

• Gossip Protocol: Devices now share their neighbor lists in announcements.
• Network Graph: Each device maintains a complete map of the mesh.
• Route Planning: The shortest path to a destination is calculated before sending.
• Packet Modification: Packets now carry the pre-planned route.
• Intelligent Forwarding: Intermediate nodes relay messages to the specific next hop instead of broadcasting.
• Visualization: A new debug tool shows a real-time graph of the network topology.

This feature makes the Bitchat network significantly more efficient and scalable, especially as more devices join the mesh.

Announcement Gossip TLV (Direct Neighbors)

This document specifies an optional TLV extension to the BitChat ANNOUNCE message that allows peers to gossip which other peers they are currently connected to directly over Bluetooth. Implementations can use this to build a mesh topology view (nodes = peers, edges = direct connections).

Status: optional and backward-compatible.

Layering Overview

• Outer packet: BitChat binary packet with type = 0x01 (ANNOUNCE). Header is unchanged.
• Payload: A sequence of TLVs. Unknown TLVs MUST be ignored for forward compatibility.
• Signature: The packet MAY be signed using the Ed25519 public key carried in TLV 0x03. The gossip TLV (if present) is part of the payload and therefore covered by the signature.

TLV Format

Each TLV uses a compact layout:

• type: 1 byte
• length: 1 byte (0..255)
• value: length bytes

Existing TLVs (unchanged):

• 0x01 NICKNAME: UTF‑8 string (≤ 255 bytes)
• 0x02 NOISE_PUBLIC_KEY: Noise static public key bytes (typically 32 bytes for X25519)
• 0x03 SIGNING_PUBLIC_KEY: Ed25519 public key bytes (typically 32 bytes)

New TLV (optional):

• 0x04 DIRECT_NEIGHBORS: Concatenation of up to 10 peer IDs, each encoded as exactly 8 bytes. There is no inner count; the number of neighbors is length / 8. If length is not a multiple of 8, trailing partial bytes MUST be ignored.

Peer ID Binary Encoding (8 bytes)

Peer IDs are represented as 8 raw bytes (16 hex chars) in “network order” (left‑to‑right):

• Take the peer ID hex string, lowercase/truncate to at most 16 hex chars.
• Convert each 2 hex chars to 1 byte from left to right.
• If fewer than 16 hex chars are available, pad the remaining bytes with 0x00 at the end to reach 8 bytes.

This matches the on‑wire 8‑byte senderID/recipientID encoding used in the BitChat packet header.

Sender Behavior

• Build the base announcement payload by emitting TLVs 0x01..0x03 as usual.
• Optionally append TLV 0x04 with up to 10 unique, directly connected peer IDs.
  • Remove duplicates before encoding.
  • Order is arbitrary and not semantically significant.
• Sign the ANNOUNCE packet so the gossip TLV is covered (recommended):
  • Signature algorithm: Ed25519 using the key in TLV 0x03.
  • Signature input: the binary packet encoding with the signature field omitted and the TTL normalized to 0. This allows TTL to change during relays without invalidating the signature.
• The payload may be compressed per the base protocol; the gossip TLV is encoded prior to optional compression.

Receiver Behavior

• Decompress payload if the packet’s compression flag is set, then parse TLVs in order.
• Parse TLVs 0x01..0x03 as usual; ignore any unknown TLVs.
• If a 0x04 TLV is present:
  • Interpret the value as N = length / 8 peer IDs (ignore trailing non‑aligned bytes).
  • Each 8‑byte chunk is decoded back to a 16‑hex‑char peer ID string (lowercase).
  • De‑duplicate neighbors.
• Topology maintenance guidance (optional, but recommended for consistent behavior):
  • Maintain, for each announcing peer A, the last announcement timestamp and the neighbor list from TLV 0x04.
  • When a newer announcement from A arrives (use the 8‑byte unsigned timestamp in the BitChat packet header), replace A’s previously recorded neighbor list with the new one. If older or equal, ignore the neighbor update.
  • Treat the neighbor list as a set of undirected edges {A, B} in your topology visualization; i.e., if A reports direct peers [B, C], add edges A–B and A–C.

Limits and Compatibility

• Max neighbors per TLV: 10. Senders MAY send fewer; receivers MUST accept any number N ≥ 0 implied by length / 8 up to the received length.
• Omission: If the TLV 0x04 is omitted, the announce remains valid. Peers can still chat and interoperate normally; the topology graph will just not include edges reported by that peer (other peers that include A in their neighbor lists can still introduce edges to A).
• Unknown TLVs MUST be ignored. This makes the extension safe for older implementations.

Minimal Example (conceptual)

ANNOUNCE payload TLVs (concatenated):

• 01 [len=N] [UTF‑8 nickname]
• 02 [len=32] [32 bytes X25519 pubkey]
• 03 [len=32] [32 bytes Ed25519 pubkey]
• 04 [len=8*M] [peerID1(8) || peerID2(8) || ... || peerIDM(8)] (optional)

Where each peerIDk(8) is the 8‑byte binary form of the peer ID as specified above.

That’s the entire change; the outer packet header, message type, and relay/TTL behavior are unchanged.

Source-Based Routing for BitChat Packets

This document specifies an optional source-based routing extension to the BitChat packet format. A sender may attach a hop-by-hop route (list of peer IDs) to instruct relays on the intended path. Relays that support this feature will try to forward to the next hop directly; otherwise, they fall back to regular broadcast relaying.

Status: optional and backward-compatible.

Layering Overview

• Outer packet: BitChat binary packet with unchanged fixed header (version/type/ttl/timestamp/flags/payloadLength).
• Flags: adds a new bit HAS_ROUTE (0x08).
• Variable sections (when present, in order):
  1. SenderID (8 bytes)
  2. RecipientID (8 bytes) if HAS_RECIPIENT
  3. Route (if HAS_ROUTE): count (1 byte) + count * 8 bytes hop IDs
  4. Payload (with optional compression preamble)
  5. Signature (64 bytes) if HAS_SIGNATURE

Unknown flags are ignored by older implementations (they will simply not see a route and continue broadcasting as before).

Route Field Encoding

• Presence: Signaled by the HAS_ROUTE (0x08) bit in flags.
• Layout (immediately after optional RecipientID):
  • count: 1 byte (0..255)
  • hops: concatenation of count peer IDs, each encoded as exactly 8 bytes
• Peer ID encoding (8 bytes): same as used elsewhere in BitChat (16 hex chars → 8 bytes; left-to-right conversion; pad with 0x00 if shorter). This matches the on‑wire senderID/recipientID encoding.
• Size impact: 1 + 8*N bytes, where N = count.
• Empty route: HAS_ROUTE with count = 0 is treated as no route (relays ignore it).

Sender Behavior

• Applicability: Intended for addressed packets (i.e., where recipientID is set and is not the broadcast ID). For broadcast packets, omit the route.
• Path computation: Use Dijkstra’s shortest path (unit weights) on your internal mesh topology to find a route from the sender (your peerID) to the recipient (the destination peerID). The BitchatPacket already contains dedicated senderID and recipientID fields. The Route field's hops list SHOULD contain the sequence of intermediate peer IDs that the packet should traverse. It SHOULD NOT duplicate the senderID or recipientID if they are already present in the BitchatPacket's dedicated fields. Instead, the hops list represents the explicit path between the sender and recipient, starting from the first relay and ending with the last relay before the recipient.
• Encoding: Set HAS_ROUTE, write count = path.length, then the 8‑byte hop IDs in order. Keep count <= 255.
• Signing: The route is covered by the Ed25519 signature (recommended):
  • Signature input is the canonical encoding with signature omitted and ttl = 0 (TTL excluded to allow relay decrement) — same rule as base protocol.

Relay Behavior

When receiving a packet that is not addressed to you:

1. If HAS_ROUTE is not set, or the route is empty, relay using your normal broadcast logic (subject to TTL/probability policies).
2. If HAS_ROUTE is set:
   • Route Sanity Check: Before processing, the relay MUST validate the route. If the route contains duplicate hops (i.e., the same peer ID appears more than once), the packet MUST be dropped to prevent loops.
   • If your peer ID appears at index i in the hop list:
     • If there is a next hop at i+1, attempt a targeted unicast to that next hop if you have a direct connection to it.
       • If successful, do NOT broadcast this packet further.
       • If not directly connected (or the send fails), fall back to broadcast relaying.
     • If you are the last hop (no i+1), the packet has reached the end of its explicit route. The relay should then attempt to deliver it to the final recipientID if directly connected, but SHOULD NOT relay it further as a broadcast.

TTL handling remains unchanged: relays decrement TTL by 1 before forwarding (whether targeted or broadcast). If TTL reaches 0, do not relay.

Receiver Behavior (Destination)

• This extension does not change how addressed packets are handled by the final recipient. If the packet is addressed to you (recipientID == myPeerID), process it normally (e.g., decrypt Noise payload, verify signatures, etc.).
• Signature verification MUST include the route field when present; route tampering will invalidate the signature.

Compatibility

• Omission: If HAS_ROUTE is omitted, legacy behavior applies. Relays that don’t implement this feature will ignore the route entirely, because they won’t set or check HAS_ROUTE.
• Partial support: If any relay on the path cannot directly reach the next hop, it will fall back to broadcast relaying; delivery is still probabilistic like the base protocol.

Minimal Example (conceptual)

• Header (fixed 13 bytes): unchanged.
• Variable sections (ordered):
  • SenderID(8)
  • RecipientID(8) (if present)
  • HAS_ROUTE set → count=1, hops = [H1] where H1 is 8 bytes
  • Payload (optionally compressed)
  • Signature (64)

In this example, SENDER_ID is the sender, RECIPIENT_ID is the final recipient, and H1 is the single intermediate relay. The hops list explicitly defines the path between the sender and recipient. The receiver verifies the signature over the packet encoding (with ttl = 0 and signature omitted), which includes the hops when HAS_ROUTE is set.

Operational Notes

• Routing optimality depends on the freshness and completeness of the topology your implementation has learned (e.g., via gossip of direct neighbors). Recompute routes as needed.
• Route length should be kept small to reduce overhead and the probability of missing a direct link at some hop.
• Implementations may introduce policy controls (e.g., disable source routing, cap max route length).