backup commit

2026-03-17 00:03:08 +00:00
parent f23be63bba
commit 253c305cc8
43 changed files with 9514 additions and 308 deletions
--- a/.claude/plans/luminous-snacking-snowflake.md
+++ b/.claude/plans/luminous-snacking-snowflake.md
@@ -0,0 +1,247 @@
+# Phase 2: Mesh as Federation Transport
+
+## Context
+
+Phase 1 built three independent communication silos: Tor-HTTP for federation, Meshcore LoRa for offline mesh, Nostr relays for discovery. They share no common abstraction — federation hardcodes Tor SOCKS5, mesh has its own serial listener, peer messaging uses a separate HTTP POST. Adding a new transport (LAN/mDNS) or changing routing logic means touching every caller.
+
+Phase 2 unifies these under a `NodeTransport` trait with automatic transport selection per peer (Mesh > LAN > Tor), chunked LoRa messaging with Reed-Solomon FEC for payloads >160 bytes, CBOR delta compression for state sync, mDNS LAN discovery, and a "mesh only" off-grid mode.
+
+The v2.0 roadmap (`docs/roadmap-v2.0.md`) also calls for a visual topology map and CRDT-based state sync — those are deferred to Phase 5 (Mesh Intelligence), but the transport abstraction here enables them.
+
+## Dependencies (Verified)
+
+```toml
+ciborium = "0.2.2"            # CBOR serde (134M downloads, active)
+reed-solomon-erasure = "6.0"  # FEC (4M downloads, MIT)
+mdns-sd = "0.18.2"            # LAN peer discovery (1.86M, active)
+```
+
+## Architecture
+
+```
+core/archipelago/src/transport/
+├── mod.rs              — NodeTransport trait, TransportKind, TransportRouter, PeerRegistry
+├── tor.rs              — TorTransport (wraps node_message.rs SOCKS5 logic)
+├── mesh_transport.rs   — MeshTransport (bridges MeshService cmd channel)
+├── lan.rs              — LanTransport (mdns-sd + direct HTTP)
+├── chunking.rs         — Split/reassemble with Reed-Solomon FEC
+└── delta.rs            — CBOR delta encoding for NodeStateSnapshot
+```
+
+### Unified Peer Identity
+
+DID is the canonical identifier. Transport-specific addresses are attributes:
+
+```rust
+pub struct PeerRecord {
+    pub did: String,
+    pub pubkey_hex: String,
+    pub name: Option<String>,
+    pub trust_level: TrustLevel,
+    // Transport capabilities
+    pub mesh_contact_id: Option<u32>,
+    pub lan_address: Option<SocketAddr>,
+    pub onion_address: Option<String>,
+    // Freshness
+    pub last_mesh: Option<String>,
+    pub last_lan: Option<String>,
+    pub last_tor: Option<String>,
+}
+```
+
+### Transport Routing
+
+```
+TransportRouter::send_to_peer(did, payload)
+  1. Look up PeerRecord by DID
+  2. Filter to transports where peer has address AND transport is available AND not stale
+  3. Sort by priority: Mesh(1) > LAN(2) > Tor(3)
+  4. If mesh_only_mode: only allow Mesh
+  5. Try each in order, fall back on failure
+```
+
+### Chunking Protocol (for LoRa >160B)
+
+Per-chunk structure (8 byte header + 124 byte payload = 132B, fits in 160B after encryption):
+```
+[0x01] [msg_id: u32 LE] [chunk_idx: u8] [total: u8] [is_parity: u8] [payload: 124B]
+```
+
+Reed-Solomon: 25% parity shards. 4 data chunks = 1 parity chunk. Any N-of-(N+parity) reconstructs.
+Max practical: ~10 chunks (~1.2KB) before LoRa airtime becomes unreasonable.
+
+### CBOR Delta Sync
+
+Full JSON state: ~500-2000 bytes. CBOR delta when only CPU/mem changed: ~30-50 bytes. Fits single LoRa chunk.
+
+```rust
+pub struct StateDelta {
+    pub ts: String,
+    pub v: u8,
+    pub apps: Option<Vec<AppStatus>>,      // Only changed apps
+    pub apps_rm: Option<Vec<String>>,       // Removed app IDs
+    pub cpu: Option<f64>,
+    pub mem_u: Option<u64>,
+    // ... only changed fields, short names for wire size
+}
+```
+
+## Implementation Steps
+
+### Step 1: Add deps to Cargo.toml
+**File**: `core/archipelago/Cargo.toml`
+Add ciborium, reed-solomon-erasure, mdns-sd.
+
+### Step 2: Create `transport/mod.rs` — Core trait + PeerRegistry + TransportRouter
+**New file**: `core/archipelago/src/transport/mod.rs`
+- `NodeTransport` trait: `kind()`, `is_available()`, `send(address, payload)`
+- `TransportKind` enum: `Mesh = 1, Lan = 2, Tor = 3`
+- `TransportMessage`: `{from_did, payload: Vec<u8>, message_type}`
+- `PeerRegistry`: unified peer store (DID -> PeerRecord), JSON persistence to `transport-peers.json`
+- `TransportRouter`: holds Vec<Box<dyn NodeTransport>>, routes by priority with fallback
+- Register `mod transport;` in `core/archipelago/src/lib.rs` or main.rs
+
+### Step 3: Create `transport/chunking.rs` — Reed-Solomon FEC
+**New file**: `core/archipelago/src/transport/chunking.rs`
+- `encode_chunked(data: &[u8]) -> Vec<Vec<u8>>` — split + FEC parity shards
+- `ChunkReassembler` — state machine tracking pending messages, attempts reconstruction when enough chunks arrive
+- `decode_chunked(chunks: &[Option<Vec<u8>>]) -> Result<Vec<u8>>` — reconstruct from data+parity
+- Unit tests: roundtrip, missing chunks, corrupted chunks
+
+### Step 4: Create `transport/delta.rs` — CBOR delta sync
+**New file**: `core/archipelago/src/transport/delta.rs`
+- `compute_delta(prev, curr: &NodeStateSnapshot) -> StateDelta`
+- `apply_delta(base: &NodeStateSnapshot, delta: &StateDelta) -> NodeStateSnapshot`
+- `encode_cbor(delta) -> Vec<u8>` via ciborium
+- `decode_cbor(bytes) -> Result<StateDelta>`
+- Unit tests: delta correctness, CBOR roundtrip, size comparison vs JSON
+
+### Step 5: Create `transport/tor.rs` — TorTransport
+**New file**: `core/archipelago/src/transport/tor.rs`
+- Wraps existing `node_message::send_to_peer()` pattern (reqwest + SOCKS5)
+- Reuses: `node_message.rs:66` send logic, `node_message.rs:115` health check
+- `is_available()` checks SOCKS5 proxy reachable at 127.0.0.1:9050
+- Receives handled by existing HTTP handler in `api/handler.rs`
+
+### Step 6: Create `transport/mesh_transport.rs` — MeshTransport
+**New file**: `core/archipelago/src/transport/mesh_transport.rs`
+- Holds reference to `Arc<RwLock<Option<MeshService>>>` (same as RpcHandler)
+- `send()` routes through `MeshService::send_message()` for small payloads
+- For payloads >124B: chunk with `chunking::encode_chunked()`, send each chunk via `MeshCommand::SendText`
+- 200ms inter-chunk delay for LoRa airtime fairness
+- `is_available()` checks mesh device connected via `MeshService::status()`
+
+### Step 7: Create `transport/lan.rs` — LanTransport
+**New file**: `core/archipelago/src/transport/lan.rs`
+- Uses `mdns-sd::ServiceDaemon` for discovery
+- Advertises: `_archipelago._tcp.local.` with TXT records: `did=..., pubkey=..., version=...`
+- Browses for peers, updates PeerRegistry when found
+- `send()`: direct HTTP POST to `http://{ip}:5678/archipelago/node-message` (same endpoint as Tor, no proxy)
+- Non-blocking init — if Avahi not available, fail gracefully
+
+### Step 8: Add `mesh_only_mode` to MeshConfig
+**File**: `core/archipelago/src/mesh/mod.rs`
+- Add `pub mesh_only_mode: Option<bool>` to MeshConfig (with `#[serde(default)]`)
+- TransportRouter reads this flag to restrict routing
+
+### Step 9: Wire TransportRouter into server.rs
+**File**: `core/archipelago/src/server.rs` (~line 136, after mesh service init)
+- Create PeerRegistry, load from disk
+- Create TorTransport, MeshTransport, LanTransport
+- Start LAN advertisement + discovery
+- Create TransportRouter with all three + mesh_only flag
+- Spawn background task bridging `MeshEvent::IdentityReceived` -> PeerRegistry
+- Store router on RpcHandler
+
+### Step 10: Add transport RPC endpoints
+**New file**: `core/archipelago/src/api/rpc/transport.rs`
+**Modify**: `core/archipelago/src/api/rpc/mod.rs` (add dispatch routes)
+
+Endpoints:
+- `transport.status` — available transports, mesh_only flag, peer count
+- `transport.peers` — unified peer list with per-peer transport capabilities + preferred transport
+- `transport.send` — send message via best transport (by DID)
+- `transport.set-mode` — toggle mesh_only mode
+
+### Step 11: Add opt-in federation sync via transport
+**File**: `core/archipelago/src/federation.rs`
+- Add `sync_with_peer_via_transport()` alongside existing `sync_with_peer()`
+- Uses CBOR delta encoding when transport is mesh (saves bandwidth)
+- Falls back to full JSON over Tor for backward compat
+- Background sync task checks for TransportRouter, uses it if available
+
+### Step 12: Frontend — transport store + UI indicators
+**New file**: `neode-ui/src/stores/transport.ts` — Pinia store for transport status
+**Modify**: `neode-ui/src/views/Mesh.vue` — "OFF-GRID" indicator when mesh_only mode
+**Modify**: `neode-ui/mock-backend.js` — add transport.* mock RPC responses
+
+### Step 13: Mock backend transport data
+**File**: `neode-ui/mock-backend.js`
+Add `transport.status`, `transport.peers`, `transport.send`, `transport.set-mode` mock responses with realistic data showing mixed transport capabilities.
+
+## Files Summary
+
+### New (8)
+1. `core/archipelago/src/transport/mod.rs`
+2. `core/archipelago/src/transport/tor.rs`
+3. `core/archipelago/src/transport/mesh_transport.rs`
+4. `core/archipelago/src/transport/lan.rs`
+5. `core/archipelago/src/transport/chunking.rs`
+6. `core/archipelago/src/transport/delta.rs`
+7. `core/archipelago/src/api/rpc/transport.rs`
+8. `neode-ui/src/stores/transport.ts`
+
+### Modified (8)
+1. `core/archipelago/Cargo.toml` — add 3 deps
+2. `core/archipelago/src/lib.rs` or main — `mod transport;`
+3. `core/archipelago/src/server.rs` — init router, bridge events
+4. `core/archipelago/src/api/rpc/mod.rs` — dispatch + store router
+5. `core/archipelago/src/mesh/mod.rs` — `mesh_only_mode` config field
+6. `core/archipelago/src/federation.rs` — `sync_with_peer_via_transport()`
+7. `neode-ui/src/views/Mesh.vue` — off-grid indicator
+8. `neode-ui/mock-backend.js` — transport.* mock data
+
+### Reused Existing Code
+- `node_message.rs:66-112` — Tor SOCKS5 send logic -> wrapped by TorTransport
+- `node_message.rs:115-135` — Tor health check -> TorTransport.is_available()
+- `mesh/mod.rs:209-282` — MeshService::send_message() -> called by MeshTransport
+- `mesh/listener.rs:35-37` — MeshCommand channel -> used for chunked sends
+- `mesh/crypto.rs` — X25519 ECDH + ChaCha20-Poly1305 -> reused as-is
+- `federation.rs:36-49` — FederatedNode struct -> PeerRecord wraps this
+- `federation.rs:179-191` — update_node_state() -> adapted for CBOR deltas
+
+## Verification
+
+1. **Unit tests** (run on dev server):
+   ```bash
+   cargo test --all-features -- transport
+   cargo test --all-features -- chunking
+   cargo test --all-features -- delta
+   ```
+
+2. **Frontend type check**:
+   ```bash
+   cd neode-ui && npm run type-check
+   ```
+
+3. **Dev mode test**:
+   ```bash
+   cd neode-ui && npm start
+   # Navigate to /dashboard/mesh — verify OFF-GRID toggle
+   ```
+
+4. **Deploy + integration test**:
+   ```bash
+   ./scripts/deploy-to-target.sh --live
+   # On .228: curl transport.status RPC
+   # Verify mDNS advertises _archipelago._tcp.local.
+   # If mesh device connected: send chunked message >160B to .198
+   # Verify federation sync uses CBOR delta when available
+   ```
+
+## Implementation Order
+
+**Week 1**: Steps 1-4 (deps, trait, chunking, delta — pure library code with unit tests)
+**Week 2**: Steps 5-7 (three transport implementations)
+**Week 3**: Steps 8-11 (integration: server wiring, RPC, federation migration)
+**Week 4**: Steps 12-13 (frontend UI, mock data, deploy + test)
--- a/.claude/plans/polished-napping-squid.md
+++ b/.claude/plans/polished-napping-squid.md
@@ -0,0 +1,108 @@
+# Meshcore Mesh Networking — Phase 1 Implementation Plan
+
+## Context
+
+Adding mesh networking to Archipelago using Heltec V3 devices running Meshcore firmware (Companion USB). Two nodes (.228 and .198) will exchange encrypted identity and text messages over LoRa radio with no internet required. The existing `mesh.rs` wraps the Meshtastic CLI — this replaces it with a native Meshcore serial protocol driver.
+
+## Architecture
+
+Convert `mesh.rs` into `mesh/` module directory:
+
+```
+core/archipelago/src/mesh/
+├── mod.rs         — Public API, MeshService, config (migrated from mesh.rs)
+├── types.rs       — MeshPeer, MeshMessage, MeshStatus, DeviceType
+├── protocol.rs    — Meshcore binary frame protocol (encode/decode/commands)
+├── serial.rs      — MeshcoreDevice: async serial driver (serial2-tokio)
+├── crypto.rs      — X25519 ECDH + ChaCha20-Poly1305 per-message encryption
+└── listener.rs    — Background tokio task: serial reader + message dispatcher
+```
+
+Frontend:
+```
+neode-ui/src/stores/mesh.ts    — Pinia store
+neode-ui/src/views/Mesh.vue    — Mesh status, peers, messaging UI
+```
+
+## Dependency
+
+Add to `core/archipelago/Cargo.toml`:
+```toml
+serial2-tokio = "0.1"
+```
+
+All crypto deps already present (chacha20poly1305, ed25519-dalek, curve25519-dalek).
+
+## Meshcore Protocol Summary
+
+- **Frame format**: `>` + 2-byte LE length + data (outbound), `<` + 2-byte LE length + data (inbound)
+- **Baud**: 115200, 8N1
+- **Max message**: 160 bytes
+- **Init sequence**: CMD_DEVICE_QUERY (0x16) -> CMD_APP_START (0x01) -> CMD_SET_DEVICE_TIME (0x06)
+- **Key commands**: SEND_TXT_MSG (0x02), SEND_CHANNEL_TXT_MSG (0x03), GET_CONTACTS (0x04), SYNC_NEXT_MESSAGE (0x0A), SEND_SELF_ADVERT (0x07)
+- **Push events** (async, >=0x80): NEW_CONTACT (0x8A), ACK (0x82), MESSAGES_WAITING (0x83)
+
+## Encryption Design
+
+Reuses existing identity.rs X25519 key agreement:
+1. Nodes broadcast identity on mesh channel: `ARCHY:1:{did}:{ed25519_pubkey}:{x25519_pubkey}`
+2. Receiving node derives shared secret: X25519(our_secret, their_x25519_pub)
+3. All DMs encrypted: ChaCha20-Poly1305 with random 12-byte nonce
+4. Wire format: [nonce 12B] + [ciphertext] + [tag 16B] — fits in 160B limit for ~130B plaintext
+
+## RPC Endpoints
+
+| Method | Action |
+|--------|--------|
+| `mesh.status` | Device + mesh status (updated) |
+| `mesh.peers` | **NEW** — list discovered mesh peers |
+| `mesh.messages` | **NEW** — get message history (last 100) |
+| `mesh.send` | **NEW** — send encrypted message to peer |
+| `mesh.broadcast` | Broadcast identity (updated for Meshcore) |
+| `mesh.configure` | Update config (updated) |
+
+## Implementation Steps
+
+1. **Create mesh/ module, migrate existing code** — types.rs + mod.rs from mesh.rs
+2. **protocol.rs** — Binary frame encode/decode, command builders, response parsers + unit tests
+3. **crypto.rs** — X25519 ECDH + ChaCha20-Poly1305 encrypt/decrypt + unit tests
+4. **serial.rs** — MeshcoreDevice with open/init/send/recv + device auto-detection
+5. **listener.rs** — Background task: serial reader, peer cache, message store, reconnect
+6. **mod.rs MeshService** — Wraps listener + config, start/stop lifecycle
+7. **Update RPC handlers** — New endpoints, wire MeshService into RpcHandler
+8. **Update RPC dispatch** — Add routes in mod.rs ~line 622
+9. **Frontend store + view** — mesh.ts Pinia store, Mesh.vue with glass-card UI, router + nav
+10. **Deploy + test** — Deploy to .228 and .198, plug in Heltec V3s, test end-to-end
+
+## Key Files to Modify
+
+- `core/archipelago/src/mesh.rs` -> delete, replace with `mesh/` directory
+- `core/archipelago/src/api/rpc/mesh.rs` — update handlers
+- `core/archipelago/src/api/rpc/mod.rs` — add routes (~line 622)
+- `core/archipelago/Cargo.toml` — add serial2-tokio
+- `neode-ui/src/router/index.ts` — add /dashboard/mesh route
+- `neode-ui/src/views/Dashboard.vue` — add Mesh nav item
+
+## Reusable Existing Code
+
+- `identity.rs` lines 140-152: Ed25519 -> X25519 conversion (CompressedEdwardsY -> Montgomery)
+- `identity.rs` `pubkey_bytes_from_did_key()`: extract raw pubkey from DID string
+- `node_message.rs` pattern: IncomingMessage store with max 100 circular buffer
+- `mesh.rs` `MeshConfig` + `load_config`/`save_config`: migrate directly into mod.rs
+- `mesh.rs` `detect_meshtastic_devices()`: keep as fallback, add Meshcore probe-based detection
+
+## Prerequisites
+
+- Flash both Heltec V3 with Meshcore **Companion USB** role
+- Add `archipelago` user to `dialout` group: `usermod -aG dialout archipelago`
+- Connect Heltec V3 to USB on .228 and .198
+
+## Verification
+
+1. `cargo clippy --all-targets` passes with zero warnings
+2. Unit tests pass: protocol encode/decode, crypto encrypt/decrypt roundtrip
+3. Device detected on /dev/ttyUSB0 or /dev/ttyACM0
+4. Init handshake completes (visible in tracing logs)
+5. Identity broadcast from .228, received on .198
+6. Encrypted DM sent .228 -> .198, decrypted and visible in UI
+7. Mesh.vue shows device status, peer list, message history