lfg2025/archy
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases 33 Wiki Activity

bug fixing and deploy and build diagnostics

Browse Source
This commit is contained in:
Dorian
2026-03-22 03:30:21 +00:00
parent 1f8287c4c3
commit e4e0ef4f11
198 changed files with 21703 additions and 19587 deletions
Download Patch File Download Diff File Expand all files Collapse all files

326
docs/architecture.md
View File

@@ -1,186 +1,190 @@
# Archipelago Bitcoin Node OS - Architecture Documentation
# Archipelago — Architecture
## Overview
> **Bitcoin Node OS** — Flash to USB, install on hardware, manage via web UI.
Archipelago is a self-sovereign Bitcoin Node OS built on Debian Linux with Podman containerization. Flash to USB, install on hardware, manage via web UI. It runs 30+ containerized apps (Bitcoin, Lightning, BTCPay, Mempool, DWN, and more) with multi-node federation over Tor, W3C DID identity, and encrypted backups.
**Stack**: Rust backend + Vue 3 + TypeScript (strict) + Vite 7 + Tailwind CSS + Pinia + rootless Podman
**Target OS**: Debian 12 (Bookworm) — x86_64 and ARM64
**Status**: Beta freeze (Phase 1: Feature Testing)
## System Architecture
For the full interactive architecture review with diagrams and learning guide, see [`architecture-review.html`](architecture-review.html).
---
## System Layers
```
┌─────────────────────────────────────────────────────────┐
│ Debian Linux Base (Bookworm) │
│ - Stable, well-supported kernel │
│ - Systemd service management │
│ - Extensive hardware support │
└─────────────────────────────────────────────────────────┘
│
┌───────────────┼───────────────┐
│ │ │
┌───────▼──────┐ ┌──────▼──────┐ ┌─────▼──────┐
│ Podman │ │ Rust Backend│ │ Vue.js UI │
│ (rootless) │ │ (core/) │ │ (neode-ui/) │
└───────┬──────┘ └──────┬──────┘ └─────────────┘
│ │
└───────┬───────┘
│
┌───────────▼───────────┐
│ Container Orchestration│
│ Layer │
│ - Manifest parser │
│ - Podman client │
│ - Dependency resolver │
│ - Health monitor │
└───────────┬───────────┘
│
┌───────────▼───────────┐
│ Containerized Apps │
│ - Bitcoin Core │
│ - LND / CLN │
│ - BTCPay Server │
│ - Nostr Relays │
│ - Meshtastic │
│ - Web5 DWN │
└───────────────────────┘
┌──────────────────────────────────────────────────────┐
│ YOUR BROWSER │
│ Vue 3 SPA (Composition API + Pinia) │
└──────────────────────┬───────────────────────────────┘
│ HTTP / WebSocket
┌──────────────────────┴───────────────────────────────┐
│ NGINX │
│ /rpc/v1 → backend /app/{id}/ → container │
└──────────────────────┬───────────────────────────────┘
│ port 5678
┌──────────────────────┴───────────────────────────────┐
│ RUST BACKEND (core/) │
│ Auth, RPC, containers, federation, mesh, health │
└──────────────────────┬───────────────────────────────┘
│ Podman REST API socket
┌──────────────────────┴───────────────────────────────┐
│ ROOTLESS PODMAN CONTAINERS │
│ 30 apps: Bitcoin, LND, Mempool, BTCPay, etc. │
└──────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────┐
│ DEBIAN 12 (Bookworm) │
│ systemd, UFW, Tor, AppArmor, filesystem │
└──────────────────────────────────────────────────────┘
```
## Key Components
## Codebase Stats
### 1. Debian Linux Base
| Component | Lines | Files |
|-----------|-------|-------|
| Rust backend (`core/`) | ~49,000 | 146 |
| TypeScript/Vue (`neode-ui/src/`) | ~47,000 | 155 |
| Shell scripts | — | 78 |
| Container apps | — | 30 |
- **Distribution**: Debian 12 (Bookworm) - stable, LTS support
- **Init System**: Systemd for service management
- **Security**: AppArmor, standard Debian hardening
- **Multi-arch**: ARM64 (Raspberry Pi) and x86_64 support
## Backend Crates (`core/`)
### 2. Container Orchestration Layer
| Crate | Purpose |
|-------|---------|
| `archipelago` | Main binary — RPC endpoints, auth, identity, federation, mesh, health |
| `container` | PodmanClient (REST API), manifest parser, dependency resolver |
| `models` | Shared data types across the workspace |
| `helpers` | Common utilities |
| `js-engine` | Embedded JS runtime for extensibility |
| `security` | Secrets encryption, AppArmor profiles, Cosign verification |
| `performance` | CPU/memory/disk monitoring |
| `container-init` | Container initialization and first-run setup |
| `parmanode` | Parmanode compatibility layer |
Located in `core/container/`:
- **manifest.rs**: Parses YAML app manifests
- **podman_client.rs**: Wraps Podman API for container management
- **dependency_resolver.rs**: Resolves app dependencies and conflicts
- **health_monitor.rs**: Monitors container health and auto-restarts
### Key Backend Files
### 3. Backend API Extensions
```
core/archipelago/src/
├── api/handler.rs — HTTP routing (/rpc, /health, /dwn, /ws)
├── api/rpc/mod.rs — RPC dispatcher (100+ endpoints)
├── api/rpc/package.rs — App install/start/stop/configure
├── session.rs — Auth sessions, rate limiting, persistence
├── health_monitor.rs — Container health + auto-restart
├── crash_recovery.rs — Boot-time container recovery
├── federation.rs — Multi-node federation over Tor
├── identity.rs — Ed25519 node identity, did:key
├── identity_manager.rs — Multi-identity (Personal/Business/Anonymous)
├── credentials.rs — W3C Verifiable Credentials
├── nostr_discovery.rs — Nostr presence (kind 30078)
├── nostr_handshake.rs — NIP-44 encrypted peer comms
├── network/dwn_store.rs — DWN message persistence
├── network/dwn_sync.rs — Bidirectional DWN replication over Tor
├── monitoring/ — System metrics
├── update.rs — Update checking + scheduling
└── wallet/ — LND integration, ecash
```
New RPC endpoints in `core/archipelago/src/container/`:
- `container-install`: Install app from manifest
- `container-start/stop/remove`: Container lifecycle
- `container-status/logs`: Status and debugging
- `container-list`: List all containers
- `container-health`: Health status aggregation
## Frontend (`neode-ui/src/`)
### 4. Vue.js UI Integration
```
├── api/ — RPC client, WebSocket, container client
├── views/ — Dashboard, Marketplace, Settings, Web5, Mesh, Login
├── components/ — BootScreen, SplashScreen, SpotlightSearch, etc.
├── stores/ — Pinia: app, container, mesh, appLauncher, etc.
├── composables/ — useToast, useAudioPlayer, etc.
├── types/ — TypeScript type definitions
├── router/ — Vue Router
└── style.css — Global glassmorphism theme
```
New components in `neode-ui/`:
- **ContainerApps.vue**: List of containerized apps
- **ContainerAppDetails.vue**: Detailed app view with logs
- **ContainerStatus.vue**: Status indicator component
- **container-client.ts**: API client for container operations
- **container.ts**: Pinia store for container state
## Container Apps (30)
### 5. App Manifest System
Standardized YAML format in `apps/`:
- Defines container image, resources, dependencies
- Security policies and health checks
- Bitcoin/Lightning/Web5 integration metadata
### 6. Identity & Federation
Located in `core/archipelago/src/`:
- **identity.rs**: Ed25519 node identity, did:key generation, DID Documents
- **identity_manager.rs**: Multi-identity manager (Personal/Business/Anonymous)
- **credentials.rs**: W3C Verifiable Credentials (issue, verify, present, revoke)
- **federation.rs**: Multi-node federation with invite codes, trust levels, Tor sync
- **network/dwn_store.rs**: DWN message persistence (JSON files on disk)
- **network/dwn_sync.rs**: Bidirectional DWN replication over Tor
- **nostr_discovery.rs**: Nostr presence publishing (kind 30078, no onion addresses)
- **nostr_handshake.rs**: NIP-44 encrypted peer communication
### 7. Security Modules
Located in `core/security/`:
- **container_policies.rs**: Generates AppArmor profiles
- **secrets_manager.rs**: Encrypted secrets storage
- **image_verifier.rs**: Cosign signature verification
### 8. Performance Optimization
Located in `core/performance/`:
- **resource_manager.rs**: CPU/memory/disk allocation
- **optimize-debian.sh**: OS-level optimizations
## App Categories
### Bitcoin & Lightning
- Bitcoin Core (full node)
- LND (Lightning Network Daemon)
- Core Lightning (CLN)
- BTCPay Server
- Mempool (blockchain explorer)
### Web5 & Decentralized Protocols
- Nostr relays (nostr-rs-relay, strfry)
- Web5 DWN (Decentralized Web Node)
- DID Wallet
- Bitcoin Domain Names
### Mesh Networking & Routing
- Meshtastic (LoRa mesh networking)
- Router (mesh routing, device discovery)
- Local network management
### Self-Hosted Services
- Home Assistant
- Grafana
- SearXNG
- OnlyOffice
- Ollama (local AI)
- Penpot
## Security Model
1. **OS Level**: Debian hardening, AppArmor, minimal installed packages
2. **Container Level**: Rootless Podman, capability dropping, network isolation
3. **Secrets**: Encrypted storage, runtime injection only
4. **Supply Chain**: Signed images (Cosign), SBOM generation
5. **Network**: Firewall (nftables/iptables), rate limiting, Tor integration
6. **Audit**: Journald logging, configuration tracking
**Bitcoin & Lightning**: Bitcoin Knots, LND, BTCPay Server, ThunderHub, Mempool, ElectrumX, Fedimint
**Nostr**: nostr-rs-relay, Nostrudel
**Self-Hosted**: Nextcloud, Jellyfin, Immich, PhotoPrism, Vaultwarden, Home Assistant, OnlyOffice, Penpot, SearXNG, FileBrowser
**Dev/Ops**: Grafana, Portainer, Ollama, Nginx Proxy Manager
**Networking**: Tailscale
**Custom**: DWN, IndeedHub, Router
**External**: BotFights, NWNN, 484 Kitchen, Call the Operator, Arch Presentation, Syntropy Institute, T-Zero
## Networking
- **Container Networks**: `archy-net` (main), `immich-net`, `penpot-net` via Podman
- **Aardvark DNS**: Container hostname resolution within networks
- **UFW Firewall**: Podman subnets (10.88.0.0/16, 10.89.0.0/16) allowed for DNS
- **Tor Integration**: System Tor daemon, SOCKS5 proxy on port 9050
- **Federation**: Inter-node communication over Tor hidden services
- **Nginx**: Reverse proxy on port 80/443, proxies to backend on 5678
- **Container DNS**: `archy-net` for inter-container communication (Bitcoin, LND, ElectrumX, Mempool, BTCPay, Fedimint)
- **Aardvark DNS**: Container hostname resolution within Podman networks
- **Tor**: System Tor daemon, SOCKS5 on port 9050, hidden services per node
- **Federation**: Inter-node communication over Tor with DID-based trust
- **UFW**: `DEFAULT_FORWARD_POLICY="ACCEPT"` required for LAN container access
## Data Persistence
## Security Model
- **App Data**: `/var/lib/archipelago/{app-id}/`
- **Identity**: `/var/lib/archipelago/identity/` (Ed25519 node key)
- **Multi-Identity**: `/var/lib/archipelago/identities/` (per-identity JSON)
- **DWN Messages**: `/var/lib/archipelago/dwn/messages/` (JSON files)
- **Credentials**: `/var/lib/archipelago/credentials/`
- **Federation**: `/var/lib/archipelago/federation/`
- **Content Catalog**: `/var/lib/archipelago/content/`
- **Backups**: `/var/lib/archipelago/backups/` (ChaCha20-Poly1305 encrypted)
- **Secrets**: `/var/lib/archipelago/secrets/{app-id}/` (encrypted)
| Layer | Measures |
|-------|----------|
| OS | Debian hardening, AppArmor, minimal packages |
| Nginx | CSP headers, rate limiting, auth_request, session validation |
| Backend | Input validation, CSRF, session auth, bind 127.0.0.1 only |
| Containers | Rootless Podman, cap-drop ALL, readonly root, non-root user, no-new-privileges, memory limits |
| Crypto | Ed25519 signatures, ChaCha20-Poly1305 encryption, Argon2 password hashing, constant-time comparisons |
| Network | Tor hidden services, UFW firewall, SSRF prevention |
## Multi-Node Federation
## Data Paths
Nodes form a federated network over Tor:
- **Invite-based joining**: Generate invite code, peer joins via Tor
- **Trust levels**: Trusted, Verified, Untrusted
- **State sync**: Federation state, DWN messages, file catalogs sync bidirectionally
- **DWN protocols**: 4 interoperable schemas (node-identity, file-catalog, federation, app-deploy)
- **Verifiable Credentials**: W3C VCs for inter-node trust attestation
| Data | Path |
|------|------|
| App data | `/var/lib/archipelago/{app-id}/` |
| Identity | `/var/lib/archipelago/identity/` |
| Multi-identity | `/var/lib/archipelago/identities/` |
| Federation | `/var/lib/archipelago/federation/` |
| DWN messages | `/var/lib/archipelago/dwn/messages/` |
| Credentials | `/var/lib/archipelago/credentials/` |
| Backups | `/var/lib/archipelago/backups/` (ChaCha20-Poly1305) |
| Secrets | `/var/lib/archipelago/secrets/{app-id}/` (encrypted) |
| Sessions | `/var/lib/archipelago/sessions.json` |
| Frontend | `/opt/archipelago/web-ui/` |
| Backend binary | `/usr/local/bin/archipelago` |
## Future Enhancements
## Active Nodes (5)
- Time-travel snapshots (ZFS/BTRFS)
- Decentralized app marketplace (IPFS + Nostr)
- Multi-node clustering
- Hardware attestation (TPM 2.0)
- Protocol-agnostic design (multi-chain support)
| Node | Access | Notes |
|------|--------|-------|
| Arch 1 | 192.168.1.228 (LAN) | Primary dev, 16GB RAM, 1.8TB NVMe |
| Arch 2 | 192.168.1.198 (LAN) | Secondary, 8GB RAM |
| Arch 3 | 100.82.97.63 (Tailscale) | Has mesh radio |
| Arch 4 | 100.122.84.60 (Tailscale) | — |
| Arch 5 | 100.124.105.113 (Tailscale) | — |
All nodes federated over Tor with bidirectional DWN sync. Deploy via SSH key from Mac.
## Key Features (Working)
- 30 containerized apps with one-click install/manage
- Multi-node federation with invite-based joining and trust levels
- DWN sync (bidirectional message replication over Tor)
- W3C DID identity (did:key, DID Documents, Verifiable Credentials)
- NIP-07 Nostr signing (iframe apps sign events with consent)
- File sharing with access controls (free/peers-only/paid)
- Encrypted backups (Argon2 + ChaCha20-Poly1305)
- Health monitoring with tiered auto-restart (exponential backoff)
- Tiered container startup (databases → core → applications)
- LoRa mesh networking (Meshcore protocol)
- Three-mode UI (Pro/Easy/Chat)
- Real-time WebSocket updates
- Glassmorphism web UI
- Bootable ISO installer
## Further Documentation
| Doc | Purpose |
|-----|---------|
| [`architecture-review.html`](architecture-review.html) | Full interactive architecture guide with diagrams |
| [`developer-guide.md`](developer-guide.md) | Dev setup, workflow, code conventions |
| [`api-reference.md`](api-reference.md) | Complete RPC endpoint reference |
| [`app-developer-guide.md`](app-developer-guide.md) | Building and publishing apps |
| [`user-walkthrough.md`](user-walkthrough.md) | End-user installation and usage guide |
| [`troubleshooting.md`](troubleshooting.md) | Diagnostic scenarios and solutions |
| [`operations-runbook.md`](operations-runbook.md) | Ops commands and emergency recovery |
| [`multi-node-architecture.md`](multi-node-architecture.md) | Federation protocol design |
| [`marketplace-protocol.md`](marketplace-protocol.md) | Decentralized app discovery via Nostr |
| [`security-code-audit-2026-03.md`](security-code-audit-2026-03.md) | Penetration test findings (33 findings, all remediated) |
| [`MASTER_PLAN.md`](MASTER_PLAN.md) | Phased roadmap and task tracking |
| [`BETA-PROGRESS.md`](BETA-PROGRESS.md) | Current beta stabilization progress |