Snapshots the in-flight hardening work so subsequent reconcile/Quadlet
phases land on a clean before/after diff.
Changes:
- core/container/src/podman_client.rs: image_uses_insecure_registry()
whitelist for the OVH (146.59.87.168:3000) and legacy Hetzner
(23.182.128.160:3000) HTTP mirrors; podman_network_settings() lifts
custom networks into the Networks map so containers can join them.
- core/archipelago/src/container/prod_orchestrator.rs:
ensure_container_network() creates per-manifest networks on demand;
apply_data_uid() now goes through host_sudo for mkdir -p + chown so
bind-mount roots get created and chowned without password prompts.
- core/archipelago/src/api/rpc/package/{install,update,stacks}.rs:
podman pull adds --tls-verify=false only for whitelisted registries.
- core/archipelago/src/bootstrap.rs: removes stale dev-mode systemd
override on startup (live nodes carried it from old installers).
- core/archipelago/src/config.rs: ignore ARCHIPELAGO_DEV_MODE in prod
binaries — it had been silently rerouting volumes to /tmp.
- apps/bitcoin-{core,knots}/manifest.yml: locate bitcoind at runtime
so image-layout differences don't break entrypoint.
- scripts/app-catalog-image-smoke-test.py: production catalog/image
smoke test that probes a target node before users click Install.
- .gitignore: cover .codex, .pnpm-store, __pycache__, *.bak.
Removes filebrowser.rs.bak and two stale catalog.json.bak files
(verified identical to live counterparts).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resilience-validated release. Three full sweeps of the new resilience
harness against .228 confirm no shipstoppers.
Big user-visible:
- Bitcoin RPC auth durably correct via host-rendered nginx.conf bind-mount,
replaces fragile post-start exec that failed under restricted-cap rootless
podman ("crun: write cgroup.procs: Permission denied")
- Multi-container stack installs (indeedhub, immich, btcpay, mempool) now
emit phase events at every boundary so the progress bar advances
- Apps no longer vanish from the dashboard mid-install (absent-scanner skips
packages in transitional states)
- Indeedhub fresh installs work end-to-end (was 8500+ restart loop): five
missing env vars (DATABASE_PORT, QUEUE_HOST, QUEUE_PORT,
S3_PRIVATE_BUCKET_NAME, AES_MASTER_SECRET) added to install code
- Tailscale install fixed: --entrypoint string was being passed as a single
shell-line arg; switched to custom_args array
- Catalog cleaned of broken entries (dwn, endurain, ollama removed; nextcloud
restored on docker.io)
- Bitcoin Core update path uses correct image (was looking for nonexistent
lfg2025/bitcoin:28.4)
- ISO installs now allocate swap on the encrypted data partition
Infra:
- New resilience harness (scripts/resilience/) — black-box state-machine
tester, every app × every transition. Run before each release.
Sweep #3 final: PASS 107 / FAIL 12 / SKIP 14. The 12 fails are 1 cosmetic
(homeassistant trusted_hosts), 8 harness/timing false-positives, and 3
non-shipstopper tracked items. Down from 23 in baseline sweep #1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
self-update.sh previously rebuilt only the backend binary and Vue
frontend. The custom UI containers (archy-bitcoin-ui, archy-lnd-ui,
archy-electrs-ui) were left untouched forever. That meant any change to
docker/<ui>/{Dockerfile, nginx.conf, index.html, ...} never reached a
running node through OTA; it required a manual SSH + rebuild. This is
exactly why the lnd-ui port fix didnt reach .228 in v1.7.43-alpha.
Add a sync-and-rebuild stage:
1. Hash each docker/<ui>/ tree (content-only, path-stable via
`cd && find` so src and dst compare equal when identical).
2. rsync changed trees to /opt/archipelago/docker/<ui>/.
3. For each changed UI: rebuild image as the archipelago user
(rootless podman), then stop+remove+recreate the container using
the canonical spec from scripts/container-specs.sh. Port mappings,
caps, memory, and security opts all come from the spec, so the
runtime cant drift from the tree.
Also install first-boot-containers.sh into /opt/archipelago/scripts/ so
a later reconciler run or reboot picks up current orchestration logic.
Idempotent: if no UI tree changed since the last update, the whole stage
is a no-op beyond the hash compare. Verified end-to-end on .228 with a
synthetic change to lnd-ui: detection, sync, build, recreate, and HTTP
200 on both the direct container port and the host-nginx /app/lnd/
proxy.
The LND UI container was unreachable on .228 after the v1.7.43-alpha
deploy because three sources of truth disagreed on which port nginx
listens on inside the container:
- docker/lnd-ui/nginx.conf listen 8081
- docker/lnd-ui/Dockerfile EXPOSE 8080
- apps/lnd-ui/manifest.yml host networking, ports: []
- scripts/first-boot-containers.sh -p 8081:8080
- scripts/deploy-to-target.sh -p 8081:80 (de-facto)
- scripts/deploy-tailscale.sh -p 8081:80
- scripts/container-specs.sh SPEC_PORTS=8081:80
Result: podman published host 8081 to container port 80, but no one was
listening on 80 inside, so connections were reset. Canonicalize on
container:80 with host:8081 publish, matching the three deploy paths
already in agreement.
Changes:
- docker/lnd-ui/nginx.conf: listen 8081 -> listen 80
- docker/lnd-ui/Dockerfile: EXPOSE 8080 -> EXPOSE 80
- apps/lnd-ui/manifest.yml: replace host-network (never true) with
bridge networking and explicit 8081:80 port mapping, correcting a
documentation-vs-reality mismatch
- scripts/first-boot-containers.sh: -p 8081:8080 -> -p 8081:80, and
fix the internal-port comment
Verified on .228 after rebuild: curl http://127.0.0.1:8081/ returns HTTP
200 and the /app/lnd/ host-nginx proxy resolves cleanly.
Releases no longer ship as bootable ISOs. Archipelago updates are
distributed as the backend binary plus a frontend tarball referenced by
releases/manifest.json. Nodes OTA-update via scripts/self-update.sh.
Filebrowser and AIUI remain bundled inside the frontend tarball and
deployed atomically, verified present in v1.7.43-alpha release artifact
(189 AIUI files, filebrowser-client bundle).
Archived under image-recipe/_archived/ (resurrectable if ISO distribution
is reintroduced):
- build-auto-installer-iso.sh
- build-unbundled-iso.sh
- test-iso-qemu.sh
- scripts/convert-iso-to-disk.sh
- BUILD-ISO-STATUS.md, ISO-BUILD-CHECKLIST.md
- branding/isohdpfx.bin
- .gitea/workflows/build-iso-dev.yml
Updated release process docs to drop ISO references:
- scripts/create-release.sh (next-steps text)
- docs/BETA-RELEASE-CHECKLIST.md
- docs/hotfix-process.md
- README.md
Every OTA self-update and every ISO capture was implicitly relying on
/opt/archipelago/web-ui/aiui/ already being present on disk. Any node that
had its web-ui directory atomically swapped (for example by a manual
deployment shipping only neode-ui dist output) lost aiui entirely and the
AI Assistant tab fell through to the "needs to be enabled" placeholder.
self-update.sh: drop the rsync --exclude aiui preservation trick and
instead stage demo/aiui into the freshly-built dist tree before rsync.
demo/aiui in the repo is now the source of truth; every update overwrites
the on-disk copy with a matching version rather than carrying forward
whatever stale bundle happened to survive.
build-auto-installer-iso.sh: prepend demo/aiui to the AIUI search list so
ISO builds from a fresh repo clone pick it up automatically, without
requiring a side-checkout of the AIUI project or a live dev server.
This matches create-release-manifest.sh which already bakes demo/aiui
into the release tarball (lines 86-89).
The backend runs as `archipelago` and calls `install_log()` to append
audit lines to the install log on every install / update / remove /
start / stop / restart. Target path was /var/log/archipelago-container-installs.log,
which does not exist and cannot be created by the service because
/var/log/ is root-owned. OpenOptions errors were silently swallowed,
so the log was never written on any node.
Ship a tmpfiles.d rule that pre-creates /var/log/archipelago/ and
container-installs.log with archipelago:archipelago ownership. Move
the const path to match, keeping logs inside the directory logrotate
already rotates (image-recipe/configs/logrotate.conf). Install the
rule from both the ISO build and self-update, and apply it
immediately on self-update so existing nodes get a working log
without needing a reboot.
Verified on .228: file created, backend user can write, backend
binary rebuilt with new const.
The OTA self-update path only refreshed image-versions.sh, leaving
reconcile-containers.sh and container-specs.sh frozen at whatever
version was baked into the ISO that originally provisioned the
node. Any fix to those scripts (notably the --create-missing flag
and the DISK_GB detection fix shipped this round) never reached
existing nodes, and on .228 both scripts were outright missing
because the node predated their inclusion in the ISO recipe.
Install all three helper scripts to /opt/archipelago/scripts/ on
every self-update run. Also preserve the legacy copy of
image-versions.sh at /opt/archipelago/image-versions.sh for any
older backend binaries still looking there first.
The reconcile spec for bitcoin-knots auto-enables prune=550 when
DISK_GB < 1000. DISK_GB was measured via `df /`, which on every
archy install reports the ~30 GB OS partition because user data
lives on a separate encrypted /var/lib/archipelago volume.
Result: every archy node with a 2 TB data drive was silently being
configured as a pruned node, and any bitcoin-knots container
recreated by reconcile would delete its historical blocks down to
the 550 MB prune window on next start.
Observed on .228 (2 TB box): blocks dir went from 384 GB to 926 MB
after a reconcile-triggered restart. Historical archive unrecoverable
without full re-IBD from genesis.
Fix: check /var/lib/archipelago first (where bitcoin data actually
lives). Fall back to / only on first-boot before the data partition
is mounted.
Context: when package update fails after remove-old-container but
before reconcile-recreate, the rollback path in update.rs tries to
restart the old container by name. If the container is already gone
(removed in step 3 of the update), rollback fails silently and the
node is left with no live container for that app but on-disk data
still intact. This is exactly the state .228 ended up in after the
reconcile-script-missing bug killed bitcoin-knots and lnd.
Reconcile was designed to only repair existing containers for
optional apps (SPEC_OPTIONAL=true): it skips "not installed" entries
on the assumption that the install RPC creates them. That safety
check is correct for normal operation but blocks recovery when an
optional-marked container has been destroyed by a failed update.
Fix: add --create-missing flag that overrides the SPEC_OPTIONAL skip.
When set, reconcile treats absent containers exactly the same as
broken containers — it creates them from the canonical spec using
the existing on-disk data directory. Narrow-scope override; the
default behaviour is unchanged.
Updated --help to document all four flags.
Verified on .228: after the failed bitcoin-core update took out both
bitcoin-knots and lnd, running reconcile --container=bitcoin-knots
--create-missing --force (as the archipelago user, not root —
podman is rootless) brought bitcoin-knots back using the pruned
chainstate at /var/lib/archipelago/bitcoin. Repeated for lnd. All
containers now running; electrumx reconnecting; UIs recovering.
Does NOT fix the underlying update-flow rollback hole (rollback
should be able to re-create a container from spec, not just restart
by name). That is a separate commit — this flag is the manual
recovery tool plus the primitive the improved rollback will call.
The Hetzner VPS at 23.182.128.160 was decommissioned. Replace it
everywhere with the OVH VPS at 146.59.87.168, which was previously
the tertiary mirror.
- update.rs: drop DEFAULT_TERTIARY_MIRROR_URL, promote .168 into
the secondary slot as "Server 1 (OVH)"; tx1138 becomes Server 2.
Default mirror list shrinks from 3 to 2.
- container/registry.rs: default RegistryConfig drops .23, promotes
.168 to Server 1 / priority 0, tx1138 stays Server 2 / priority 10.
- api/rpc/package/config.rs: trusted-registry allowlist swaps .23
for .168.
- api/handler/mod.rs: app-catalog fallback URL uses .168.
- neode-ui/views/marketplace/marketplaceData.ts: REGISTRY uses .168.
- scripts/image-versions.sh: ARCHY_REGISTRY_FALLBACK uses .168.
- image-recipe/build-auto-installer-iso.sh: installer ISO registries
use .168 (both podman registries.conf and backend registries.json).
Tests updated to assert on the new 2-entry default lists (registry +
mirror). URL-parser fixture tests in update.rs retain .23 strings —
they exercise string-parsing logic, not mirror policy.
Git remotes: dropped `gitea-vps` and the .23 push URL on the `origin`
multi-push alias (not part of this commit — pure working-copy change).
The previous code computed HOST_GATEWAY from `ip route show default` to
work around an alleged podman 4.3.x limitation. Two problems:
1. The comment was wrong. Podman 4.4+ supports --add-host=host-gateway
natively, and we ship 5.4.2.
2. More critically, `ip route show default` returns the LAN router
(e.g. 192.168.1.254) — the gateway to the internet, not the gateway
to the host. Every container configured with DAEMON_URL or
--bitcoind.rpchost=host.containers.internal was therefore dialing
the WiFi router instead of the host machine, silently failing.
Symptoms this caused on .228:
- LND crash-looped with "dial tcp 192.168.1.254:8332: connection refused"
- Dashboard showed no LND connect details or QR
- ElectrumX DAEMON_URL broken; stuck at 2 KB index for days
- Any service reaching bitcoin-core through the `archy-net` bridge
Replace the computed value with the literal string "host-gateway",
which podman translates to the correct in-network gateway at container
start. Also drop the stale HOST_GATEWAY reference in the Tor-bootstrap
branch (it always fell back to TARGET_IP anyway). Verified on .228:
after recreating bitcoin-core/electrumx/lnd with the new flag, LND
reached the chain backend, ElectrumX resumed indexing, and the
dashboard /lnd-connect-info endpoint succeeded.
Closes failure mode FM5 from docs/bulletproof-containers.md: the v1.7.38 +
v1.7.39 rollouts left every affected node on an unreachable UI (nginx 500)
with no recovery path short of SSH. This release adds a self-check
guardrail to the update flow.
What changed:
- apply_update() writes a pending-verify marker with old+new version and
a 150s deadline immediately before scheduling the service restart.
- verify_pending_update() runs from main.rs startup. If the marker is
present and within its freshness window, the new binary waits 15s for
nginx + backend to settle, then probes https://127.0.0.1/ every 5s for
up to 90s (self-signed certs accepted).
- On any probe success within the window, the marker is cleared and
nothing else happens.
- On window-exhaust, the new binary:
1. Moves the broken /opt/archipelago/web-ui to web-ui.failed.<ts>
(quarantined, not deleted, so we can post-mortem).
2. Restores web-ui.bak on top of web-ui.
3. Calls rollback_update() to restore the previous binary.
4. Updates state.current_version to reflect the rollback.
5. systemctl --no-block restart archipelago so the OLD binary boots.
- Markers older than 10 minutes are treated as stale and cleared without
probing, so a crashed-during-startup marker from weeks ago cannot
spontaneously roll back a healthy node on a later reboot.
- rollback_update() binary copy now goes through host_sudo instead of
tokio::fs::copy, so it escapes the service's ProtectSystem=strict
mount namespace. Without this, the rollback silently failed with
EROFS on /usr/local/bin and orphaned the rollback - the exact
opposite of what auto-rollback is for.
Tests: 4 new unit tests in update::tests covering marker round-trip,
absent-marker noop, no-panic on verify_pending_update with nothing to
verify, and an invariant assert that the 90s probe window stays below
the 600s stale threshold. All passing.
Side fix: scripts/create-release-manifest.sh was dying with exit 141
(SIGPIPE from tar tvzf pipe head pipe awk) under set -euo pipefail.
Replaced with a single awk NR==1 that doesn't short-circuit the upstream
pipe, so the release-build flow is idempotent again.
v1.7.38 and v1.7.39 both shipped with `./` inside the frontend tarball marked
drwx------ (700). Tar extraction preserves archive perms, so every node that
pulled the OTA landed with /opt/archipelago/web-ui at 700, nginx (www-data)
returned 500 "permission denied" on every page, and the browser showed
"Internal Server Error nginx". .116 hit this on both v1.7.38 and v1.7.39
rollouts. The v1.7.39 runtime self-heal in main.rs was the wrong layer —
systemd's ReadOnlyPaths namespace made /opt/archipelago read-only from inside
the archipelago service, so chmod from there returned EROFS.
Root cause: create-release-manifest.sh used mktemp -d (700 default umask) for
staging, then tar preserved that 700 in the archive's root entry.
Fix the archive itself:
- chmod 755 staging dir + `find -type d -exec chmod 755` + `-type f chmod 644`
before tar, so the on-disk entries are correct.
- tar --owner=0 --group=0 --mode='u=rwX,go=rX' to normalize archive perms
belt-and-braces in case file-mode drift ever reappears.
- Post-tar verify: `tar tvzf | head -1` must show drwxr-xr-x at root, or
the release script aborts before the manifest is even generated.
Binary unchanged semantically — the main.rs self-heal stays in as a last-
resort belt (can't hurt on nodes whose FS isn't namespace-isolated), and the
update.rs in-extractor chmod stays in so v1.7.40-onwards extractors are
double-safe. The authoritative fix is the archive.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- auth.rs now infers onboarding-complete from setup_complete + password_hash so
nodes stop bouncing users through the intro wizard after browser clear / update
/ reboot; the flag self-heals to disk on next check
- frontend: "backend uncertain" no longer defaults to /onboarding/intro —
useOnboarding returns null + callers poll / retry instead of flashing the wizard
- login sounds (synthwave, welcome voice, pop, whoosh, oomph) gated by
isFirstInstallPhase(); typing sounds unaffected
- removed FIPS app, Nostr Relay, Nostr VPN, Routstr, Penpot from catalog,
frontend config, Rust AppMetadata + install dispatch + install_penpot_stack;
docker/fips-ui + docker/nostr-vpn-ui + apps/penpot dirs and 5 icons deleted;
15 image versions deleted from tx1138, .168, gitea-local registries (.160
Gitea was 502 at release time — follow-up)
- AIUI baked into frontend release tarball via demo/aiui/; deploy-to-target
falls back to demo/aiui/ when the AIUI sibling checkout is missing
- prebuild hook syncs app-catalog/catalog.json → public/catalog.json so the
two copies can no longer drift (was the source of the "apps still visible"
bug — public/ had stale data)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- core/archipelago/src/bootstrap.rs (NEW): embed scripts/container-doctor.sh
and image-recipe/configs/archipelago-doctor.{service,timer} via
include_str! and sync to disk + enable the timer on every archipelago
startup. Idempotent (content-hash compare), dev-box symlink guard keeps
the git checkout untouched, best-effort (warn-only on failure) so
bootstrap never blocks server readiness. Wired in main.rs as a
background tokio task.
- scripts/container-doctor.sh: add fix_rootless_netns_egress(). Detects
when the rootless-netns has lost its pasta tap (container-to-container
still works but outbound DNS/TCP fails) via an nsenter probe into
aardvark-dns; with a two-probe 10s debounce to rule out transients and
a host-precheck that bails out if the host itself is offline. When the
rootless-netns is truly broken, does a graceful podman stop --all /
start --all so pasta + aardvark-dns rebuild the netns from scratch.
Bitcoin-knots and every other outbound container recover in one cycle.
- core/archipelago/src/update.rs: host_sudo → pub(crate) so bootstrap.rs
can reuse the existing systemd-run escape hatch.
- apps/bitcoin-core/manifest.yml: bump app version 24.0.0 → 28.4.0 and
image bitcoin/bitcoin:24.0 → bitcoin/bitcoin:28.4. Resources aligned
with the real container-specs.sh large-disk tune (4 GiB memory cap,
cpu_limit: 0 so bitcoind can run -par=auto across every core).
- neode-ui/src/views/apps/AppCard.vue + Apps.vue: add an Update button
+ Updating spinner to every app card that has available-update set.
Wires through serverStore.updatePackage(id) — the same RPC the detail
view already calls. common.update / common.updating i18n keys added in
en.json and es.json.
- core/archipelago/src/identity_manager.rs: add create_from_signing_key()
that mirrors an existing Ed25519 key as a manager-level identity with
a deterministic id (`node-<pubkey16>`). Idempotent across restarts,
gets the hex-SVG master avatar.
- core/archipelago/src/server.rs: the auto-create path on first boot now
mirrors the node's own signing_key (seed-derived on onboarded installs)
as a "Node" identity instead of generating a random "Default" keypair.
Once this ships, the DID on the Web5 DID Status card (via node.did
RPC), the Node entry on the Identities page (via identity.list), and
the DID used for peer-to-peer connects (via server_info.pubkey) all
resolve to the same seed-derived pubkey.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Backend: install.rs registry reachability probe now strips the
`host[:port]/namespace` suffix before appending `/v2/` (the Docker
V2 API lives at the host root, not under the namespace) and accepts
HTTP 405 in addition to 200/401 as "registry daemon alive". This
fixes false "unreachable" reports on the Test button for Gitea and
other registries that protect their /v2/ endpoint.
- Backend: stacks.rs install_indeedhub_stack now force-removes any
leftover indeedhub-* containers and indeedhub-net before creating
the stack. A partial install (or the old first-boot stub racing the
installer) used to leave containers around that blocked re-install
with "name already in use". Re-running the App Store install now
self-heals.
- Backend: registry.rs load_registries auto-merges any default
registry URLs missing from the saved config (appended with priority
max+10+i, persisted). Lets new default mirrors (e.g. Server 3 OVH)
roll out to existing nodes without manual config edits. Explicit
removals still stick — URLs absent from disk AND absent from
defaults stay gone.
- Backend: update.rs adds DEFAULT_TERTIARY_MIRROR_URL at
http://146.59.87.168:3000/ (Server 3 OVH) to default_mirrors, with
the same auto-merge-on-load behavior as registries. Test updated
for 3-mirror default (.160, tx1138, .168).
- Scripts: dropped the first-boot IndeedHub stub (~38 lines in
first-boot-containers.sh §8b). It predated the proper stack
installer, raced it, and was the main source of the name-conflict
mess the stacks.rs cleanup above now also guards against.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to 8b7cb002 (no version bump — same v1.7.0-alpha manifest):
* WireGuard peer persistence. Kernel peer state is ephemeral; the add-peer
RPC wrote each peer to data_dir/nostr-vpn/peers/*.json but nothing
re-pushed them on reboot. Result on .198: wg0 came up listening with zero
peers after last night's reboot. Added vpn::restore_wg_peers() — reads
the peers dir, waits up to 30s for wg0 to exist, then replays each via
`archipelago-wg add-peer`. Spawned from main.rs alongside the other
startup tasks.
* Reconcile + filebrowser drift. scripts/container-specs.sh load_spec_
filebrowser now declares SPEC_NETWORK="archy-net" (to match what
first-boot-containers.sh creates) and pins the filebrowser-data volume
+ wget-style healthcheck so the reconciler stops reporting network
drift. Without this, reconcile would kill the healthy first-boot
filebrowser container and recreate it on bridge, breaking the archy-net
DNS name the backend proxies to.
Manifest binary sha/size refreshed:
6c178a76…3582cc, 40361912 bytes.
Rebuilt ISO at image-recipe/results/archipelago-installer-unbundled-x86_64.iso
(Apr 20 07:10) carries both fixes baked in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes bundled into the OTA:
1. update.download hard-fail on git-path nodes. handle_update_check's git
branch reported update_available=true + update_method="git" but never
populated state.available_update, so update.download returned "No update
available to download" even though the UI showed one. SystemUpdate.vue
now routes update_method=="git" through update.git-apply (pull+rebuild+
restart via self-update.sh); manifest-path nodes keep the download→apply
flow. i18n strings + confirm modal added for the git path.
2. Reconciler creating containers behind the user's back. On fresh
unbundled installs (.198, .253) archy-mempool-db and archy-btcpay-db
materialised ~10 min after first boot because reconcile-containers.sh
walked container-specs.sh's canonical tier list and created any
"missing" container. reset_spec() now defaults SPEC_OPTIONAL="true",
so reconcile is strictly a repair tool — baseline comes from
first-boot-containers.sh (filebrowser on unbundled), everything else
from the install RPC.
Also forces OTA trigger for nodes on 1.6.0-alpha that otherwise saw
"I'm at manifest.version, nothing to do" and skipped the refreshed 1.6
artifacts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Real 413 root cause on .116 and .228 turned out not to be the body-size
limit — their /etc/nginx/sites-enabled/archipelago was a stale regular
FILE, not a symlink to sites-available, so every nginx update since
someone froze the active config had been invisible to running nginx.
The /api/blob location, added at some point after that freeze, didn't
exist in sites-enabled, so every attachment upload hit nginx's default
1m client_max_body_size and returned 413 regardless of attachment
size.
Deploy now re-creates the symlink on every run: if sites-enabled is a
regular file or missing, we replace it with a symlink to
sites-available. Idempotent if it's already correct.
Also applied the fix live on all 4 fleet nodes — /api/blob now
responds 401 (session-auth required, as designed) instead of 413 on
2MB+ test uploads.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Versioning was drifting on three axes — fixed all of them:
1. Cargo.toml → 1.5.0-alpha (was 1.5.0). User wants `-alpha` suffix
on every pre-stable release; this is the current state of main.
2. neode-ui/package.json was still 1.3.5 — brought in line.
3. /opt/archipelago/build-info.txt was stale on .198 (1.3.4) and
.253 (1.3.5), absent on .116/.228. That file OVERRIDES the
binary's CARGO_PKG_VERSION for the UI sidebar, which is why
.198/.253 kept showing old versions even with fresh binaries.
scripts/deploy-to-target.sh now writes build-info.txt on every
deploy, reading the version straight from Cargo.toml — so the
sidebar can never drift from the binary again.
Release artifacts + manifest:
- releases/v1.5.0-alpha/archipelago (40M, sha in manifest)
- releases/v1.5.0-alpha/archipelago-frontend-1.5.0-alpha.tar.gz (51M)
- releases/manifest.json bumped with full 7-line changelog covering
FIPS-first routing, Settings toggle, transitive federation, cancel
button, transport badges, peer listener, and the build-info fix.
- scripts/check-release-manifest.sh — new pre-publish guard. Refuses
to pass if: Cargo.toml ≠ manifest version, changelog is empty
(release notes are mandatory), or any component's sha256/size
doesn't match the file on disk. Run locally or from CI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- TransportPrefsCard.vue: import from '@/api/rpc-client' (not
'@/api/rpc') so vue-tsc resolves the module during build.
- scripts/fleet-fips-unpair.sh: companion to the fleet-pair script —
rewrites each node's fips.yaml to anchor-only (fips.v0l.io) so we
can prove the general-case deployment works without the LAN
fast-path. Prints per-node peer counts + DHT AAAA resolution for
every cross-node pair after the change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/fleet-fips-pair.sh writes a deterministic /etc/fips/fips.yaml
on each of our 4 dev fleet nodes (.116/.198/.228/.253), listing the
other three as static FIPS peers over their LAN IPs (UDP 2121 / TCP
8443). Also flips `node.identity.persistent: true` so the npub stays
stable across restarts — without this the daemon rolls a new keypair
on every restart and federation invites that carried the previous
npub go stale.
The script is NOT the general deployment mechanism:
- Every archipelago install already ships fips.v0l.io as an anchor
peer, so any node can DHT-route to any npub that has ever announced
on the public mesh.
- Federation invites (v1.4+) carry the peer's fips_npub, so accepting
an invite is enough for crate::fips::dial::peer_base_url(npub) to
reach the peer through the anchor network.
- This script is a LAN fast-path optimization so intra-fleet traffic
stays on the wire instead of bouncing through fips.v0l.io.
Usage:
scripts/fleet-fips-pair.sh # apply to all nodes
scripts/fleet-fips-pair.sh --verify # print current peer state
Verified: all 4 fleet nodes now report 3 authenticated peers each
(their 3 fleet siblings), plus fips.v0l.io in the identity cache.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The .github/workflows/ci.yml Rust job runs cargo fmt --check, clippy
with -D warnings, and tests. All three were failing. This commit:
- Applies rustfmt across the tree (the bulk of the diff — untouched
since the last toolchain bump, so a wide sweep was unavoidable).
- Fixes the correctness-level clippy errors:
container/bitcoin_simulator.rs wildcard-in-or-pattern
container/manifest.rs from_str rename to parse (reserved name)
container/podman_client.rs .get(0) -> .first()
container/runtime.rs manual += collapse
archipelago/src/constants.rs doc-comment → module-doc
api/rpc/package/install.rs stray /// comment above a non-item
container/docker_packages.rs redundant field init
streaming/advertisement.rs missing Metric import in tests
tests/orchestration_tests.rs `vec!` in non-Vec contexts
mesh/listener/dispatch.rs unused store_plain_message import
api/rpc/tor/mod.rs and mesh/steganography.rs: push-after-new → vec!
- Quiets wide legacy surfaces with crate-level allows in main.rs for
stylistic lints (too_many_arguments, type_complexity, doc indent,
enum variant prefix, wildcard-in-or, assertions-on-constants,
drop_non_drop, unused_io_amount, ptr_arg) — these fired in dozens
of places with no correctness payoff and have been churning every
toolchain bump.
- Tags intentional-dead-code helpers: wallet/ and streaming/ modules
are WIP, mesh::send_chunked_payload and DM_V1_MARKER are kept for
rollback compatibility, vpn::get_nostr_vpn_status is surface-area
for a not-yet-landed RPC.
cargo fmt --check, cargo clippy --all-targets --all-features
-- -D warnings, and cargo test --all-features now all pass locally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add deploy_secondary() function for deploying to multiple LAN nodes
- --both now deploys to .198 and .253 (previously .198 only)
- Fleet deploy updated for 3 LAN nodes
- Mesh DM fixes: protocol frame format, DM-via-channel routing
- Federation pending requests, discover modal
- VPN status UI improvements
- Image versions and container specs updates
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Swapped all registry references: image-versions.sh, marketplaceData.ts,
curatedApps.ts, catalog.json. git.tx1138.com is now fallback only.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Registry fallback now only tries DIFFERENT registries (skips original
that already failed). 120s timeout per fallback attempt. WireGuard
keys generated on unbundled first-boot. Gitea ROOT_URL uses port 3001.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. registries.conf includes docker.io search + fallback 23.182.128.160
2. First-boot pull_with_fallback() tries primary then fallback registry
3. FileBrowser created with noauth config on persistent volume
4. Backend dynamic registries.json pre-created in ISO
5. Filebrowser password secret created for token flow
Fixes: apps stuck at 0% download, filebrowser not working, dynamic
catalog not loading on fresh installs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
App catalog served from Gitea repos (app-catalog) with 35 apps.
Nodes fetch catalog dynamically — new apps appear without frontend
rebuild. Test app added and removed to verify pipeline.
Gitea manifest updated with internal_port/nginx_proxy for iframe.
Updated catalog.json, nginx configs, app session configs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When primary registry (git.tx1138.com) fails, image pull automatically
retries from Gitea registry at 23.182.128.160:3000. Tags pulled image
with original name so install continues seamlessly. Gitea added as
external app in app session config.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
MediaLightbox: full glassmorphic redesign with dark backdrop, smooth
transitions, proper video/audio/image support. FileBrowser: noauth
config on persistent volume. Deploy script: fixed sed quoting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cashu ecash protocol (BDHKE blind signatures, cashuA token format,
mint HTTP client) replacing the stub wallet. TollGate-inspired streaming
data payment system with step-based pricing (bytes/time/requests),
session management with incremental top-ups, usage metering, and
Nostr kind 10021 service advertisements.
13 new streaming.* RPC endpoints. Content server now verifies real
Cashu tokens. Profits tracking includes streaming revenue.
Frontend: GlobalAudioPlayer (persistent bottom bar across all pages),
video lightbox with full controls, audio in MediaLightbox, free file
previews (no blur), paid 10% audio/video previews, separated play
vs download buttons in PeerFiles.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Apps are now installed exclusively via the Marketplace UI.
The deploy script handles code sync, backend/frontend builds,
and service restarts only. The legacy container creation code
is wrapped in `if false` to preserve git history.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ISO builder: run npm ci before npm run build to prevent stale UI artifacts
- Unbundled ISO: clean container-images dir to prevent bundled tars leaking
- WireGuard: use After=network.target instead of network-online.target for
faster wg0 startup on install
- VPN status: check actual nvpn0 interface instead of config tunnel_ip to
prevent NostrVPN from showing standalone WireGuard IP
- ContainerApps: filter out not-installed bundled apps (fixes Bitcoin Knots
appearing on clean unbundled installs)
- Kiosk: persist kiosk mode to localStorage before /kiosk redirect so
App.vue can skip remote relay (fixes input doubling with companion app)
- IndeedHub: fix port mapping and X-Forwarded-Prefix passthrough
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All hardcoded references to the old IP-based registry replaced across
Rust backend, Vue frontend, shell scripts, Dockerfiles, CI, and docs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Unbundled ISO: first-boot only creates FileBrowser (marker file .unbundled)
Users install apps from Marketplace — no more bitcoin/mempool on clean install
- VPN status: read tunnel IP from config file (instant) instead of nvpn status (22s)
- Kiosk: App.vue skips remote relay on /kiosk path (prevents duplicate input)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Linux-only option that mirrors ISO install exactly: builds backend
(release), frontend (with typecheck), syncs all configs, and restarts
all system services (Tor, WireGuard, NostrVPN, nginx, backend).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update all references from Debian 12 (Bookworm) to Debian 13 (Trixie)
- Enable SystemCallArchitectures, RestrictAddressFamilies, RestrictRealtime
in archipelago.service (safe on systemd 256+ which respects NoNewPrivileges=no)
- Update GLIBC compatibility checks from 2.36 to 2.40
- ISO filename, build container, and docs updated throughout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace aardvark-dns container names with host.containers.internal
for all cross-app connections (LND→Bitcoin, ElectrumX→Bitcoin,
Mempool→ElectrumX, Fedimint→Bitcoin, NBXplorer→Bitcoin P2P+RPC)
- Add BTCPay multi-container stack installer (postgres + nbxplorer +
btcpay-server) with proper secrets, data dir ownership, NOAUTH
- Add Mempool multi-container stack installer (mariadb + mempool-api +
mempool-frontend) with host.containers.internal for RPC
- Immediately remove apps from state on uninstall (no 3-min ghost delay)
- Include archy-bitcoin-ui in bitcoin uninstall container list
- Fix LND UI port 8081 (was 8080, conflicting with LND gRPC)
- Fix ElectrumX UI: proxy /electrs-status to backend, cache-busting
headers, graceful fallback when backend returns HTML
- Add Tor hidden services for ElectrumX and LND in torrc template
- Remove unused detect_bitcoin_container_name() (replaced by
host.containers.internal)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Standalone WireGuard (wg0:51820):
- New archipelago-wg.service creates wg0 independent of NostrVPN
- Keypair generated on first-boot, persisted on LUKS partition
- vpn.create-peer uses wg genkey/pubkey (no nvpn dependency)
- wg-address service depends on archipelago-wg, not nostr-vpn
Networking fixes:
- Remove nos.lol from default relays (requires PoW, events rejected)
- Add Tor hidden service for private relay (port 7777) — NAT'd peers
can reach relay over Tor for NostrVPN signaling
- Fix Tor hostname sync race: wait loop before copying hostname files
- Add tor-hostnames + wireguard dirs to LUKS partition setup
- Include relay in hostname sync loops (setup-tor.sh + first-boot)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The nvpn daemon config at /var/lib/archipelago/nostr-vpn/ is owned by
root, but the backend runs as archipelago. Direct write silently failed,
so adding a second phone's npub never reached the daemon — service
restarted with stale config. Now falls back to sudo cp for root-owned
paths, and first-boot sets ownership to archipelago.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
