release: LND 0.21.0-beta — SQLite backend, gRPC/REST TLS passthrough, native onion messages#165
release: LND 0.21.0-beta — SQLite backend, gRPC/REST TLS passthrough, native onion messages#165helix-nine wants to merge 16 commits into
Conversation
Move LND to the SQLite backend with native SQL. Existing bolt nodes (and Umbrel / pre-0.21 StartOS imports) are converted on first start by a migrate-sqlite oneshot that gates the lnd daemon; a Database Migration health check shows progress. Fresh installs are born on SQLite. - enforce db.backend=sqlite + db.use-native-sql=true in lnd.conf - migrate-sqlite oneshot: run lnd once on bolt to finalize the kv schema, then lndinit migrate-db to copy buckets to SQLite; finalize run overrides --db.backend=bolt --db.use-native-sql=false on the CLI - track progress in startup-flags.json (no .const-watched writes -> no restart on completion); oneshot + health check are absent on non-migrating starts - build image from ./Dockerfile (lightninglabs/lnd + lndinit binary) - also: default tor.skip-proxy-for-clearnet-targets to true in the model and drop its install seed (completes the pending TODO) - update README + instructions Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bind both ports as passthrough (protocol/addSsl null + secure.ssl) so LND's own StartOS-issued cert reaches the client, instead of StartOS terminating TLS at the edge with the device cert and breaking clients that pin LND's cert via the lndconnect URL. protocol 'https' does not do this — the SDK still synthesizes an addSsl config for it. Add the cert to the REST lndconnect query so REST clients can pin it the same way gRPC clients do, and update the release notes.
…ion-messages config
The 0.21 bump + SQLite backend never actually ran; found and fixed via an
end-to-end test on a real bitcoind node:
- native SQL is set on the lnd daemon CLI, not the conf — a conf-level bool
can't be turned off on the CLI, so the bolt schema-finalize run failed
("cannot use native SQL with bolt backend"). The schema renders it out.
- restore curl in the image: LND 0.21 switched to Alpine and dropped it,
breaking every REST call we shell out for, including wallet unlock.
- supply --fee.url for the neutrino-override schema-finalize run, so it starts
cleanly on bitcoind-backed nodes (neutrino on mainnet requires it).
- rely on lndinit's idempotent exit code for resume (re-running migrate-db on a
tombstoned source exits 0); drop the best-effort substring guess.
- db-migration health check shows its loading message (statusTrigger needs a
short `starting` interval, else the first poll waits the 10-min default).
Onion messages are native in LND 0.21 (feature bit 39), so the pre-0.21 custom
protocol entries (custom-init/nodeann=39) are obsolete and now crash the node
("feature bit 39 already set"). Remove the onion-messages config toggle and its
form mapping, and force custom-init/nodeann/message to undefined so the schema
strips them from upgraded confs. bolt12-pay drops its autoconfig task and bumps
its lnd versionRange separately.
Comprehensive release notes for 0.21.0-beta:0.
Lightning Labs never published a final v0.1.35-beta-lnd-v0.21.0-beta pairing; it went straight from the .rc3 tag to bumping lndinit to v0.1.36. v0.1.36-beta is a final (non-rc) release paired with lnd v0.21.0-beta — newer than the required v0.1.35, so it carries the same channeldb-schema-v35 support the bolt → SQLite migration needs. Drops the dependency on an RC tag and resolves the TODO. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Patch release: Tor v3 onion creation fix, DNS fallback crash fix, on-chain forward-interceptor settlement fix, and stricter final-hop CLTV-expiry validation. Bumps the lnd and lndinit images to v0.21.1-beta.
48c8f22 to
5ed5b95
Compare
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F5qZh4QiVySBLkV5ya7XSr
…n real address changes Doctrine v3 (issue start-technologies#3407): dependency addresses go through a utils.ts bridgeAddress helper chained .const() — the mapped value is the minimal bridge address, so main never restarts on dependency updates, and heals with one restart when a dependency is installed after this service, uninstalled, or changes port. Cross- package container IPs and .startos dials are gone; static config defaults become loopback placeholders; tor consumers fold the allocator-guaranteed 9050 fallback into the map. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PpJxhk25X42WTafMdkUx6t
- no fabricated dependency addresses: absent dependency => absent config/env value (optional fields, omitted keys), never a fake 127.0.0.1 dial default - dependency-task accept entries match absent-when-default fields (prune: 0 or absent both accepted) - neutral naming: the Bitcoin dependency is 'Bitcoin', not 'Bitcoin Core'/'Bitcoin Knots', across prose, release notes, and i18n - instructions.md keeps no version/migration notes - remove orphaned i18n entries added by the migration - normalize the bridgeAddress helper to an explicit early-return and prettier-format wave-touched files Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PpJxhk25X42WTafMdkUx6t
Bumps @start9labs/start-sdk 2.0.1 -> 2.0.3 (dependency and lockfile only). - 2.0.3 fixes a dependency-gated daemon wedging permanently after its dependency's readiness flaps: Daemon.term() unconditionally destroyed the daemon's SubContainer, so the next start() threw "already destroyed" and the daemon never recovered. Adds a non-destroying Daemon.stop() for dependency-driven pauses, and serializes pause/resume transitions. - 2.0.2 runs s9pk.mk's check-deps before packing, not just before install. No packaging API changed between 2.0.1 and 2.0.3, so no source changes were needed. Verified with `npm run check` (tsc --noEmit).
|
Moved the bolt→SQLite migration out of What changed
Verified: Two things worth a look:
One behavioral note on failure handling: |
Move the conversion out of setupMain's daemon chain into the migrate init step,
and drive it the idiomatic way — two temporary runUntilSuccess daemon chains
instead of a hand-rolled sub.spawn/child.kill.
- init/migrateSqlite.ts: needsSqliteMigration() gate, then runSqliteMigration.
Ordered after setupCerts (the conversion curls a temporary LND on 127.0.0.1,
and setupCerts writes the tls.cert covering that SAN).
- sqliteBackend.ts:
- finalizeBoltSchema runs LND as a managed daemon in a runUntilSuccess chain;
a dependent oneshot unlocks the wallet and waits for UNLOCKED, then the SDK
tears LND down. The chain owns its own subcontainer, which term() destroys —
so lndinit later opens a fully-closed db.
- lndinit migrate-db runs as a second runUntilSuccess oneshot in a fresh
subcontainer (the first is destroyed on teardown).
- drop the manual spawn/kill, the per-call waitForState timeout (the
runUntilSuccess timeout is now the single bound), and sqliteMigrationComplete.
- main.ts: drop the migrate-sqlite oneshot, the db-migration health check, and
the needMigration gating; the lnd daemon no longer requires the migration.
- remove the now-dead Database Migration i18n strings across all locales; update
the README migration section and drop its health-check row.
tsc + ncc + `make x86` pass. End-to-end exercise needs a real pre-0.21 bolt node.
|
Rewrote the migration to the idiomatic Design — two
Why the design is shaped this way (from reading the SDK 2.0.3 source, since the old internals weren't trustworthy):
Also dropped the per-call Verified: Not verified end-to-end: exercising the actual conversion needs a real pre-0.21 bolt LND node with channel state, which I can't stand up in the build sandbox (old LND + wallet + mainnet channel.db). The mechanics follow the documented |
|
Tested the upgrade end-to-end — it works. ✅ Installed the released 0.20.1-beta:13 (bolt) on a StartOS master VM, set backend=neutrino, initialized a wallet, and started it → bolt After the migration:
So the two |
The init framework hands each handler a FullProgressTracker (3rd arg) that auto-syncs to setInitProgress. Report the conversion as two named phases so the StartOS update UI shows what's happening instead of a bare "initializing": - runSqliteMigration adds "Finalizing database schema" and "Copying database to SQLite" phases, start()/complete() around each stage (schema phase completes even on resume). Typed against a minimal structural view of the tracker. - localized phase names across all locales. Verified on a real upgrade: both phases surface in the package's update state during the migration, and the node comes up on SQLite.
|
Added init-progress reporting for the migration ( The init framework hands each Each is Verified on the same real upgrade path: during the conversion the package sits in — then it flips to |
…rtos pins Bumps @start9labs/start-sdk 2.0.3 -> 2.0.5 (dependency and lockfile only). 2.0.5 fixes ExVer range operations ignoring the downstream revision: compareVersionRangePoints and adjacentVersionRangePoints compared the upstream twice, so two points sharing an upstream but differing downstream (1.0.0:3 vs 1.0.0:15) collapsed into one. normalize() then dropped the lower of the two, which made packed manifests advertise a canMigrateFrom/canMigrateTo range narrower than the truth for any package whose `other` versions share current's upstream. No upgrade path actually broke — StartOS resolves migrations through the version graph rather than this field — but the manifests were wrong. Also re-resolves the git-ref dependencies (bitcoin-core-startos, tor-startos) to the current head of the branch each spec points at, so they carry start-sdk 2.0.5 too and dedupe against the top-level copy instead of nesting a stale one. No packaging API changed, so no source changes were needed. Verified with `npm run check` (tsc --noEmit).
…mbeds the lnd version UPDATING.md pointed at a dockerTag in startos/manifest/index.ts that does not exist — the manifest uses dockerBuild, and the real pin is `FROM lightninglabs/lnd:<ver>` in the Dockerfile. It also missed a second pin entirely: `COPY --from=lightninglabs/lndinit:<tag>`, whose tag embeds the lnd version (v0.1.36-beta-lnd-v0.21.1-beta). Bumping lnd alone would point at a nonexistent lndinit image. Both pins are now documented, each with an existence check.
…gration Older LND (<v0.14.0-beta) created wtclient.db even when the watchtower client was never used, leaving it at schema version 0. lndinit's migrate-db version-checks channel.db and wtclient.db before copying and hard-aborts if either isn't at its latest schema, so those legacy nodes fail the bolt→SQLite conversion with 'refusing to migrate source database with version 0'. Activate the watchtower client in the schema-finalize run when a wtclient.db is present, so LND applies its pending wtclient migrations (in BuildDatabase, before unlock — the existing UNLOCKED gate already covers it). Lossless: an empty (v0) db is just initialized; one from prior watchtower use keeps its session data. Absent wtclient.db is left untouched (lndinit skips a missing optional db).
wtclient.db version-0 migration blockerAdded handling for a bolt→SQLite conversion failure @DRBoneZ hit on upgrade: Cause. Fix. In Why activate-and-migrate rather than delete (the error's other suggestion): it's lossless in every case. An empty (v0) db is simply initialized to v8; a db from prior watchtower use (versions 1–7 are real, data-bearing schemas) keeps its session data. A blind Verified against lndinit Build: Side note (out of scope, possible follow-up): the neutrino-based finalize run creates a fresh |
…visions from satisfying dependency
LND 0.21.0-beta release (
next→master)The full 0.21 release, now consolidated on
next:lndinitplus native-SQL conversion. Irreversible; release notes warn to back up first. Verified end-to-end on a real bitcoind node (137 MB channel.db): schema-finalize →lndinit migrate-db→ node back up, synced, wallet intact.custom-init/nodeann=39are stripped on upgrade (they otherwise crash 0.21 withfeature bit: 39 already set).TODO: waiting on Lightning Labs to publish the final
lightninglabs/lndinit:v0.1.35-beta-lnd-v0.21.0-betaimage. TheDockerfilepins thelndinitbinary from the.rc3release-candidate image because no final…-lnd-v0.21.0-betalndinit pairing exists yet. Before merge/release: swap theCOPYsource inDockerfileto the final tag (drop.rc3) — see theTODO(release)comment there.Coordinated downstream change
bolt12-paymust drop its LND autoconfig task (onion messages are native now) and bump its lndversionRangeto this release — tracked separately.