db+waddrmgr: add scan-horizon scaffolding and rollback eviction by yyforyongyu · Pull Request #1269 · btcsuite/btcwallet

yyforyongyu · 2026-06-17T11:21:46Z

Adds the scan-horizon derivation surface: scan-batch request types, SQL
scan-batch account queries, scan-horizon account types, and the horizon
extension helpers (bounded by MaxAddressIndex, skipping ErrInvalidChild).
Also adds the waddrmgr rollback primitives RestoreSyncedTo (restore a
rolled-back sync tip) and EvictDerivedAddresses (evict rolled-back derived
horizons), with cache-eviction tests.

Split out of the runtime Store PR for reviewability; content-preserving, no
commit changes. Stacked on the scan-batch scaffolding PR (prep-runtime-2,
#1263).

Copilot

Pull request overview

Adds the recovery scan-horizon “derivation surface” across the SQL store and waddrmgr, including SQL helpers to advance account next-index counters, shared horizon-extension logic (bounded by MaxAddressIndex and skipping hdkeychain.ErrInvalidChild), and rollback primitives to undo in-memory state when a walletdb batch transaction is rolled back.

Changes:

Add AdvanceNext{External,Internal}Index SQL queries (SQLite/Postgres) and wire them into prepared sqlc query sets.
Introduce shared db scan-horizon extension scaffolding (ScanHorizon, ScanBatchParams, ExtendScanHorizon + tests).
Add waddrmgr rollback helpers (RestoreSyncedTo, EvictDerivedAddresses) plus cache-eviction tests and interface/mocks updates.

Reviewed changes

Copilot reviewed 11 out of 17 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
wallet/internal/sql/sqlite/sqlc/querier.go	Expose new prepared query methods for monotonic next-index advancement (SQLite).
wallet/internal/sql/sqlite/sqlc/db.go	Prepare/close/carry new statements through `Queries` and `WithTx` (SQLite).
wallet/internal/sql/sqlite/sqlc/accounts.sql.go	Generated bindings for new next-index advancement updates (SQLite).
wallet/internal/sql/sqlite/queries/accounts.sql	Source SQL for monotonic next-index advancement (SQLite).
wallet/internal/sql/pg/sqlc/querier.go	Expose new prepared query methods for monotonic next-index advancement (Postgres).
wallet/internal/sql/pg/sqlc/db.go	Prepare/close/carry new statements through `Queries` and `WithTx` (Postgres).
wallet/internal/sql/pg/sqlc/accounts.sql.go	Generated bindings for new next-index advancement updates (Postgres).
wallet/internal/sql/pg/queries/accounts.sql	Source SQL for monotonic next-index advancement (Postgres).
wallet/internal/db/scan_batch_common.go	New shared horizon-extension logic (`ExtendScanHorizon`) + backend ops abstraction.
wallet/internal/db/scan_batch_common_test.go	Unit tests for horizon extension semantics (no-op, bounds, invalid-child skipping, imported-vs-derived account number).
wallet/internal/db/data_types.go	Add recovery scan types (`ScanHorizon`, `ScanBatchParams`) and `MaxAddressIndex`; clarify derivation-path semantics on `AddressInfo`.
waddrmgr/sync.go	Add `Manager.RestoreSyncedTo` to undo in-memory sync-tip on rollback.
waddrmgr/scoped_manager.go	Add `ScopedKeyManager.EvictDerivedAddresses` to undo in-memory derived-address/cache effects on rollback.
waddrmgr/scoped_manager_test.go	Add tests ensuring eviction removes only the intended cached/queued entries and invalidates account cache.
waddrmgr/interface.go	Extend exported interfaces to include rollback primitives.
bwtest/mock/addr_store.go	Update mocks for `AddrStore` interface expansion.
bwtest/mock/account_store.go	Update mocks for `AccountStore` interface expansion.

Files not reviewed (6)

wallet/internal/sql/pg/sqlc/accounts.sql.go: Generated file
wallet/internal/sql/pg/sqlc/db.go: Generated file
wallet/internal/sql/pg/sqlc/querier.go: Generated file
wallet/internal/sql/sqlite/sqlc/accounts.sql.go: Generated file
wallet/internal/sql/sqlite/sqlc/db.go: Generated file
wallet/internal/sql/sqlite/sqlc/querier.go: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

coveralls · 2026-06-17T11:37:08Z

Coverage Report for CI Build 28395390432

Warning

No base build found for commit 23b1d1e on prep-runtime-2.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 55.397%

Details

Patch coverage: 357 uncovered changes across 9 files (221 of 578 lines covered, 38.24%).

Uncovered Changes

File	Changed	Covered	%
wallet/internal/db/pg/txstore_applyscanbatch.go	113	0	0.0%
wallet/internal/db/sqlite/txstore_applyscanbatch.go	110	0	0.0%
wallet/internal/sql/pg/sqlc/accounts.sql.go	28	0	0.0%
wallet/internal/sql/sqlite/sqlc/accounts.sql.go	28	0	0.0%
wallet/internal/sql/pg/sqlc/db.go	24	0	0.0%
wallet/internal/sql/sqlite/sqlc/db.go	24	3	12.5%
wallet/internal/db/scan_batch_common.go	102	84	82.35%
waddrmgr/scoped_manager.go	68	60	88.24%
wallet/db_ops.go	77	70	90.91%
Total (10 files)	578	221	38.24%

Coverage Regressions

Requires a base build to compare against. How to fix this →

Coverage Stats


Relevant Lines:	57613
Covered Lines:	31916
Line Coverage:	55.4%
Coverage Strength:	12967.62 hits per line

💛 - Coveralls

yyforyongyu · 2026-06-17T13:58:02Z

No issues found by gpt-5.5 xhigh 🦦

yyforyongyu · 2026-06-17T14:20:28Z

No issues found by claude-opus-4-8[1m] 🧭

yyforyongyu · 2026-06-17T19:57:34Z

No issues found by claude-opus-4-8[1m] 🛰️

yyforyongyu · 2026-06-17T20:09:43Z

No issues found by gpt-5.5 🪁

yyforyongyu · 2026-06-22T16:40:34Z

No issues found by claude-opus-4-8[1m] 🧩

yyforyongyu · 2026-06-22T16:41:33Z

No issues found by gpt-5.5 xhigh ✅

yyforyongyu · 2026-06-22T18:13:25Z

No issues found by gpt-5.5 🧭

yyforyongyu · 2026-06-22T18:21:25Z

No issues found by claude-opus-4-8[1m] 🪶

yyforyongyu · 2026-06-22T21:23:40Z

No issues found by gpt-5.5 🧪

yyforyongyu · 2026-06-22T22:40:04Z

No issues found by gpt-5.5 xhigh ✅

Add EvictDerivedAddresses to the AccountStore interface and ScopedKeyManager. It drops recent-address cache entries and pending unlock derivations for a derived account branch range after a horizon rollback.

Add unit tests covering EvictDerivedAddresses on an unlocked manager, dropping pending unlock-derivation entries when locked, and keeping other accounts' pending entries intact.

Add the recovery scan-batch request shapes before the backend implementations: ScanHorizon and ScanBatchParams, the MaxAddressIndex horizon bound matching waddrmgr.MaxAddressesPerAccount, and the AddressInfo.HasDerivationPath flag distinguishing HD-derived addresses (normal and imported-xpub) from raw single imports.

Add the pg and sqlite SQL and generated sqlc for advancing derived account branch indexes during recovery horizon extension. The scan-batch implementation that calls these queries lands in a following commit.

Add the backend-independent scan-batch horizon types: the HorizonAccount account-state snapshot with its branch-state accessor, the branch number constants, and the ScanHorizonOps interface abstracting the per-backend SQL the horizon extension drives.

yyforyongyu added this to the Introduce SQL store milestone Jun 17, 2026

yyforyongyu self-assigned this Jun 17, 2026

yyforyongyu requested a review from Copilot June 17, 2026 11:30

Copilot started reviewing on behalf of yyforyongyu June 17, 2026 11:31 View session

Copilot AI reviewed Jun 17, 2026

View reviewed changes

Comment thread wallet/internal/sql/sqlite/queries/accounts.sql

Comment thread wallet/internal/sql/sqlite/queries/accounts.sql

Comment thread waddrmgr/interface.go

Comment thread waddrmgr/interface.go

Comment thread waddrmgr/sync.go Outdated

yyforyongyu force-pushed the prep-runtime-2 branch from 7aee35e to 3f7db6e Compare June 17, 2026 12:21

yyforyongyu force-pushed the prep-runtime-3 branch from c11f2ba to 68bac61 Compare June 17, 2026 12:21

saubyk added this to lnd v0.22 Jun 17, 2026

github-project-automation Bot moved this to Backlog in lnd v0.22 Jun 17, 2026

yyforyongyu force-pushed the prep-runtime-3 branch from 68bac61 to e4840d1 Compare June 17, 2026 14:47

yyforyongyu force-pushed the prep-runtime-2 branch 2 times, most recently from c6def60 to c0de5c2 Compare June 17, 2026 15:23

yyforyongyu force-pushed the prep-runtime-3 branch from e4840d1 to 0019b73 Compare June 17, 2026 15:23

yyforyongyu force-pushed the prep-runtime-2 branch from c0de5c2 to ed5d9ae Compare June 17, 2026 16:01

yyforyongyu force-pushed the prep-runtime-3 branch 2 times, most recently from fa9bf6c to da711fd Compare June 17, 2026 16:42

yyforyongyu force-pushed the prep-runtime-2 branch from ed5d9ae to 06be21c Compare June 17, 2026 16:42

yyforyongyu force-pushed the prep-runtime-3 branch from da711fd to b458889 Compare June 17, 2026 17:32

yyforyongyu force-pushed the prep-runtime-2 branch from 06be21c to 1424387 Compare June 17, 2026 17:32

yyforyongyu force-pushed the prep-runtime-3 branch from b458889 to b30f2f9 Compare June 17, 2026 19:05

yyforyongyu force-pushed the prep-runtime-2 branch from 1424387 to 2f9eda8 Compare June 17, 2026 19:05

yyforyongyu force-pushed the prep-runtime-3 branch from b30f2f9 to d1d05aa Compare June 17, 2026 19:17

yyforyongyu commented Jun 17, 2026

View reviewed changes

Comment thread wallet/internal/db/data_types.go Outdated

yyforyongyu force-pushed the prep-runtime-3 branch from d1d05aa to 1a3cac9 Compare June 18, 2026 06:15

yyforyongyu force-pushed the prep-runtime-2 branch from 8e24141 to ae98442 Compare June 18, 2026 06:15

yyforyongyu force-pushed the prep-runtime-2 branch from f731b01 to 73f7574 Compare June 22, 2026 17:39

yyforyongyu force-pushed the prep-runtime-3 branch from 7eaff4e to 3c3e52c Compare June 22, 2026 17:39

yyforyongyu commented Jun 22, 2026

View reviewed changes

Comment thread wallet/internal/db/data_types.go

yyforyongyu force-pushed the prep-runtime-3 branch from 3c3e52c to 2509351 Compare June 22, 2026 20:57

yyforyongyu force-pushed the prep-runtime-2 branch from 73f7574 to 0e793a2 Compare June 22, 2026 20:57

yyforyongyu force-pushed the prep-runtime-3 branch from 2509351 to 90f6def Compare June 22, 2026 21:11

yyforyongyu force-pushed the prep-runtime-2 branch from 0e793a2 to 27c7852 Compare June 22, 2026 21:11

yyforyongyu requested a review from GustavoStingelin June 23, 2026 15:02

yyforyongyu force-pushed the prep-runtime-2 branch from 27c7852 to 3804fef Compare June 27, 2026 12:12

yyforyongyu force-pushed the prep-runtime-3 branch from 90f6def to 735dd98 Compare June 27, 2026 12:12

yyforyongyu force-pushed the prep-runtime-2 branch from 3804fef to fd82802 Compare June 27, 2026 12:51

yyforyongyu added 13 commits June 30, 2026 02:52

waddrmgr: add derived address eviction

50dd9b7

Add EvictDerivedAddresses to the AccountStore interface and ScopedKeyManager. It drops recent-address cache entries and pending unlock derivations for a derived account branch range after a horizon rollback.

waddrmgr: test derived address cache eviction

d2dada2

Add unit tests covering EvictDerivedAddresses on an unlocked manager, dropping pending unlock-derivation entries when locked, and keeping other accounts' pending entries intact.

db/sql: add scan batch account queries

16d5092

Add the pg and sqlite SQL and generated sqlc for advancing derived account branch indexes during recovery horizon extension. The scan-batch implementation that calls these queries lands in a following commit.

db: add scan horizon account types

497bbe9

Add the backend-independent scan-batch horizon types: the HorizonAccount account-state snapshot with its branch-state accessor, the branch number constants, and the ScanHorizonOps interface abstracting the per-backend SQL the horizon extension drives.

db: add scan horizon extension helpers

0c02d2f

db: test scan horizon extension helpers

41a47a6

waddrmgr: test multi-branch address eviction

6d68c0e

db/sql: add scan horizon account queries

afaa123

wallet: evict scan horizons on rollback

db21e4a

wallet: test scan horizon rollback eviction

d9c8850

db/pg: implement scan horizon ops

b5ad6dd

db/sqlite: implement scan horizon ops

f893d51

Uh oh!

Conversation

yyforyongyu commented Jun 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coveralls commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report for CI Build 28395390432

Coverage: 55.397%

Details

Uncovered Changes

Coverage Regressions

Coverage Stats

💛 - Coveralls

Uh oh!

yyforyongyu commented Jun 17, 2026

Uh oh!

yyforyongyu commented Jun 17, 2026

Uh oh!

yyforyongyu commented Jun 17, 2026

Uh oh!

yyforyongyu commented Jun 17, 2026

Uh oh!

Uh oh!

yyforyongyu commented Jun 22, 2026

Uh oh!

yyforyongyu commented Jun 22, 2026

Uh oh!

yyforyongyu commented Jun 22, 2026

Uh oh!

yyforyongyu commented Jun 22, 2026

Uh oh!

Uh oh!

yyforyongyu commented Jun 22, 2026

Uh oh!

yyforyongyu commented Jun 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

coveralls commented Jun 17, 2026 •

edited

Loading