db+waddrmgr: add scan-batch scaffolding and rollback support#1263
Open
yyforyongyu wants to merge 20 commits into
Open
db+waddrmgr: add scan-batch scaffolding and rollback support#1263yyforyongyu wants to merge 20 commits into
yyforyongyu wants to merge 20 commits into
Conversation
Coverage Report for CI Build 28395386750Warning No base build found for commit Coverage: 55.353%Details
Uncovered Changes
Coverage RegressionsRequires a base build to compare against. How to fix this → Coverage Stats
💛 - Coveralls |
yyforyongyu
force-pushed
the
prep-runtime-1
branch
from
f244206 to
19ba9fd
Compare
yyforyongyu
force-pushed
the
prep-runtime-2
branch
from
80e77e9 to
5250af9
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds foundational pieces needed for recovery scan-batch support across the db layer and address manager, including scan-horizon extension helpers, backend query primitives to advance account derivation indexes monotonically, and rollback helpers to undo in-memory state when a write transaction is rolled back.
Changes:
- Added db scan-horizon data types and shared horizon-extension logic (
ExtendScanHorizon) to derive/persist recovered addresses and advance branch next-indexes. - Added SQLite/PostgreSQL SQLC queries to monotonically advance
next_external_index/next_internal_indexduring horizon extension. - Added
AddressInfo.HasDerivationPathpopulation andwaddrmgrrollback helpers (RestoreSyncedTo,EvictDerivedAddresses) with unit tests and mock/interface updates.
Reviewed changes
Copilot reviewed 12 out of 18 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| wallet/internal/sql/sqlite/sqlc/querier.go | Extends SQLC Querier interface with next-index advancement methods (SQLite). |
| wallet/internal/sql/sqlite/sqlc/db.go | Prepares/closes new prepared statements and threads them through WithTx (SQLite). |
| wallet/internal/sql/sqlite/sqlc/accounts.sql.go | Adds generated exec methods for AdvanceNext{External,Internal}Index (SQLite). |
| wallet/internal/sql/sqlite/queries/accounts.sql | Adds SQL definitions for monotonic next-index advancement (SQLite). |
| wallet/internal/sql/pg/sqlc/querier.go | Extends SQLC Querier interface with next-index advancement methods (PostgreSQL). |
| wallet/internal/sql/pg/sqlc/db.go | Prepares/closes new prepared statements and threads them through WithTx (PostgreSQL). |
| wallet/internal/sql/pg/sqlc/accounts.sql.go | Adds generated exec methods for AdvanceNext{External,Internal}Index (PostgreSQL). |
| wallet/internal/sql/pg/queries/accounts.sql | Adds SQL definitions for monotonic next-index advancement (PostgreSQL). |
| wallet/internal/db/scan_batch_common.go | Introduces scan-horizon extension scaffolding and derivation loop shared by scan-batch engines. |
| wallet/internal/db/kvdb/addressstore.go | Populates AddressInfo.HasDerivationPath for kvdb-backed address info. |
| wallet/internal/db/data_types.go | Adds scan-batch types (ScanHorizon, ScanBatchParams) and HasDerivationPath; introduces MaxAddressIndex. |
| wallet/internal/db/addresses_common.go | Adjusts address-path conversion to allow imported-xpub HD paths and sets HasDerivationPath for SQL rows. |
| waddrmgr/sync.go | Adds RestoreSyncedTo to revert in-memory synced tip after tx rollback. |
| waddrmgr/scoped_manager.go | Adds EvictDerivedAddresses to undo in-memory derived-address effects after tx rollback. |
| waddrmgr/scoped_manager_test.go | Adds unit tests validating eviction behavior for unlocked/locked managers and multi-account pending derivations. |
| waddrmgr/interface.go | Extends AddrStore/AccountStore interfaces with rollback/eviction methods. |
| bwtest/mock/addr_store.go | Updates test mock to satisfy new AddrStore interface. |
| bwtest/mock/account_store.go | Updates test mock to satisfy new AccountStore interface. |
Files not reviewed (6)
- wallet/internal/sql/pg/sqlc/accounts.sql.go: Language not supported
- wallet/internal/sql/pg/sqlc/db.go: Language not supported
- wallet/internal/sql/pg/sqlc/querier.go: Language not supported
- wallet/internal/sql/sqlite/sqlc/accounts.sql.go: Language not supported
- wallet/internal/sql/sqlite/sqlc/db.go: Language not supported
- wallet/internal/sql/sqlite/sqlc/querier.go: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
yyforyongyu
commented
Jun 10, 2026
Collaborator
Author
|
LGTM ✅ ( |
yyforyongyu
force-pushed
the
prep-runtime-1
branch
from
19ba9fd to
e0df3ca
Compare
yyforyongyu
force-pushed
the
prep-runtime-2
branch
from
5250af9 to
dc68c3c
Compare
yyforyongyu
force-pushed
the
prep-runtime-1
branch
from
e0df3ca to
04092cf
Compare
yyforyongyu
force-pushed
the
prep-runtime-2
branch
from
dc68c3c to
f0f9ee4
Compare
yyforyongyu
force-pushed
the
prep-runtime-1
branch
from
04092cf to
135d4c0
Compare
yyforyongyu
force-pushed
the
prep-runtime-2
branch
from
f0f9ee4 to
b70bd9e
Compare
yyforyongyu
force-pushed
the
prep-runtime-1
branch
from
135d4c0 to
0bb918f
Compare
yyforyongyu
force-pushed
the
prep-runtime-2
branch
from
b70bd9e to
9f3c156
Compare
yyforyongyu
force-pushed
the
prep-runtime-1
branch
from
0bb918f to
fd33e81
Compare
yyforyongyu
force-pushed
the
prep-runtime-2
branch
from
9f3c156 to
b5b24b9
Compare
yyforyongyu
force-pushed
the
prep-runtime-1
branch
from
fd33e81 to
77bb5b0
Compare
yyforyongyu
force-pushed
the
prep-runtime-2
branch
from
b5b24b9 to
fcc3248
Compare
yyforyongyu
force-pushed
the
prep-runtime-1
branch
from
77bb5b0 to
c524831
Compare
yyforyongyu
force-pushed
the
prep-runtime-2
branch
from
fcc3248 to
49492af
Compare
yyforyongyu
force-pushed
the
prep-runtime-1
branch
from
c524831 to
63bd1f8
Compare
yyforyongyu
force-pushed
the
prep-runtime-2
branch
from
49492af to
e0a093b
Compare
yyforyongyu
force-pushed
the
prep-runtime-1
branch
from
63bd1f8 to
a310a1a
Compare
yyforyongyu
force-pushed
the
prep-runtime-2
branch
from
e0a093b to
5cdce95
Compare
yyforyongyu
commented
Jun 22, 2026
yyforyongyu
commented
Jun 22, 2026
yyforyongyu
commented
Jun 22, 2026
yyforyongyu
commented
Jun 22, 2026
yyforyongyu
commented
Jun 22, 2026
yyforyongyu
commented
Jun 22, 2026
Collaborator
Author
|
No issues found by |
Collaborator
Author
|
No issues found by |
Collaborator
Author
|
No issues found by gpt-5.5 🛰️ |
Collaborator
Author
|
No issues found by |
Add the pg and sqlite SQL and generated sqlc for reading the UTXOs that recovery scans should watch. The concrete backend wrappers, db.Store interface exposure, mock, and tests land in the following implementation commit.
Add ListOutputsToWatch to the kvdb UTXO store, walking the legacy wtxmgr OutputsToWatch credits and adapting them into the db-native UtxoInfo view, with a unit test covering a currently-leased credit.
Add WatchOutputFromRow to convert a ListOutputsToWatch result row into the public UtxoInfo shape, reading the watch script from the funding transaction's own output so a partly-owned bare-multisig output is watched by its on-chain script, mirroring the kvdb credit walk.
Add the sqlite ListOutputsToWatch wrapper over the ListOutputsToWatch query, converting each row through WatchOutputFromRow.
Add the postgres ListOutputsToWatch wrapper over the ListOutputsToWatch query, mirroring the sqlite row conversion.
Expose ListOutputsToWatch on the UTXOStore interface now that every backend implements it, and wire the mock store through to its testify expectations.
Add TestListOutputsToWatchBareMultisigUsesOutputScript verifying the recovery watch set reports the on-chain multisig output script for a partly-owned bare-multisig output, not the credited member address's own script, locking in parity with the kvdb backend.
Validate duplicate transaction observations against the stored row shape before treating them as idempotent.
Add TxBatchParams describing a wallet transaction batch and optional sync-tip update applied atomically, before backend implementations consume it. Add the shared batch helpers the backends apply before writing: parents-first topological sorting and a nil-Tx validator that runs before sorting.
Add legacyCreditEntries, converting db-native credit addresses into legacy credit entries with the same output-script membership check CreateTx applies and a nil-credit fallback keyed on the output script, and kvdbTxBlockMeta, converting store block metadata into wtxmgr block metadata. These back the upcoming ApplyTxBatch kvdb implementation.
Add ApplyTxBatch on the kvdb tx store, recording transaction notifications and an optional wallet sync-tip update in one atomic walletdb write. Empty batches return before requiring an address manager. Validate the batch and sort it parents-first before applying it. This matches the SQL backends and prevents child-before-parent ordering from dropping spend edges.
Cover ApplyTxBatch rollback paths that must keep tx state and the live sync-tip cache atomic.
Add the sqlite ApplyTxBatch implementation, advancing the sync tip before creating transactions so a tx confirmed in the synced-to block finds its block row, and skipping transactions that already exist.
Add the postgres ApplyTxBatch implementation, mirroring the sqlite sync-tip-first ordering and duplicate-tx skip.
Expose ApplyTxBatch on the TxStore interface now that every backend implements it, and wire the mock store through to its testify expectations.
Add shared itests verifying ApplyTxBatch stores a tx and advances the sync tip, and confirms a tx in the same block it syncs to.
Add conformance coverage for duplicate transaction idempotence, edge replay, and late-discovered child-spend reconciliation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the scan-batch scaffolding —
db.ScanHorizon/ scan-batch query types,account-name queries, scan-horizon extension helpers, and the
AddressInfo.HasDerivationPathpopulation — together with thewaddrmgrrollback support it builds on (
RestoreSyncedTo,EvictDerivedAddresses).Split out of the runtime Store PR (#1256) for reviewability; content-preserving,
no commit changes. Stacked on
prep-runtime-1.