kvdb: implement the ApplyScanBatch scan engine by yyforyongyu · Pull Request #1256 · btcsuite/btcwallet

yyforyongyu · 2026-06-03T10:33:19Z

Implements the ApplyScanBatch scan engine across kvdb and SQLite/PostgreSQL
behind the db.Store interface, with the scan / tx-batch unit tests + itests and
the RollbackToBlock sync-tip atomicity.

The runtime query/batch methods and scan scaffolding this builds on were split
out into prep-runtime-1 + prep-runtime-2 for reviewability. Stacked on
prep-runtime-2.

coveralls · 2026-06-03T10:41:28Z

Coverage Report for CI Build 28448439256

Warning

No base build found for commit a0e618a on prep-runtime-3.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 55.49%

Details

Patch coverage: 156 uncovered changes across 6 files (330 of 486 lines covered, 67.9%).

Uncovered Changes

File	Changed	Covered	%
wallet/internal/db/kvdb/txstore.go	191	123	64.4%
wallet/internal/db/pg/txstore_applyscanbatch.go	63	40	63.49%
wallet/internal/db/sqlite/txstore_applyscanbatch.go	61	40	65.57%
wallet/internal/db/pg/txstore_batch.go	73	54	73.97%
wallet/internal/db/sqlite/txstore_batch.go	73	54	73.97%
wallet/internal/bwtest/mock/store.go	6	0	0.0%
Total (7 files)	486	330	67.9%

Coverage Regressions

Requires a base build to compare against. How to fix this →

Coverage Stats


Relevant Lines:	58045
Covered Lines:	32209
Line Coverage:	55.49%
Coverage Strength:	13349.83 hits per line

💛 - Coveralls

yyforyongyu

LGTM (deepseek-v4-pro)

Copilot

Pull request overview

Implements additional runtime and scan-batch Store surface area for the legacy kvdb backend by adapting operations onto existing waddrmgr and wtxmgr paths, along with adding the associated request/parameter types and tests.

Changes:

Add kvdb.Store.ListSyncedBlocks and a shared helper for converting db.Block to legacy waddrmgr.BlockStamp.
Add kvdb implementations for recovery/runtime UTXO helpers (ListOutputsToWatch, DeleteExpiredLeases).
Add batch-oriented tx write paths (ApplyTxBatch, ApplyScanBatch) plus RollbackToBlock, with new unit tests and new db parameter types.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
wallet/internal/db/kvdb/walletstore.go	Adds synced-block listing and a block conversion helper for legacy sync/scan paths.
wallet/internal/db/kvdb/walletstore_test.go	Adds unit test coverage for `ListSyncedBlocks`.
wallet/internal/db/kvdb/utxostore.go	Implements outputs-to-watch listing and expired-lease deletion via legacy `wtxmgr`.
wallet/internal/db/kvdb/utxostore_test.go	Adds tests for expired lease deletion and watch-list behavior with leased credits.
wallet/internal/db/kvdb/txstore.go	Implements tx/scan batch application and rollback via legacy managers.
wallet/internal/db/kvdb/txstore_test.go	Adds tests for tx batch, scan batch, and rollback behavior.
wallet/internal/db/data_types.go	Introduces query/params types for synced-block listing and batch apply operations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

yyforyongyu · 2026-06-03T12:28:45Z

LGTM 🐳 (gemini-3.5-flash)

yyforyongyu · 2026-06-03T12:29:11Z

LGTM 🧱
(claude-opus-4-8[1m])

yyforyongyu

LGTM (deepseek-v4-pro)

yyforyongyu

LGTM (deepseek-v4-pro)

yyforyongyu · 2026-06-04T08:00:35Z

LGTM 🪨
(claude-opus-4-8[1m])

yyforyongyu · 2026-06-05T17:45:38Z

Branch hygiene: this PR was updated via merge this round rather than rebased — its range carries 2 Merge branch 'impl-psbt-store' into 'impl-runtime-store' commits. Prior rounds it was a clean linear rebase, and the project preserves commit history (no squash), so these merge commits would land on the target branch.

Please rebase onto the base to drop the Merge branch … commits and keep the stack linear. (See the 1258 thread — the same merge-based update also produced a duplicated stop precreating commit there.) The code changes themselves look correct.

(claude-opus-4-8[1m])

yyforyongyu · 2026-06-19T08:58:38Z

No issues found by gpt-5.5 🦉

yyforyongyu · 2026-06-19T19:32:07Z

No issues found by claude-opus-4-8[1m] ⚙️

yyforyongyu · 2026-06-22T09:42:35Z

No issues found by claude-opus-4-8[1m] ✅

yyforyongyu · 2026-06-23T11:40:32Z

No issues found by gpt-5.5 🧩

yyforyongyu · 2026-06-23T11:41:07Z

No issues found by claude-opus-4-8[1m] ✅

yyforyongyu · 2026-06-23T12:01:00Z

No issues found by gpt-5.5 xhigh ✅

yyforyongyu · 2026-06-28T01:19:10Z

No issues found by claude-opus-4-8[1m] 🧱

yyforyongyu · 2026-06-28T01:44:52Z

No issues found by gpt-5.5 🐙

Add the kvdb scan-horizon machinery: classifyLegacyHorizonBranch and resolveLegacyHorizonAccount validate a horizon and resolve its real account by scope-unique name, applyLegacyScanHorizons extends each branch through the legacy ExtendAddresses path and records a scanHorizonRollback range, and applyLegacySyncedBlocks connects discovered blocks.

Add ApplyScanBatch, extending horizons before recording the tx batch so freshly derived credit addresses are persisted first, then connecting discovered blocks, all in one walletdb write. On failure rollbackScanBatchCaches evicts the derived addresses, invalidates the account caches, and restores the synced tip so no rolled-back in-memory advance stays observable.

Add CanSkipCreateTxDuplicate, reporting whether a batch insert may treat an ErrTxAlreadyExists result as an idempotent no-op: only an exact current-shape duplicate (same live status and block) is skippable, while terminal history rows must not be skipped.

Refactor the sqlite ApplyTxBatch tx loop into applyBatchTransaction and canSkipBatchDuplicate so the upcoming ApplyScanBatch shares one duplicate-tolerant tx writer, treating an exact-shape duplicate as an idempotent retry via CanSkipCreateTxDuplicate.

Refactor the postgres ApplyTxBatch tx loop into applyBatchTransaction and canSkipBatchDuplicate, mirroring the sqlite shared duplicate-tolerant tx writer used by ApplyScanBatch.

Add the SQLite ApplyScanBatch, extending every reported horizon, then connecting discovered synced blocks before recording transactions (so a tx confirmed in a discovered block finds its block row), all in one write transaction via applyBatchSyncedBlocks and applyBatchTransaction.

Add the postgres ApplyScanBatch, mirroring the sqlite ordering: extend horizons, connect discovered synced blocks before recording transactions, all in one write transaction via applyBatchSyncedBlocks and applyBatchTransaction.

Expose ApplyScanBatch on the TxStore interface now that the kvdb and SQL backends implement it, and wire the mock store through to its testify expectations.

yyforyongyu requested a review from Copilot June 3, 2026 12:21

Copilot started reviewing on behalf of yyforyongyu June 3, 2026 12:21 View session

yyforyongyu self-assigned this Jun 3, 2026

yyforyongyu added this to the Introduce SQL store milestone Jun 3, 2026

yyforyongyu commented Jun 3, 2026

View reviewed changes

Copilot AI reviewed Jun 3, 2026

View reviewed changes

Comment thread wallet/internal/db/kvdb/walletstore.go Outdated

Comment thread wallet/internal/db/kvdb/walletstore.go Outdated

Comment thread wallet/internal/db/kvdb/txstore.go Outdated

yyforyongyu commented Jun 3, 2026

View reviewed changes

Comment thread wallet/internal/db/kvdb/txstore.go Outdated

saubyk added this to lnd v0.22 Jun 3, 2026

github-project-automation Bot moved this to Backlog in lnd v0.22 Jun 3, 2026

yyforyongyu force-pushed the impl-runtime-store branch from c5d47e5 to 49471d8 Compare June 3, 2026 14:43

yyforyongyu force-pushed the impl-psbt-store branch from 077faba to 0802858 Compare June 4, 2026 07:08

yyforyongyu force-pushed the impl-runtime-store branch from 49471d8 to 50a31f0 Compare June 4, 2026 07:08

yyforyongyu commented Jun 4, 2026

View reviewed changes

yyforyongyu force-pushed the impl-psbt-store branch from 0802858 to 6012ecd Compare June 4, 2026 08:17

yyforyongyu force-pushed the impl-runtime-store branch from 50a31f0 to 1232175 Compare June 4, 2026 08:20

yyforyongyu commented Jun 4, 2026

View reviewed changes

Comment thread wallet/internal/db/kvdb/txstore.go Outdated

yyforyongyu force-pushed the impl-psbt-store branch from 6012ecd to 861974a Compare June 5, 2026 00:09

yyforyongyu force-pushed the impl-runtime-store branch from 1232175 to 56aa683 Compare June 5, 2026 00:09

yyforyongyu force-pushed the impl-psbt-store branch from 861974a to 7adbbb2 Compare June 5, 2026 12:12

yyforyongyu force-pushed the impl-runtime-store branch from 56aa683 to 9b2ec64 Compare June 5, 2026 12:12

yyforyongyu force-pushed the impl-psbt-store branch from 7adbbb2 to aeb6c40 Compare June 5, 2026 14:22

yyforyongyu force-pushed the impl-runtime-store branch from 9b2ec64 to a56abdf Compare June 5, 2026 14:22

yyforyongyu commented Jun 5, 2026

View reviewed changes

Comment thread wallet/internal/db/kvdb/txstore.go

yyforyongyu force-pushed the impl-psbt-store branch from 7e7f10b to 15cbd1a Compare June 5, 2026 18:05