[draft]: Introduce SQL schema for wallet#1110
Draft
yyforyongyu wants to merge 553 commits into
Draft
Conversation
yyforyongyu
force-pushed
the
sql-wallet
branch
from
72921ef to
8b399ed
Compare
yyforyongyu
force-pushed
the
sql-wallet
branch
2 times, most recently
from
c93d0a1 to
8eade91
Compare
yyforyongyu
force-pushed
the
sql-wallet
branch
3 times, most recently
from
0f68bb3 to
2c7f7a1
Compare
yyforyongyu
force-pushed
the
sql-wallet
branch
2 times, most recently
from
04cfa4b to
d6ffdfb
Compare
yyforyongyu
force-pushed
the
sql-wallet
branch
4 times, most recently
from
823ddd5 to
7bd08f6
Compare
GustavoStingelin
force-pushed
the
sql-wallet
branch
2 times, most recently
from
715a0c0 to
aac7047
Compare
Collaborator
|
just rebased from master |
yyforyongyu
force-pushed
the
sql-wallet
branch
from
aac7047 to
037828b
Compare
Collaborator
Author
|
Pushed a rebase to fix conflicts |
GustavoStingelin
force-pushed
the
sql-wallet
branch
from
037828b to
04b61be
Compare
Pull Request Test Coverage Report for Build 21810355529Details
💛 - Coveralls |
yyforyongyu
force-pushed
the
sql-wallet
branch
from
9175582 to
bf55cd1
Compare
yyforyongyu
force-pushed
the
sql-wallet
branch
from
bf55cd1 to
2092024
Compare
yyforyongyu
force-pushed
the
sql-wallet
branch
3 times, most recently
from
9778f28 to
1a85fc7
Compare
yyforyongyu
force-pushed
the
sql-wallet
branch
3 times, most recently
from
ee70821 to
d3c67d4
Compare
yyforyongyu
force-pushed
the
interface-wallet
branch
from
d3ff891 to
1fc8b3c
Compare
Add spendsWalletOutput and walletOwnedOutputs so a sweep that pays no wallet-owned outputs but spends a wallet output is still recognized as ours. Relevance derives from the parent tx's recorded owned outputs, not current-UTXO membership, so a conflicting no-change re-spend is not misclassified as wallet-unrelated. These feed the recording path wired up next.
Route transaction recording and ownership filtering through db.Store. Owned outputs resolve through Store address lookups; wallet-owned replacements without owned outputs consult the parent tx before CreateTx arbitrates input conflicts. Duplicate rows are handled explicitly: published rows stay idempotent, pending rows are promoted to published, and retained failed/replaced rows return ErrTxRetainedInvalid before broadcast.
Implement InvalidateUnminedTx on the kvdb adapter through the legacy wtxmgr unmined-tx removal path. Tests follow.
Route invalid transaction cleanup through db.Store to keep failed publishes consistent with the store-backed recording and rebroadcast paths. Guard the cleanup with a recorded flag so it only runs when a tx row was actually written. addTxToWallet now returns ([]btcutil.Address, bool, error): recorded is true for both recording paths (the debit-only sweep, which records yet returns no addresses, and the main credited path) and false for a wallet-unrelated tx. Broadcast invalidates only when recorded is true, so a publish failure on a never-recorded tx returns the original error instead of clobbering it with the ErrTxNotFound surfaced by InvalidateUnminedTx.
Add ValidateCreditAddrMembership (and scriptPushesData) so a non-nil CreateTx credit address can be checked against the output it claims to credit before the SQL backends trust it. Membership, not strict equality, keeps bare-multisig members valid while rejecting unrelated pushed pubkeys. Document the resolved-owner Credits contract on CreateTxParams.
Resolve non-nil CreateTx credits through the supplied owner address script (creditLookupScript) rather than the full output script, so the sqlite and postgres backends record wallet-owned bare-multisig member credits; nil-credit callers keep the output-script fallback. The credit is validated against the credited output before insertion.
Add SQL conformance regressions for owner-address credit resolution: TestCreateTxCreditBareMultisigMember records a wallet-owned multisig member, and TestCreateTxCreditAddrMismatch rejects a credit the output does not pay.
Add the ListAddressesByScriptPubKeys SQL query for PostgreSQL and SQLite, plus the generated sqlc bindings. The query returns wallet-owned address rows for a caller-supplied set of script pubkeys.
Add ResolveOwnedAddressesQuery and the backend methods that resolve a supplied script set down to the wallet-owned subset. SQL backends use the new batched sqlc query; kvdb performs the equivalent lookup in one walletdb read transaction.
Add ResolveOwnedAddresses to AddressStore and the wallet store mock. Cover the contract in kvdb unit tests and shared DB integration tests: mixed owned and foreign scripts, duplicate scripts, empty input, and wallet scoping.
filterOwnedAddresses resolved ownership with one store call per unique output address, an N+1. Gather the per-address scripts up front and resolve the whole set with a single ResolveOwnedAddresses call, then classify ownership by presence in the returned map. Add Broadcast-level regressions for the retained-invalid record guard, which drive ownership through the batched path.
Add countingAddrStore and BenchmarkFilterOwnedAddresses, reporting a store_ops/op metric that shows the filter stage stays at a single store operation per transaction as the output count grows (N -> 1).
When mempool acceptance is unavailable, Broadcast records the tx and then calls publishTx directly. publishTx only treated ErrTxAlreadyInMempool as success, so ErrTxAlreadyKnown and ErrTxAlreadyConfirmed bubbled up to Broadcast and invalidated a tx the network had in fact already accepted or confirmed. Mirror checkMempool's already-broadcast handling: treat all three errors as a successful publish so the recorded tx keeps being tracked. All other send errors still return to Broadcast for invalidation.
Add reusable store-backed PSBT fixtures before the routing commits so each PSBT path can assert the new store interactions directly.
Move PSBT UTXO reads through db.Store so SQL-backed wallets can fund and sign PSBTs without direct legacy txstore access.
Route PSBT lease checks through db.Store to match the LeaseOutput migration and avoid wallet-level reads of legacy lock buckets.
Load PSBT parent transactions through db.Store so signing no longer depends on legacy txstore reads for parent output data.
Remove the PSBT legacy credit helper after every PSBT credit and parent lookup has moved to the store-backed path.
Taproot PSBT inputs must use TaprootBip32Derivation without also setting the legacy Bip32Derivation field. The wallet signer rejects both derivation forms on the same input as ambiguous.
ListAccounts is documented to return an empty result when no accounts match. Preserve that contract when a kvdb key scope is absent instead of translating the missing scoped manager into ErrAccountNotFound.
Recovery can extend an account horizon while scanning one block. The same block may then contain later spends to addresses inside the newly extended horizon, so retry detection against the block with a copied spend snapshot before advancing.
When a block row already exists for a height, callers must not silently reuse it for a different hash or timestamp. Re-read the stored row after insert-or-ignore and report ErrBlockMismatch on conflicting metadata.
GetWallet previously left WalletInfo.IsWatchOnly at the zero value for kvdb. Populate it from the legacy address manager so callers see the same wallet-level watch-only state through the db.Store API.
Rollback deletes every block at or above the rollback boundary. Rewind wallet sync references to the greatest stored block below that boundary, because the SQL blocks table is not guaranteed to contain every height.
validateExtendedPubKey dereferenced the key before checking for nil. Return ErrInvalidAccountKey for a nil account key instead of panicking during account validation.
Coin-source switches only matched pointer forms for ScopedAccount and CoinSourceUTXOs. Accept value forms in validation, change-source selection, and UTXO lookup so callers can pass either representation.
Post-rebase fixup that finishes the btcd v2 module upgrade for the sql-wallet feature files that the per-commit rebase carried over with v1 imports/symbols. - Sweep remaining v1 btcd imports to v2 across waddrmgr, wallet, bwtest and the SQL db packages (per scripted-diff path mapping); move the address-family symbols off btcutil onto the address package. - Add SubmitPackage to the bwtest Chain and MempoolChain mocks so they satisfy the updated chain.Interface. - Restore feature deps via go mod tidy (root + wtxmgr); keep the local wtxmgr/txsizes replace directives and pin testcontainers to v0.40.0 to match the wait.ForSQL API the itest is written against.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Open this PR to track rebase conflicts.
TODO: add descriptions and link PRs and issues.