feat: live sync witness cache oracle#759
Conversation
0xOsiris
changed the title
Benchmark ResultsCompared on the same runner in the same workflow run.
|
…com/worldcoin/world-chain into osiris/live-preimage-witness-oracle
0xOsiris
requested review from
0xForerunner,
Dzejkop,
Kemperino,
alessandromazza98,
cichaczem,
karankurbur,
kilianglas,
murph and
piohei
as code owners
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 13239c3. Configure here.
…com/worldcoin/world-chain into osiris/live-preimage-witness-oracle
alessandromazza98
left a comment
alessandromazza98
left a comment
There was a problem hiding this comment.
generally lgtm. I left some nits that you can address before merging
| @@ -0,0 +1,3 @@ | |||
| [toolchain] | |||
There was a problem hiding this comment.
why adding this? I'd remove it to be fair, in particular because the nightly changes every day, therefore it's high likely that we don't have this version anymore
There was a problem hiding this comment.
if we want a toolchain, then I can be down to pin to a stable version of the rust compiler, not a nightly
There was a problem hiding this comment.
I have not been able to find a way to implement this without min_specialization. We need a monomorphization of MaybeWitness over a generic StateDB because of Reth's trait bounds over the BlockExecutor, BlockBuilder, and BlockExecutorFactory.
There is no way to create a blanket implementation of MaybeWitness over T: StateDB with a granular implementation over T: StateDB = State<DB>.
There is also no way to easily capture the witness outside of the executors internals as we need to capture this from engine_newPayload execution.
| @@ -0,0 +1,222 @@ | |||
| # Live Pre-image Witness Oracle — Design | |||
|
|
|||
| Linear: [PROTO-4718 — witness data oracle: design](https://linear.app/worldcoin/issue/PROTO-4718/witness-data-oracle-design) | |||
There was a problem hiding this comment.
nit: I'd remove this. The linear ticket doesn't include any detail that is not already well documented in this readme
| - `StateProofProvider::witness(input, hashed_state, mode)` (`storage-api/src/trie.rs`) produces the | ||
| trie-node `state` set by walking pre-state cursors for the touched hashed keys (no execution). | ||
|
|
||
| ## Implementation plan |
There was a problem hiding this comment.
you can remove the implementation plan now. I'd leave this document as a readme to explain why we build this live preimage oracle and its architecture. The impl plan is useless now that the code is already finalized
| } | ||
|
|
||
| impl<T> MaybeWitness for T { | ||
| default fn witness(&self) -> Option<ExecutionWitnessRecord> { |
There was a problem hiding this comment.
instead of this impl box, you can write the default impl inside the trait itself
There was a problem hiding this comment.
instead of this impl box, you can write the default impl inside the trait itself
Yeah this is very weird tbh aha
2 participants
PROTO-4718
Creates an optional live witness cache that accumulates pre-image witnesses into a fixed size in memory circular ring buffer. The witnesses can then be fetched over a range externally with enormous compute savings.
As historical witnesses require unwinding the database to the beginning of the range. Witnesses are deterministic based on execution (and the last proven state), and can be easily cached during live sync in the Block Executor minimizing both compute, and complexity of the pre-image witness collection in the proof system.
Note
Medium Risk
Touches block-import execution via an executor wrapper on the engine validation path; default-off limits blast radius, but enabled mode adds async witness assembly and a new debug RPC surface on proof nodes.
Overview
Introduces an optional live pre-image witness oracle for proof workflows: when
--witness.collectis set, the node captures witnesses during normalnewPayloadimport instead of relying on repeateddebug_executePayloadre-execution.New
world-chain-witnesscrate definesBlockWitness/RangeWitnessand a boundedWitnessCache(depth from--witness.depth, default 1024).EVM layer:
WorldChainEvmConfigis no longer a type alias—it wrapsOpEvmConfigwithWorldChainBlockExecutorFactory/WorldChainBlockExecutor, which snapshotExecutionWitnessRecordfrom the live revmStateon executorfinishand sendBlockExecutionWitnessover a channel. A background collector assembles fullBlockWitnessvalues (parent state trie, headers, RLP header/txs) into the cache. With collection off, behavior stays a passthrough (no sender). Requires#![feature(min_specialization)]for theStateDB witness hook.Wiring: CLI
WitnessArgs, node context creates channel + cache when enabled,WorldChainExecutorBuilderarms the sender, and add-ons spawn the collector and registerdebug_collectRangeWitnesson thedebugmodule when collection is on.Adds design doc
docs/proof/live-preimage-witness-oracle.md. Kona host pre-seeding (design Phase 5) is not included in this diff.Reviewed by Cursor Bugbot for commit 13239c3. Configure here.