[WIP] make SDK great again#555
Draft
karolk91 wants to merge 13 commits into
Draft
Conversation
Adds pipelineStore() to the TypeScript SDK. Drives bulk submission from a single chainHead subscription on the first RPC and broadcasts each signed tx to every configured RPC, gating completion on finalization rather than pool acceptance. Optimisations: * Offline API: metadata is decoded once at init via getOfflineApi rather than on every signTx call. * Fast signer: signed extensions are pre-decoded once; per-tx signing bypasses PAPI's ~100ms decAnyMetadata work and stays under 5ms. * Speculative pre-signing: while one batch broadcasts, the next batch is signed concurrently so the predicted-nonce path is broadcast-only on the next bestBlockChanged. * Mortality period 64 blocks: short eras (8) deadlock the pool when a tx expires before inclusion but blocks new submissions at the same nonce. Per-item latency tracking: each broadcast records its first-broadcast timestamp; bestBlockChanged and finalized handlers attribute the inclusion / finalization observation back to those timestamps. The result exposes count / min / max / mean / p50 / p90 / p99 plus the raw arrays on PipelineResult so callers can build their own distributions. LatencyStats and the rest of the pipeline types are re-exported from sdk/typescript/src/index.ts.
pipeline-stress.ts is a standalone benchmark harness. It authorises a fresh derived submitter, generates the requested payloads, runs pipelineStore, prints the latency summary (p50 / p90 / p99 plus min / max / mean), and writes the full PipelineResult plus raw latency arrays to a JSON file via --output-json. plot-pipeline-results.py reads one or more of those JSON files and produces a 2-panel chart: finalised throughput on the left (tx/s and KB/s) and stacked finalisation latency on the right (each colour band is the additional latency from p50 -> p90 -> p99). --latency inclusion switches the right panel to broadcast -> best-block. The committed pipeline-results.png is a payload sweep against Versi (4 RPCs, single account): | items | size | tx/s | KB/s | final p50 | final p99 | |------:|-------|------:|-----:|----------:|----------:| | 2000 | 1 KB | 30.44 | 30 | 31.0s | 44.7s | | 1000 | 10 KB | 25.69 | 257 | 31.5s | 33.8s | | 500 | 100 KB| 6.79 | 679 | 39.7s | 48.2s | | 500 | 128 KB| 4.92 | 630 | 40.9s | 70.9s | | 250 | 256 KB| 2.60 | 665 | 33.6s | 70.3s | | 150 | 512 KB| 1.48 | 756 | 43.9s | 74.1s | | 100 | 1 MB | 0.67 | 683 | 69.2s | 89.7s | Bandwidth peaks around 512 KB (~756 KB/s) which is where length and count budgets balance. Below 100 KB the count cap (512 tx/block) idles the length budget; above 256 KB length pressure starts trimming the batch. Result JSON files live in sdk/typescript/test/stress/results/ and are gitignored.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TBD