ci: split relay runtime from polkadot node binary#541
Draft
karolk91 wants to merge 19 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
polkadotbinary normally embeds a Rust-compiled runtime at build time. To get short-epoch behavior for tests, the project built polkadot with--features fast-runtime. This coupled the node SHA to the runtime version and its features — every CI run paid a full polkadot recompile just to obtain the short-epoch runtime.The PR splits the relay runtime into a separately-cached artifact and hands it to the node via a chain-spec at startup. The node binary becomes runtime-agnostic.
Flow
.github/envexposes two independent version pins:POLKADOT_NODE_VERSIONandRELAY_RUNTIME_VERSION.scripts/get_polkadot_binaries.shbuilds each group from one bare clone ofpolkadot-sdkusing per-SHA git worktrees + per-SHACARGO_TARGET_DIR.polkadot-nodegroup buildspolkadotwithSKIP_WASM_BUILD=1— no embedded runtime.relay-runtimegroup buildswestend-runtime --features fast-runtimeand cacheswestend_runtime.compact.compressed.wasm.scripts/create_westend_local_spec.shrunschain-spec-builderagainst that WASM to emitzombienet/westend-local-spec.json.polkadot --chain westend-local-spec.json— the runtime is loaded from the spec at startup.Independent caches
Each group's cache key is
polkadot-binaries-${runner.os}-<group>-<VERSION_VAR>. BumpingRELAY_RUNTIME_VERSIONinvalidates only the relay-runtime cache; the node cache stays warm.Paseo support
Paseo zombienet tests now run against a real paseo-local relay built from
paseo-network/runtimes, not against the westend relay as before.paseo-relay-runtimegroup inget_polkadot_binaries.shsource-buildspaseo-runtime --features fast-runtimefrompaseo-network/runtimesand cachespaseo_runtime.compact.compressed.wasm. Pinned viaPASEO_RELAY_RUNTIME_VERSIONin.github/env.ed51b038e2scripts/create_paseo_local_spec.shmirrors the westend script and emitszombienet/paseo-local-spec.json.zombienet/bulletin-paseo-local.tomlsetschain = "paseo-local"andchain_spec_path = "./zombienet/paseo-local-spec.json".justfilezombienet recipes parameterize the relay spec script + path by{{runtime}};examples/justfiledoes the same for the integration-test harness (relay + parachain chain-spec paths now branch on the runtime).Source-build workspace hint
The source-build path now exports
WASM_BUILD_WORKSPACE_HINT=<worktree>. Without it, substrate-wasm-builder walks up fromOUT_DIR(under our external
CARGO_TARGET_DIR) and lands on the bulletin-chain repo'sCargo.lock, which doesn't containpaseo-runtime— the build script panics with Failed to find entry for package paseo-runtime. Westend happened to work without the hint because
westend-runtimeis reachable as a transitive dep from the bulletin workspace; paseo isn't.Harness wiring
RelaychainConfigBuilderuseswith_chain_spec_path(...)after the typestate-requiredwith_chain(...). Spec path fromRELAY_CHAIN_SPEC_PATHenv, default./zombienet/westend-local-spec.json.chain_spec_path = "./zombienet/westend-local-spec.json"alongside the existingchain = "westend-local"label.Both invoke
create_westend_local_spec.shbefore the test; it's a no-op cache hit when the spec is newer than the WASM.