Skip to content

ci: split relay runtime from polkadot node binary#541

Draft
karolk91 wants to merge 19 commits into
mainfrom
kk-split-node-and-relay-wasm
Draft

ci: split relay runtime from polkadot node binary#541
karolk91 wants to merge 19 commits into
mainfrom
kk-split-node-and-relay-wasm

Conversation

@karolk91
Copy link
Copy Markdown
Collaborator

@karolk91 karolk91 commented May 15, 2026

The polkadot binary normally embeds a Rust-compiled runtime at build time. To get short-epoch behavior for tests, the project built polkadot with --features fast-runtime. This coupled the node SHA to the runtime version and its features — every CI run paid a full polkadot recompile just to obtain the short-epoch runtime.

The PR splits the relay runtime into a separately-cached artifact and hands it to the node via a chain-spec at startup. The node binary becomes runtime-agnostic.

Flow

  1. .github/env exposes two independent version pins: POLKADOT_NODE_VERSION and RELAY_RUNTIME_VERSION.
  2. scripts/get_polkadot_binaries.sh builds each group from one bare clone of polkadot-sdk using per-SHA git worktrees + per-SHA CARGO_TARGET_DIR.
  3. The polkadot-node group builds polkadot with SKIP_WASM_BUILD=1 — no embedded runtime.
  4. The relay-runtime group builds westend-runtime --features fast-runtime and caches westend_runtime.compact.compressed.wasm.
  5. scripts/create_westend_local_spec.sh runs chain-spec-builder against that WASM to emit zombienet/westend-local-spec.json.
  6. Zombienet spawns polkadot --chain westend-local-spec.json — the runtime is loaded from the spec at startup.

Independent caches

Each group's cache key is polkadot-binaries-${runner.os}-<group>-<VERSION_VAR>. Bumping RELAY_RUNTIME_VERSION invalidates only the relay-runtime cache; the node cache stays warm.

Paseo support

Paseo zombienet tests now run against a real paseo-local relay built from paseo-network/runtimes, not against the westend relay as before.

  • New paseo-relay-runtime group in get_polkadot_binaries.sh source-builds paseo-runtime --features fast-runtime from paseo-network/runtimes and caches paseo_runtime.compact.compressed.wasm. Pinned via PASEO_RELAY_RUNTIME_VERSION in .github/env.
  • The pinned commit is ed51b038e2
  • scripts/create_paseo_local_spec.sh mirrors the westend script and emits zombienet/paseo-local-spec.json.
  • zombienet/bulletin-paseo-local.toml sets chain = "paseo-local" and chain_spec_path = "./zombienet/paseo-local-spec.json".
  • justfile zombienet recipes parameterize the relay spec script + path by {{runtime}}; examples/justfile does the same for the integration-test harness (relay + parachain chain-spec paths now branch on the runtime).

Source-build workspace hint

The source-build path now exports WASM_BUILD_WORKSPACE_HINT=<worktree>. Without it, substrate-wasm-builder walks up from OUT_DIR
(under our external CARGO_TARGET_DIR) and lands on the bulletin-chain repo's Cargo.lock, which doesn't contain paseo-runtime
— the build script panics with Failed to find entry for package paseo-runtime. Westend happened to work without the hint because
westend-runtime is reachable as a transitive dep from the bulletin workspace; paseo isn't.

Harness wiring

  • SDK testsRelaychainConfigBuilder uses with_chain_spec_path(...) after the typestate-required with_chain(...). Spec path from RELAY_CHAIN_SPEC_PATH env, default ./zombienet/westend-local-spec.json.
  • TOML zombienetchain_spec_path = "./zombienet/westend-local-spec.json" alongside the existing chain = "westend-local" label.

Both invoke create_westend_local_spec.sh before the test; it's a no-op cache hit when the spec is newer than the WASM.

@karolk91 karolk91 changed the title Kk split node and relay wasm ci: split relay runtime from polkadot node binary May 15, 2026
@karolk91 karolk91 changed the base branch from kk-cache-improvmenets to main May 15, 2026 12:50
@karolk91 karolk91 closed this May 15, 2026
@karolk91 karolk91 reopened this May 15, 2026
@karolk91 karolk91 added the zombienet-sync-tests Trigger zombienet sync tests CI label May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

zombienet-sync-tests Trigger zombienet sync tests CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant