Use GRANDPA hard forks as warp sync checkpoints

[UnArbosFive]

Summary

This changes GRANDPA warp-proof generation so configured, finalized AuthoritySetHardFork entries can serve as trusted checkpoints. A proof can begin again from a checkpoint and then include the ordinary authority-set transitions recorded after it.

The proof wire format is unchanged. This only changes which existing WarpSyncFragments a provider selects and serves.

Why testnet warp sync currently stalls

The affected testnet history contains GRANDPA authority transitions that cannot be replayed as a continuous proof from set 0. In particular, a historical delayed transition does not have the stored justification needed to construct a normal warp-sync fragment. Nodes that have already warp-synced may also retain authority-change history whose first available record has a nonzero set ID, plus an obsolete lower-set record after the usable checkpoint.

Before this change, WarpSyncProof::generate always called AuthoritySetChanges::iter_from(begin). That API deliberately rejects an incomplete historical prefix: if the first retained record is not set 0, it returns None, which becomes Error::MissingData. It also has no knowledge of configured GRANDPA hard forks. Consequently, a testnet proof provider cannot bridge the unprovable historical transition even though both sides are configured with a later trusted authority-set checkpoint. A fresh node remains in the Downloading warp proofs phase because it cannot obtain a complete proof chain from its peers.

Why the change is needed in the SDK

Subtensor owns the chain-specific checkpoint hashes, set IDs, and authority lists, and supplies them through the existing HardForks configuration. The generic proof-selection logic, however, lives inside sc-consensus-grandpa and has private access to the backend, finalized chain, stored authority-set changes, proof-size limits, and WarpSyncFragment construction.

Subtensor can configure a checkpoint, but it cannot make the SDK generator use that checkpoint without duplicating GRANDPA's internal proof-generation implementation. Teaching the SDK's existing generator how to consume its existing hard-fork configuration keeps chain-specific data out of Polkadot SDK while preserving one implementation of the protocol.

Implementation

• Pass the provider's HardForks configuration into WarpSyncProof::generate.
• Select only configured checkpoints that are finalized and whose exact (hash, number) is canonical in the provider's finalized chain.
• Emit canonical checkpoints after the request's starting block, ordered by height.
• After the last usable checkpoint, append recorded authority transitions whose set ID and block number are both later than the trusted checkpoint.
• Permit this post-checkpoint suffix to have an incomplete pre-checkpoint history, because the checkpoint supplies the missing authority-set trust anchor.
• Fall back to the existing iter_from behavior when there is no usable finalized checkpoint.

Canonical-hash filtering is important because hard forks are keyed by both hash and height: alternate-fork entries at the same height must not influence proof generation. Verification remains fail-closed and continues to validate the exact checkpoint hash, GRANDPA justification, authority list, and subsequent scheduled transitions.

Compatibility and mainnet safety

• No SCALE type or network wire-format change.
• No runtime or on-chain consensus-rule change; this is node-side warp-proof generation.
• Empty hard-fork maps retain the previous generator behavior.
• HardForks::new_initial_set_id, which is the existing Subtensor mainnet mode, retains the previous generator path. The regression test requires its generated proof bytes to be identical to the legacy path.
• Chain-specific testnet checkpoints remain in Subtensor and are enabled there only for the exact testnet genesis hash.

Verification

• SKIP_WASM_BUILD=1 cargo clippy -p sc-consensus-grandpa --all-targets -- -D warnings
• SKIP_WASM_BUILD=1 cargo test -p sc-consensus-grandpa warp_sync_proof_generate_verify --lib
• Combined Subtensor build and all-target Clippy against this SDK revision.
• Live testnet exercise: a fresh patched client requested a proof from a patched archive provider, verified through GRANDPA set ID 4, advanced past Downloading warp proofs, completed state sync, and caught up to the testnet head.

The regression coverage includes:

• an incomplete history beginning at a nonzero set ID;
• an obsolete lower-set record after the checkpoint;
• later valid authority transitions;
• an unfinalized future checkpoint, which must be ignored;
• a noncanonical checkpoint at the same height, which must be ignored;
• byte-identical proof generation for the mainnet-style initial-set-ID mode.

Rollout order

1. Merge this SDK PR into polkadot-stable2506-2-otf-patches.
2. Update Subtensor to the resulting merged SDK revision and merge the dependent Subtensor PR. Avoid pinning the production Subtensor branch to an unmerged feature-head commit.
3. Deploy the updated Subtensor binary to testnet archive/RPC nodes that serve warp proofs.
4. Roll out the updated binary to fresh or syncing testnet clients after providers are available.

The SDK merge alone changes no deployed network behavior. Provider-first deployment prevents updated clients from depending on peers that still use the old generator. Mainnet does not receive the testnet checkpoint configuration and continues using its existing initial-set-ID path.