[Testing] Testing framework

[franciscoaguirre]

We have a lot of ad-hoc tests floating around. I would like to make them more consistent so it's harder for bugs to fall through the cracks. The most recent bug we didn't manage to catch was the one solved by #574.

This issue aims to create a comprehensive and consistent testing framework that:

• Makes it easy to consistently check that the data we store can also be retrieved and is the same
• Makes it easy to test a full lifecycle of authorize, store, renew / hop submit, promote
• Is clear with when to use typescript vs zombienet-sdk
• Removes unnecessary boilerplate (looking at you Justfile)

Current state

Today we have:

1. Pallet unit tests

2. Runtime tests

3. Chain e2e tests with zombienet-sdk

4. Client e2e tests in examples/*.js

5. UI tests for console-ui

6. 2. and 5. are good. The purpose of this issue is to simplify how much needs to run in zombienet-sdk, only node-level issues like syncing, and move the client e2e tests from examples to an easier-to-use and more consistent typescript framework. The existing examples are better of as actual examples and not tests.

Harness sketch

test('hop promotion preserves data integrity', async ({ net }) => {
  const alice = await net.actor('//Alice', { authorize: { bytes: 10 * MiB, transactions: 100 } });
  const data = randomBytes(2 * MiB);

  const { cid } = await alice.hop.submitAndAwaitPromotion(data);

  await expect(net.ipfs).toRetrieve(cid, data); // enforced terminal assertion
});

Here:

• net spawns the network and exposes the client to interact with it. Would also allow setting a shorter retention period / authorization period
• actor(suri, opts) manages the signer and the authorizations
• submitAndAwaitPromotion is one of many possible wait helpers for making sure nothing breaks in the middle of the lifecycle
• expect(net.ipfs).toRetrieve(cid, bytes) fetch via kubo and compare bytes to make sure the data can be retrieved and is the same as the one submitted

Tasks

Phase 1 - Framework

• Scaffold
• Network fixture
• Actor and wait helpers
• Chopsticks fixture
• CI workflow
• HOP test to prove the framework

Phase 2 - More tests

• HOP suite
• Storage lifecycle suite
• XCM authorization e2e
• Everything test
• Auto-renew suite
• Connect with console-ui tests

Phase 3 - Cleanup

• Demote examples to documentation, everything should already be tested by the framework
• Clean up Justfile
• Move zombienet-sdk tests to the framework
• Clean up CI workflows