Build bridges testing framework

[serban300]

The bridges test environments are a bit more complex because we need 2 networks plus a relayer

We should build a minimal framework that can start such an environment and then run the tests. We already have some tests here and some scripting around them: https://github.com/paritytech/polkadot-sdk/tree/master/bridges/zombienet . But it would be good to make everything more modular and more extensible.

[ggwpez]

Just bridges? I think @joepetrowski wanted to have a general parachains testing thing. Dont know what you are aiming for here, but maybe it could both be considered.

[serban300]

Just bridges? I think @joepetrowski wanted to have a general parachains testing thing. Dont know what you are aiming for here, but maybe it could both be considered.

Not sure. For the moment just want to start with some simple scripting in order to be able to spawn a more complex zombienet environment and run some bridge tests. After that we can incrementally improve it and maybe reuse it for parachains.

[joepetrowski]

If you just want Zombienet you should be able to declare a lot with https://github.com/paritytech/parachains-integration-tests. Talk with @NachoPal about it.

Yes for sure we want something more general/extensible too.

[serban300]

If you just want Zombienet you should be able to declare a lot with https://github.com/paritytech/parachains-integration-tests. Talk with @NachoPal about it.

Thanks ! I didn't know about this tool. Will check it out. This might be very useful !

[xlc]

It should be relatively easy to modify Chopsticks to make it support bridges testing. It won't cover everything but should still be useful for high level runtime XCM testing.

[bkontur]

Also it should be able to run those tests:

• locally from the scratch/genesis
• In the CI (Polkadot-sdk and Polkadot-fellows repo)

[serban300]

It should be relatively easy to modify Chopsticks to make it support bridges testing. It won't cover everything but should still be useful for high level runtime XCM testing.

That would be great. We intend to also add Chopsticks tests but I didn't get to look on it yet. Will do that after we have some basic zombienet tests running in the CI

[acatangiu]

Chopstix would also allow us to easily (or easier) add regression tests for any real-world encountered issues since it can run from some real chain state (at custom block number do local fork of real chain).

The questions I have (need research) are:

• how do we define tests? we need to integrate with some DSL. (taking it even further, can we unify Zombienet and Chopsticks DSLs?)
• seeing what Chopsticks is actually running locally and what it is still interrogating real chain (RPC nodes) and change it to minimize RPC calls to real chain nodes (will hit rate limiters of real RPC nodes when we test complex multi-chain setups like bridge)

[NachoPal]

Bridging capabilities were added to xcm-emulator. You can start adding tests there also.

About e2e tests:

• Infra: the ideal would be adding bridging capabilities to Chopsticks. It is faster and lightweight compared to zombienet, plus it is possible to use the live state.
• Interface:
  • https://github.com/paritytech/parachains-integration-tests Tests written in yaml files. It works with zombienet and I have a branch to make it work for Chopsticks
  • https://www.npmjs.com/package/@acala-network/chopsticks-testing You can see it being used in https://github.com/AcalaNetwork/e2e-tests
  • thrid alternative?

More context:

• System Test Suite roadmap#56
• Replace Zombienet with Chopsticks AcalaNetwork/chopsticks#657

[bkchr]

• how do we define tests? we need to integrate with some DSL. (taking it even further, can we unify Zombienet and Chopsticks DSLs?)

We need the same for: polkadot-fellows/runtimes#171

• Tests written in yaml files.

Please no. Can we stop this right from the start? Looking at the examples gives me headaches.

[pepoviola]

Will do that after we have some basic zombienet tests running in the CI

Hi @serban300, we want to start moving some workloads to the next version of zombienet asap. With the new version you can manage multiple networks (and write test directly in rust or interact with the chain through subxt or polkadot.js)and I think we can help to simplify the current setup. Also, at some point we want to add support for easily integrate Chopsticks with zombienet.

Thx!