You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Running plutip tests take ages and lengthens the feedback loop.
It is proven to be possible to execute tests in parallel, however currently it require forking CTL and altering some parts of the framework.
Testing framework
purescript-spec is a good candidate:
It is very similar to mote, and it is able to run test cases in parallel, with flexible chunking.
However it's SpecT is a bit less general than MoteT and require altering, but that's not a big issue, and a small contribution to the library can easily fix the issue.
Parallelizing
Each testing instance should have it's own PlutipConfig, with non duplicating ports. This way we ensure they don't interfere.
Kupo workdirs will not interfere as Kupo's workdir contain a randomly generated number.
Ogmios and Plutip workirs interference have not proven to exist.
Importantly, graceful shutdown must be properly implemented so no matter what all temp directories must be cleaned up, and all the services must be shut down after tests.
Design
It is probably not the best idea to left the implementation in the CTL itself, but instead put it in the separate library.
To leave it CTL:
testnet cluster creation and running tools
contract for redistributing funds between tests
To separate:
MoteT/SpecT manipulation, creation, configuring and running a test tree.
Potential problems
Decreased cardano-node/ogmios/kupo performance on multiple parallel instances
Accidental errors will occur more often because of the CPU, Mem and disk usage.
Some operations may take longer to execute, and it should be concerned.
So for epoch-dependent (the ones that should be finished in a certain amount of epochs) operations increasing amount of parallel instances could require increasing the slot or epoch length.
Setting up tests to run in parallel may be complicated
This is not a big issue because
Parallel testing is optional
It is still better than enforcing every project that need this feature to alter part of the CTL codebase on every project and implement it themselves
The text was updated successfully, but these errors were encountered:
Summary
Running plutip tests take ages and lengthens the feedback loop.
It is proven to be possible to execute tests in parallel, however currently it require forking CTL and altering some parts of the framework.
Testing framework
purescript-spec
is a good candidate:It is very similar to
mote
, and it is able to run test cases in parallel, with flexible chunking.However it's
SpecT
is a bit less general thanMoteT
and require altering, but that's not a big issue, and a small contribution to the library can easily fix the issue.Parallelizing
Each testing instance should have it's own PlutipConfig, with non duplicating ports. This way we ensure they don't interfere.
Kupo workdirs will not interfere as Kupo's workdir contain a randomly generated number.
Ogmios and Plutip workirs interference have not proven to exist.
Importantly, graceful shutdown must be properly implemented so no matter what all temp directories must be cleaned up, and all the services must be shut down after tests.
Design
It is probably not the best idea to left the implementation in the CTL itself, but instead put it in the separate library.
To leave it CTL:
To separate:
Potential problems
Decreased cardano-node/ogmios/kupo performance on multiple parallel instances
Accidental errors will occur more often because of the CPU, Mem and disk usage.
Some operations may take longer to execute, and it should be concerned.
So for epoch-dependent (the ones that should be finished in a certain amount of epochs) operations increasing amount of parallel instances could require increasing the slot or epoch length.
Setting up tests to run in parallel may be complicated
This is not a big issue because
The text was updated successfully, but these errors were encountered: