After updating to NLW our team has begun automating small, nightly performance tests. Next on our list is automating full size performance tests. However, we would like the ability to automatically stop a running test if the test meets a certain threshold - whether it be error count, rate, or duration.
After discussing this issue with other teams, we understand there is no official solution, but have heard two possible ideas:
1. Create a loop around each userpath and dedicate a variable to keeping track of errors. If that variable breaks a limit, stop the iteration of that userpath.
This solution requires changing all existing projects and does not scale.
2. Create an application utilizing Neoload's API to monitor running tests on a machine, use NLW's API to check the live data for running tests, and then use the Runtime API to stop any test that reaches its threshold.
This solution seems the best but requires a significant amount of time and development.
Are there any other ideas or plans for this issue?
Thanks,
Adam