Questions:
1. Are you going to use ReadyAPI tool to test the performance? or LoadUI?
Other pointers:
# Consider what metrics are needed in the performance report, accordingly design your tests. For example, if certain API get call thruput with varying size of the request, similarly put call or delete calls.
# Instead of calling groovy scripts which calls another test cases, use out-of-box test cases which might help you to get right performance data.
# Run the tests in the same subnet to avoid the latencies and better throughput.
# Running tests sequentially, does not give the desired results. The right way should be design the test, and create the load test and run the load test. There should be limited number of tests based on APIs to get right performance data.
# Use commandline tool (such as loadtestrunner) to invoke the perfromance test.
Thanks for the advice Rao.
To answer your question, the plan is to use ReadyAPI (Pro). This is because we have over 200 Functional Test Cases in ReadyAPI we want to leverage.
To refine my goal, what we are trying to do is to establish individual baseline Performance metrics for each of our API endpoints (REST calls). That is, how each endpoint performs (response time) when the system is under minimal Load. The intent is to run these test on a lower environment (not our full blown Load environment) earlier on in our test cycle to identify individual calls that have notably increased response times as compared to the baseline (and work can proceed to address performance issues earlier – i.e. shift left).
The above goal is what led to the requirements to run the Test Cases in sequential order and run each a fixed number of times (e.g. 10). We are not looking to run a standard type of Load test, more of a baseline Performance test.
As for loadtestrunner, the limitation that it can run only one load test (one -n opt) per invocation makes it unsuitable for what we are trying to do (we would need one Scenario with all of our test cases in it – arduous to maintain).
Thanks,
Michael
