When *some* API tests fail, then we repeat it (say 3 times) until it succeeds. Currently, we do this by putting a "repeat" script in the tear down script (tab) of every test suite which is prone to failures. It is cumbersome to repeat the script for every problematic suite & update the script.
1 - Is it a good idea to repeat API tests until they pass? 2 - Are there better ways of repeating API tests ? 3 - Why does ready api (version 3.0) not have the option to easily repeat API tests ?
"Some tests fail and repeat three times". I'd argue that either the API is 'buggy' or you have 'flaky' tests. Either way, personally I would not keep hitting 'run' until I got green bars. I'd looking at the API and asking the devs why I didn't not get the expected response or I'd be trying to understand why my test fails and what I should do to resolve.
You should be looking for consistent behaviour in your tests, then when something does fail, you know you have something that needs to be addressed.
The only other scenario i can think of is where the test is failing because of a timing issue. E.g. Im not getting the expected response as I'm waiting for my earlier data to be persisted, so I'll try again. For this, I'd suggest a Delay test step. But again, not ideal and I'd be speaking to the devs if this was the case.
Yes, repeating tests seems like a problem with the dev code, test code or timing. IMO, repeated failures should be rare and hence tests should only be repeated manually. Automated repeating might mask serious problems. I don't know how such problems could be revealed outside of regression/functional testing. They might get revealed during performance testing or not.
BUT, I cannot prevent the repetition of failing tests for now. So, I'd like to know how to repeat tests for the short term.
Thanks @JKolosova . I'll see that example. Btw, it would be nice to have a cookbook for common ready api use cases. For many programming languages, people have written cookbooks for common tasks like getting the current time in a desired time zone, converting DB result sets to csv files, reading excel sheets etc. Looks like your link on samples is a tiny cookbook of sorts. But, it would be nice to declare it as such, add more content to it and update it regularly.