Default Off for "Abort test if an error occurs?"

[fsamwel]

Default any new testcase will have option "Abort test if an error occurs?" checked. This means that if a teststep fails any next teststep in that case will not be executed.

This makes a lot of sense in some cases, for example if one step does a post to add something and the next step does a get to check it, and the third step does a delete. Or when you're testing a non-REST API where first step would log in and second step would do something that needs sessions data.

But many tests would have independent requests. After all, REST API's are stateless, and most methods are idempotent. Test cases then, just like scenarios, are just a way to logically group related requests (e.g. by operation, by quality aspect, by feature).

Is there a way to set the default value for "Abort test if an error occurs?" to off/unchecked rather than on/checked for new testcases?

[nmrao]

Please see below thread if that is helpful.
https://stackoverflow.com/questions/16341547/continue-after-failed-assertion

[fsamwel]

the answer you link to explains how to change this setting for every testcase. What I would like, is to set the default testcase configuration, so I don't have to manually change this setting for every testcase I make. Is there a way to do that?

I would expect something like that because:

1. the out of the box default makes no sense in testing RESTful API's

2. when adding a new testcase in several ways (like from a operation-request or duplicating an existing request (F9) into a new testcase) only the testcase name is asked, and the testcase is created with these default settings.

I didn't find such setting in Preferences, but perhaps there is a way using some Global Properties or other way to change that?

[nmrao]

Hearing this for the first time in the forum someone saying "the out of the box default makes no sense in testing RESTful API's"
Can you please explain what are the steps in your test case (including method names and http methods). Also it helps to understand why those test steps in the test case?

[nmrao]

If think there is something missing, feel free to submit a feature request.

[fsamwel]

Hi nmrao,

Thanks for your support. What I do is mostly test GET requests from an already filled database. So there are no setup steps or something like that.

The same would apply to any test of other operations that would not require a second step to validate (like check validation on POST requests).

That means that one request is one testcase, and one testcase is one request (using testcase in ISTQB or TMap terms, not necessarily in ReadyApi terms). What I thus do is use the testcases (in ReadyApi) to bundle related teststeps/testcases.Otherwise there is no use for having testcases, and checking teststep failure would add another step every time (open the case first and then open the request to see what failed).

For example one ReadyApi project would have a test scenario for each operation (like "GET /pet", "GET /store" and "GET /user") and one test case for each feature (testcases like "parameter validation", "authenticate", "filter with fields", "eager loading with expand"). And then each test case would contain several teststeps.

So before submitting a feature request I first wanted to make sure I've not missed a setting somewhere. 

[nmrao]

At first glance it appears to be fine.

However, i am not sure how the data received in the GET response being validated to say test is passed.

Though REST services are stateless, when someone wants test CRUD operations, then the methods should be created in an order so that the data used in POST can be validated in the GET response. That's why a test case can have multiple steps with different methods say POST, GET, and DELETE. This way, tests are clear and validated the response from the service.

Does it make sense?

[sonya_m]

Great discussion, thank you for clearing things up nmrao!

@fsamwel here's where you can add the Idea.

[fsamwel · Accepted solution]

I found the way to do this. In Preferences I found "Default Test Case Options" with "Abort test if an error occurs". This is what I was looking for.

Thanks nmrao for your help.

[sonya_m]

@fsamwel Yay! Happy to hear you've found what you were looking for. Thank you for sharing.