Ask a Question

Issues and Feedback about Smart Assertions

steve_rivers
New Contributor

Issues and Feedback about Smart Assertions

Some issues and feedback about Smart Assertions after a day or so of using them. 

I would like to provide some hopefully helpful feedback on the newly introduced Smart Assertion.

Please set the default name generated based on the type selected by the user. e.g. Smart Assertion - Metadata or Smart Assertion - Data

 

The smart assertion does not handle simple text responses in the way that message content assertions do. The following response cannot be parsed by the Smart Assertion. Sure it isn't following good API design practice but it is a valid HTTP response and it is valid JSON data.

HTTP/1.1 200 OK
Date: Mon, 19 Jul 2021 21:43:03 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 29
Connection: keep-alive
Server: Kestrel
Cache-Control: no-cache
Pragma: no-cache
Expires: -1

"Your name has been updated."

 

It doesn't seem all that smart to automatically include and enable in the metadata assertions data that is obviously going to change from response to response. Who thought that automatically adding response headers such as Date and Content-Length was a good ides? Date is guaranteed to change with every request. Content-Length is also likely to vary.  This is not going to be an enjoyable experience for the novice user of ReadyAPI - "Use our Smart Assertion on your response, make your testing easier, you'll get a fail the moment you re-run the test". 

 

3 REPLIES 3
nmrao
Community Hero

Re: Issues and Feedback about Smart Assertions

Nice that you have tried and valid inputs.
Hope Smartbear team listens to this feedback.


Regards,
Rao.
sdeevers
Contributor

Re: Issues and Feedback about Smart Assertions

I know I'm a little late to the party on this, but ...

We use HTTP assertions on every test; header exists, header equals (with some regex thrown in the re as well) , status codes, etc.

Additionally 99% of our tests are data driven, so, those values could change on every request.

There are two operations in the Smart Assertion; "Received Data" and "Received Metadata."

After executing one specific test I performed the following

I turned off (deselected active) the "Received Data" as we have groovy code validating the response

I then turned off (deselected active) all "Received Metadata", then activated the specific HTTP data we need to validate. This was performed using conditions (equals, exists, matches regex, etc.) and updating the "valid value" column (some data driven info) as needed.

I then disabled all the "deprecated" assertions we previously used and saved the project (did not commit and push to GIT just yet --- smart move)

I then executed the test to see if everything still passed, which it did.

When I opened the Smart Assertion to see if all of my changes were still in place, the "Received Metadata" page was displayed and showed my changes were still in place (which I was very happy about).

When I changed to "Received Data" I received a popup showing all the content was being updated with the last response, which removed all the work I did above. Not a happy moment any more.

From my perspective once you have "saved" what you want to validate (Active box selection) for both options of the Smart Assertion, the Smart Assertion  should not repopulate unless you choose the "Populate from Received Data" or "Populate from Received Metadata" because everything you previously setup will be removed.

If new HTTP parameters are added in a new release of your code, one would want those to be available to configure but not have to update all previously set items as there is a very good chance one may miss something which was or was not previously configured. In previous releases, the HTTP assertions remained so if something new came along you could easily add the one, two, etc., HTTP assertions without having to reconfigure everything.

 

Please let me know if we are doing something incorrect when configuring the "new" Smart Assertion. Happy to do a demo for the support Staff !

 

Stu

sdeevers
Contributor

Re: Issues and Feedback about Smart Assertions

Guess I was a little quick on the draw ... If you click Cancel after "reviewing" what was configured, what was originally configured will be displayed.

cancel
Showing results for 
Search instead for 
Did you mean: