Forum Discussion

steve_rivers's avatar
steve_rivers
Occasional Contributor
4 years ago

Issues and Feedback about Smart Assertions

Some issues and feedback about Smart Assertions after a day or so of using them. 

I would like to provide some hopefully helpful feedback on the newly introduced Smart Assertion.

Please set the default name generated based on the type selected by the user. e.g. Smart Assertion - Metadata or Smart Assertion - Data

 

The smart assertion does not handle simple text responses in the way that message content assertions do. The following response cannot be parsed by the Smart Assertion. Sure it isn't following good API design practice but it is a valid HTTP response and it is valid JSON data.

HTTP/1.1 200 OK
Date: Mon, 19 Jul 2021 21:43:03 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 29
Connection: keep-alive
Server: Kestrel
Cache-Control: no-cache
Pragma: no-cache
Expires: -1

"Your name has been updated."

 

It doesn't seem all that smart to automatically include and enable in the metadata assertions data that is obviously going to change from response to response. Who thought that automatically adding response headers such as Date and Content-Length was a good ides? Date is guaranteed to change with every request. Content-Length is also likely to vary.  This is not going to be an enjoyable experience for the novice user of ReadyAPI - "Use our Smart Assertion on your response, make your testing easier, you'll get a fail the moment you re-run the test". 

 

Assertions
  • sdeevers's avatar
    sdeevers
    Contributor
    4 years ago

    Guess I was a little quick on the draw ... If you click Cancel after "reviewing" what was configured, what was originally configured will be displayed.

    • TwoFang's avatar
      TwoFang
      Contributor
      4 years ago

      The one major issue i have with smart assist is the idea that writing a text trying to inform the user that data will be lost if you swap between the two types is just to me proof that this feature should never have been released in its current state

      If it atleast would have been an annoying pop up confirm dialog so you are forced to read that your work will be deleted would be better and then can select cancel/continue 

      Smart insert simply needs to be split into two asserts one for each tab unless it can be changed so that you can effortless move between the two types without losing data

      • marjor's avatar
        marjor
        New Contributor
        4 years ago

        Metadata and message content assertions should be completely separate assertions because switching the type destroys all the data you have added. How can you modify previously added assertions when all the values are lost? It is the same as not having a modify smart assertion feature at all. 

  • sdeevers's avatar
    sdeevers
    Contributor
    4 years ago

    I know I'm a little late to the party on this, but ...

    We use HTTP assertions on every test; header exists, header equals (with some regex thrown in the re as well) , status codes, etc.

    Additionally 99% of our tests are data driven, so, those values could change on every request.

    There are two operations in the Smart Assertion; "Received Data" and "Received Metadata."

    After executing one specific test I performed the following

    I turned off (deselected active) the "Received Data" as we have groovy code validating the response

    I then turned off (deselected active) all "Received Metadata", then activated the specific HTTP data we need to validate. This was performed using conditions (equals, exists, matches regex, etc.) and updating the "valid value" column (some data driven info) as needed.

    I then disabled all the "deprecated" assertions we previously used and saved the project (did not commit and push to GIT just yet --- smart move)

    I then executed the test to see if everything still passed, which it did.

    When I opened the Smart Assertion to see if all of my changes were still in place, the "Received Metadata" page was displayed and showed my changes were still in place (which I was very happy about).

    When I changed to "Received Data" I received a popup showing all the content was being updated with the last response, which removed all the work I did above. Not a happy moment any more.

    From my perspective once you have "saved" what you want to validate (Active box selection) for both options of the Smart Assertion, the Smart Assertion  should not repopulate unless you choose the "Populate from Received Data" or "Populate from Received Metadata" because everything you previously setup will be removed.

    If new HTTP parameters are added in a new release of your code, one would want those to be available to configure but not have to update all previously set items as there is a very good chance one may miss something which was or was not previously configured. In previous releases, the HTTP assertions remained so if something new came along you could easily add the one, two, etc., HTTP assertions without having to reconfigure everything.

     

    Please let me know if we are doing something incorrect when configuring the "new" Smart Assertion. Happy to do a demo for the support Staff !

     

    Stu

  • nmrao's avatar
    nmrao
    Champion Level 3
    4 years ago
    Nice that you have tried and valid inputs.
    Hope Smartbear team listens to this feedback.