cancel
Showing results for 
Search instead for 
Did you mean: 

Message Content Assertion: Assertion is looking for null fields even though it worked previously

SOLVED
Highlighted
Occasional Contributor

Message Content Assertion: Assertion is looking for null fields even though it worked previously

When I run my call I get the following JSON object:

{
"data": {
"id": 159,
"zoneName": "NameofZone",
"createdAt": "2019-09-11 19:25:31",
"updatedAt": "2019-09-11 19:25:31",
"groupsId": null
},
"status": "success",
"code": "200",
"message": "Operation success",
"version": "2"
}

I setup a Content Assertion on the response to validate that the fields exist:
assertion.PNG

 

I can run this call any number of times back to back and the message content assertion will always pass.

 

However, when I run the entire Test Suite and the same step is run the message content assertion will always fail.  It will give an error as if it were trying to find null nodes in the response (shown below)

failedassertion.PNG

Any idea or insight into why this is happening?  Is there any way to prevent this?  Currently I am removing the contract assertion from the test case but that kind of defeats the purpose of the test.

1 ACCEPTED SOLUTION

Accepted Solutions
Occasional Contributor

Re: Message Content Assertion: Assertion is looking for null fields even though it worked previously

The dev team got back to me today.  The issue is with the URL.  Since this is a call that requires a template parameter for the ID, when the assert is created it is created with the ID in the URL.  When I run the test from the test suite level, a new group is created, thus a new group ID is created which does not match the ID listed in the URL for the content assert.  Below is the direct reply from the team:

Message Content Assertion is based on xml object and it converts all json nodes into xml nodes and declares namespace ns1=endpoint.
When the assertion was created the endpoint was http://{APIURL}/api/zones/165.
If you open XML tab in the response you will see something like <Response xmlns="http://{APIURL}/api/zones/179">. And the namespase for the got response and in the assertion are different, that's why the assertion fails.
The node cannot be found at all

They suggested to substitute the assertion for the JSONPath Existence Match which is more preferable for json responses and independent of endpoint.


View solution in original post

12 REPLIES 12
Community Hero

Re: Message Content Assertion: Assertion is looking for null fields even though it worked previously

See how it behaves if you uncheck the checkbox for the response elements where you may expect null. In this case, "groupId".


Regards,
Rao.
Occasional Contributor

Re: Message Content Assertion: Assertion is looking for null fields even though it worked previously

@nmrao Thank you for your response!  I'm not expecting group ID to be null. I'm expecting all fields to exist and they do, I can see them in the response.  But for some reason the assertion I set changes the fields it expects to exist to null. So even though the response is returning what I want the assertion fails.

 

I reconfigure the assertion to pass then run the request again and it passes.  I run a couple steps before the test step that fails and the failing test step passes.  But then I start the test from the Test Suite level and the test step fails and all of the fields the assertion was expecting are now changed to null.  I attached a screenshot of the failed assertions with the response next to it so you can see that everything that the assertion expects to exist actually does exist in the response.

Occasional Contributor

Re: Message Content Assertion: Assertion is looking for null fields even though it worked previously

failedassertion2.PNG

Community Hero

Re: Message Content Assertion: Assertion is looking for null fields even though it worked previously

Hi @shales,

I tried reproducing your issue but i couldnt. What version of ReadyAPI! are you running?

Its not perfect, but you could replace the Message Content Assertion with individual existence and content match assertions which would give you the same resultant as the message content assertion....its just less efficient and more tedious, more work.

Did i understand correctly....when you execute the test your response generates a payload to assert against but the Message Content Assertion youve setup with existence checks all report as failing despite the fact you can see the attributes in the payload?

That's weird. Have you tried any of the other verification types in the Message Content Assertion like = or not exists?
I'm wondering whether the problem is the exist check alone or whether its other verification points too? Also crossed my mind to ensure the underlyinf logic on the exists and not exists check hasnt been switched un the version youre running so the not exists actually executes an exists check underneath...but thats a sloppy defect so wouldnt expect it.
If you cant find the solution and it sounds like a defect id raise a tech support call with Smartbear, but id try and investigate with the other verification points first to see if theres an identifiable pattern.

Ta

Rich
if this helped answer the post, could you please mark it as 'solved'? Also if you consider whether the title of your post is relevant? Perhaps if the post is solved, it might make sense to update the Subject header field of the post to something more descriptive? This will help people when searching for problems. Ta
Occasional Contributor

Re: Message Content Assertion: Assertion is looking for null fields even though it worked previously

Thanks @richie ,

I think it has something to do with the way the message content assertion is reading the nested object in the response.

 

For instance the groupsId shows another nested null field.  This doesn't occur when I input the raw response into a Json Viewer or in the Outline tab of the response.  Here is the response, but the message content assertion window in the screenshots above show a nested null field from the groupsId field.

 

{
  "data": {
    "id": 163,
    "zoneName": "APIzone",
    "createdAt": "2019-09-13 00:44:42",
    "updatedAt": "2019-09-13 00:44:42",
    "groupsId": null
  },
  "status": "success",
  "code": "200",
  "message": "Operation success",
  "version": "2"
}
Occasional Contributor

Re: Message Content Assertion: Assertion is looking for null fields even though it worked previously

Also I am running version 2.8.0 of ReadyAPI

Occasional Contributor

Re: Message Content Assertion: Assertion is looking for null fields even though it worked previously

Still thinking about this.  Since the content assertion passes when I first configure it and it passes until I run my entire test suite which includes several calls before the one in question.  Those calls also use content assertions but their responses do not contain a nested object.  Is it possible that the content assertion for the failing tests is using cached schemas from the previous runs?  I'm just brainstorming here as I am not entirely sure how the assertions work under the hood.  It is odd that the failing tests will pass when run seperately then fail when the test case is run.  It's important to note that once the assertion fails it will always fail until it is reconfigured to the response.  But before it fails I can run the call itself and return a pass for the assertion, I can run a couple of the calls before the failing call then run the failing call and it will pass.  But once I start the test from the Test Suite level when the tests reach the calls in question they fail for the content assertion.  It seems like the way it parses the message may be causing it but it doesn't make sense as to why it would work in the first place.

Community Hero

Re: Message Content Assertion: Assertion is looking for null fields even though it worked previously

Hey @shales 

 

I'm actually running v2.7.0 and my Message Content Assertion content is slightly different to yours anyway - but I have another issue thats holding me up so I'm gonna need to upgrade to v2.8.0 - so if I get a chance I'll try and replicate what you've got.

 

As far as I'm aware - SoapUI only caches properties and .wsdls, but no assertions - and doesn't cache request response.  However - if you say it works when you've initially configured the Message Content Assertion but stops after so many requests - have you checked to see if you have any caching headers on your response that could be interferring? (just a thought - look for pragma and Content-Cache)

 

Anyway - so you're saying it always fails on the TestSuite execution - but if you run the test individually it executes fine?

Can you disable the tests that are above the test in the testsuite and then execute the testsuite?  I'm just wondering if executing at the testsuite level is a bit of a red herring - if you disable the tests above this - at least this might prove either way if some sort of caching is involved?

Or - move the test with the nested attribute in the Message Content Assertion higher in the testcase sequence in the testsuite?

 

I've previously had test steps that failed when executing the testcase or testsuite - but ran fine individually - I added  delay before the relevant teststep and the assertions that failed no longer did so.  Its a bit rubbish to think that just cos you have a nested object in there's a bit more latency causing the assertion to fail - but it's something to investigate just to confirm this isn't the case.

 

Cheers,

 

rich

 

if this helped answer the post, could you please mark it as 'solved'? Also if you consider whether the title of your post is relevant? Perhaps if the post is solved, it might make sense to update the Subject header field of the post to something more descriptive? This will help people when searching for problems. Ta
Occasional Contributor

Re: Message Content Assertion: Assertion is looking for null fields even though it worked previously

I've sent my project to SmartBear.  They agree that something isn't right.  They are forwarding the issue to their development team.  The thought is that the nested node that appears in the assertion set-up is causing an issue with the parsing.  I will update the thread when I hear back from them.