v3.0 (bug?) - swagger compliance assertion fails in v3.0 but is ok in v2.8

[JoostDG]

Hi,

I recently update to v3.0 of readyAPI. Looks nice!

However, I notice a difference in behavior on the Swagger Compliance Assertion. Might this be a bug at your end or is this assertion suddenly more strict in the v3.0? And if the latter, why does it flags this a failure?

Scenario:

Applying swagger compliance assertion to a rest response. In the assertion, I have a swagger url which contains a relative path to a yaml file. Strict validation is enabled (disabling this makes no diffrence to this issue). The validation happens and provides now these failures (note: some items are renamed for privacy reasons):

1) Instance failed to match all required schemas (matched only 0 out of 1) checked node: [/references] corresponding schema: [/properties/references]

If this is a valid assertion failure then it is not a clear error message in my opinion. In my JSON response I get back the following for the reference node:

Response of my usedResource:

   "references" : {
      "aReference" : "93269491-e89b-4f19-9d12-ec09b71aeaec",
      "bReference" : null,
      "cReference" : null
   }

Is the error saying that bReference and cReference cannot be null? In my swagger/yaml file reference is defined as: 

  usedResource:
    type: object
    description: Description of the request
    properties:
...
       references:
          readOnly: true
           description: References description
           allOf:
             - $ref: '#/definitions/ourReferences'
...
...
ourReferences:
    type: object
    description: Contains the references
    properties:
      aReference:
        type: string
        description: The reference a 
      bReference:
        type: string
        description: The reference b
      cReference:
        type: string
        description: The reference c

2) Instance type (string) does not match any allowed primitive type (allowed: ["object"]) checked node [/priority] corresponding schema [properties/priority]

In my JSON response I get back the following for the reference node:

Response of my usedResource:

"priority" : "NORMAL"

In my swagger/yaml file priority is defined as: 

usedResource:
    type: object
    description: Description of the request
    properties:
...  
    priority:
        description: The urgency.
        allOf:
          - $ref: '#/definitions/RequestPriority'
...
  RequestPriority:
    type: string
    enum:
      - NORMAL
      - URGENT

I cannot see what's wrong with this. Other similar nodes with similar type & enum are not resulting in a failure. Again, also v2.8 did not flag this as a failed compliance.

I could provide more examples, but I hope these suffice. 

[JoostDG]

Support request ticket logged where I refer to this post : 00413347

[TanyaGorbunova]

Thanks for submitting the support ticket. Please share the solution once you find it together with the support team.

[JoostDG · Accepted solution]

Got a reply from support. Basically, it's not a SmartBear bug but a more strict assertion since 3.0.

Full response from support:

---

Hi Joost,

In ReadyAPI 3.0.0, we updated some Swagger libraries. These libraries had a bug with handling the allOf statements. The issue has been fixed, so now you get a failed assertion in ReadyAPI. Please see the below comments about the assertion messages: 

1. "instance failed to match all required schemas (matched only 0 out of 1) checked node: [/externalRefs] corresponding schema: [/properties/externalRefs]"
The properties in the ExternalReference schema have the string type. But, in the response, some properties have the null value which is not valid. In this case, you need to either correct the definition or the response.

2. "instance type (string) does not match any allowed primitive type (allowed: ["object"]) checked node: [/priority] corresponding schema: [/properties/priority]"
There is an incorrect reference in the enum types. You can't use the allOf statement if you want to refer to the enum type only, because enum is a simple value, and the allOf statement is used for objects. In this case, you need to fix the definition.

3. "instance failed to match all required schemas (matched only 0 out of 1) checked node: [/study] corresponding schema: [/properties/study]"
The studyId property of the Study object is a string, but the response returns an integer value for this property. In this case, you need to either correct the definition or the response.

For more information, see the Swagger 2.0 specification (https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md) and the JSON Schema specification (https://json-schema.org/specification.html).

Also, we have created an improvement to provide more detailed error messages in assertions.

Let me know if you have any additional questions!

Regards,
Anastasia Batazova
Customer Care Engineer

---

[TanyaGorbunova]

Thanks for the update, @JoostDG