Data driven testing
My co-worker posted this question at: http://www.eviware.com/forums/index.php?topic=2442.0
Since then, I found out we had to login with the account provided when we purchased SoapUI.
The question is:
We have a very complex SOAP web service. It provides CRUD operations to several objects in our server database.
We have ~approximately 12 objects which we refer to as 'primary objects' that can be passed into these CRUD operations. A few examples are contacts and organizations.
Each primary object can have multiple sub-objects and multiple elements/attributes.
We have an operation in our WSDL that will return metadata which describes each object and element exposed in our web service. An example of the metadata for an object would be the object name, what it inherits from, a list elements/attributes. The metadata for an element/attribute contains the attribute name, data type, (base types or complex types), nullability, data range and usage for each CRUD operation, etc.
In total, we have 60+ objects and 500 + fields.
Some of our test cases include attempting a CRUD operation on a primary object with valid and invalid elements based on the usage from the metadata.
The question we have is how can we drive our testing based on this metadata?
We can live with not having metadata retrieved at runtime by dumping the metadata before executing the test suite and using that dump as a data source.
How can we do multiple iterations through the data source which will drive composition of the SOAP requests?
An example the metadata will contact a contact the contact may contain 40-50 fields. Let's say 5 of those fields are required on create, 5 of the fields are not allowed on create and the rest are allowed but not required on create. We want to execute a test request where only the 5 required fields are specified, and verify we receive a valid response. We also want to test we can specify all required and allowed fields and receive a valid response. Then, we will want to test error conditions where we build 5 requests, each one omitting one of the required fields and verify that we receive a SOAP fault.
Additionally, we want to build 5 requests that specify the required fields and each one will specify a not allowed field and verify that we receive a SOAP fault.
We would want to do similar testing with the read, update, and destroy operations. We would also want to use a similar concept for validating field data ranges, etc.
We believe the only effective way to accomplish this and have a maintainable test suite is through a data-driven approach where the data is from metadata.
Is this possible in SOAPUI.?
Since then, I found out we had to login with the account provided when we purchased SoapUI.
The question is:
We have a very complex SOAP web service. It provides CRUD operations to several objects in our server database.
We have ~approximately 12 objects which we refer to as 'primary objects' that can be passed into these CRUD operations. A few examples are contacts and organizations.
Each primary object can have multiple sub-objects and multiple elements/attributes.
We have an operation in our WSDL that will return metadata which describes each object and element exposed in our web service. An example of the metadata for an object would be the object name, what it inherits from, a list elements/attributes. The metadata for an element/attribute contains the attribute name, data type, (base types or complex types), nullability, data range and usage for each CRUD operation, etc.
In total, we have 60+ objects and 500 + fields.
Some of our test cases include attempting a CRUD operation on a primary object with valid and invalid elements based on the usage from the metadata.
The question we have is how can we drive our testing based on this metadata?
We can live with not having metadata retrieved at runtime by dumping the metadata before executing the test suite and using that dump as a data source.
How can we do multiple iterations through the data source which will drive composition of the SOAP requests?
An example the metadata will contact a contact the contact may contain 40-50 fields. Let's say 5 of those fields are required on create, 5 of the fields are not allowed on create and the rest are allowed but not required on create. We want to execute a test request where only the 5 required fields are specified, and verify we receive a valid response. We also want to test we can specify all required and allowed fields and receive a valid response. Then, we will want to test error conditions where we build 5 requests, each one omitting one of the required fields and verify that we receive a SOAP fault.
Additionally, we want to build 5 requests that specify the required fields and each one will specify a not allowed field and verify that we receive a SOAP fault.
We would want to do similar testing with the read, update, and destroy operations. We would also want to use a similar concept for validating field data ranges, etc.
We believe the only effective way to accomplish this and have a maintainable test suite is through a data-driven approach where the data is from metadata.
Is this possible in SOAPUI.?
