Ramping up your usage of SOAP UI NG ? Check out our Free weekly Training webinar!
Hello all, Have you recently bought new Ready! API licenses and are about to kick start your API tests? Register here for our Free Weekly interactive Webinar, Next one is on the 7th of September( day after Labor day) What to expect? Advice on getting up and running on your SOAP and REST projects Hidden gems within the tool that you may not uncover on your own Personalized question-and-answer with our API experts If you have 10+ users and would like to get your users on-boarded, please reply to this message and I'll be in touch. Cheers, Katleen B Snr. Customer Success Advisor EMEA kbb(at)smartbear.com
ReadyAPI 3.56.0 does not fail testcase if iteration in datasource fails
Hi, We have a datasource which takes data from excel to feed in the request. This excel has multiple rows, each row creates a different request and each row gets it's own assertion data. Since release 3.56.0, it seems like whenever an iteration fails from the datasource that is NOT the very last row in the datasource, the test case will still be reported as passed. As a workaround, I'm trying to create a custom testcase property "hasFailed" with a standard value of "false". Whenever an assertion fails, the testcase property is set on "true", but whenever I try to execute the follow script in my test case teardown script: def hasFailed = testRunner.testCase.getPropertyValue("hasFailed") if (hasFailed == "true") { log.info(hasFailed) testRunner.fail("Testcase failed, check transaction log for more details") } it doesn't seem to work. The log.info is displayed but the testRunner.fail doesn't fail the test case itself. Any tips on how to fix this?
How to build SQL query using strings whose value should be retrieved from TestCase property
Suppose i have the following properties defined in the Test Case Properties FromTime = 2023-11-15 23:00:00 ToTime = 2023-11-16 23:00:00 I want to replace the time given in the below query with the above test case property variables. How to do that in ReadyAPI JDBC datasource ? SELECT * FROM Table1 where TagId = 4314 and Time > '2023-11-15 23:00:00' and Time < '2023-11-16 23:00:00' I am able to retrieve integer values like TagId and use it by adding 'Prepared Properties' within the JDBC datasource window. But if i add the string and use it inside the query, data is not fetched as the string is not interpreted properly.
How to best assert unordered results?
One of the great features of ReadyAPI are the Smart Assertions. They are really useful when dealing with large amounts of data and can be easily adjusted when there are some changes by using "Load from Transactions". This saves a lot of time having to manually add each assertion. However, there a a few issues that make it less than ideal. 1. No Search Facility to quickly find an Assertion. 2. No Facility to add Flows like If and Else or Loops. 3. No Conversion to Script. 4. And especially no facility to deal with unordered list results where the order can be different with each call. 5. Also no Update Feature that allows preserving properties and previous setups. Therefore we unfortunately can't really use this feature with most of our calls. I would like to know if there are others that have similar problems to solve and if there is a recommended way to deal with this apart from writing complex scripts. Data-Driven Tests might be one way.oauth2 authorization dynamic test automation through event handlers
Hi, I created below script, hope you can benefit from it as well... I am relatively new to soapui & test automation, have less then 1 year experience with it and had no previous relevant coding background, so perhaps this might not be 100% efficient... but it does what it needs to do (for me :-) ). First of all, there are already a lot of features within soapui that allow you to dynamically use authorization. At project level you have the "Auth Manager" tab. You can create different profiles and specify for which operations to use which profile. The get access token screens did allow me to authenticate and retrieve an access token, but then we wanted to include my soapui testcases into the build pipeline (via TestRunner.bat command line execution) and there the integrated browser would not work. Even the automation of the browser windows (via front end automation getElementById etc.) would probably not work. So I went with the following approach which gave me sufficient customization possibilities while staying within a generic framework. It makes use of the soapui pro feature RequestFilter.filterRequest within "event handlers". First off, I need a test suite that will go get my authentication tokens. I have several, for easy demonstration I include 2, an oauth2 token (with access & refresh token validity for several hours) and an openid token generation, without refresh, but with limited 1 hour validity. I have 1 test suite which I will run at the start of the project, called "_Authorization". This includes 2 test cases, 1) "oauth2" and 2) "openid". For the first I have following test steps: 1.Groovy script to determine whether token is expired. 2. A rest request to refresh the token (if needed) called "REST Request Refresh_Token". This request uses the parameters client_id, grant_type (refresh), client_secret & refresh_token. (note: before being able to use this I had to manually obtain an access & refresh token first through grant_type (obtain authorization code). After this manual step I can get a new access token via the refresh token request automatically) 3. A datasink to write the new token (access & refresh) to an excel, together with the new expiration date etc. (if needed) 4. A datasource test step. This is were the event handler will go get my oauth2 token to inject in all my other test cases. The second test case "openid" will use a few more steps: 1. DataSource where I have an excel with all my specific "numbers" for which I want to generate a specific openid token. I can provide also a claim via this excel (first name, last name also, might depend on your openid client). In contrary to the above oauth2; the authorization code will be specific for certain parameterized request (for instance request for specific clients, id's, organisation numbers, ...). These request will be authorized with different authentication profiles (claims) that determine who can see what. 2. A rest request test step called "token" to obtain the tokens. This request uses the parameters grant_type, code & redirect_uri. 3. A groovy test step to get and set the properties for each number from my datasource. So here the tokens used by my test cases would be retrieved from the properties within the soapui project. 4. A datasink to write the new token to an excel, together with the new expiration date etc. I write it to an excel file as it is easier to have an overview via an external file. 5. A datasource loop that points back to 2. See below the scripts for: oauth2 test step 1: import java.text.DateFormat; import java.text.SimpleDateFormat; import java.util.Date; import groovy.time.TimeCategory TimeZone.setDefault(TimeZone.getTimeZone('CET')) testRunner.runTestStepByName("DataSource_external file") def expirydate = context.expand( '${DataSource_external file#expiry_date}' ) Date now = new Date() //expiry date is in string format. We need to convert to Date DateFormat datumformaat = new SimpleDateFormat("EEE MMM d HH:mm:ss z yyyy", Locale.US) Date expirydate_convertedtodate = datumformaat.parse(expirydate) if (expirydate_convertedtodate > now) { log.info "Access Token not expired. We run only the last step called DataSource_external file. This way all requests using this oauth2 authorization will be able to insert this token."; testRunner.gotoStepByName("DataSource_external file") } else { testRunner.runTestStepByName("DataSource_external file") //log.info "Access Token expired! Run the entire test suite to retrieve a new access & refresh token. If this step would fail we need to run manually an obtain access & refresh token rest request." def refresh_token = context.expand( '${DataSource_external file#refresh_token}' ) def client_secret = context.expand( '${DataSource_external file#client_secret}' ) def old_access_token = context.expand( '${DataSource_external file#access_token}' ) // we store it at test case level so rest request step to get the refresh token can use it. testRunner.testCase.setPropertyValue("refresh_token", refresh_token); testRunner.testCase.setPropertyValue("client_secret", client_secret); currentDate = new Date() use( TimeCategory ) { expirydate_minus1hour = currentDate - 3600.seconds expirydate_plus16hours = currentDate + 57600.seconds } testRunner.testCase.setPropertyValue("expirydate", expirydate_plus16hours.toString()) //I included an if statement here as I use also an old access token to test I do not get authorized. Old means that it was refreshed more than 1 hour ago, otherwise it would still be valid if (expirydate_convertedtodate < expirydate_minus1hour) { log.info ("The old access token expired on "+expirydate_convertedtodate+", this is more than 1 hour ago ("+expirydate_minus1hour+"). So this one is no longer valid and we can write it to property old_access_token") testRunner.testCase.setPropertyValue("old_access_token_DOSIS", old_access_token); } else { log.info ("Old access token expired on "+expirydate_convertedtodate+", this is LESS than 1 hour ago ("+expirydate_minus1hour+"). So this one would actually be still valid, so we do NOT write it to property old_access_token") } } Groovy script OpenId step 3: import groovy.time.TimeCategory TimeZone.setDefault(TimeZone.getTimeZone('CET')) def expires_in = context.expand( '${#Project#expires_in}' ) currentDate = new Date() use( TimeCategory ) { expirydate_plus1hour = currentDate + 3600.seconds } def numberA= context.expand( '${DataSource_OpenId#numberA}' ) def firstname = context.expand( '${DataSource_OpenId#first name}' ) def lastname = context.expand( '${DataSource_OpenId#last name}' ) def claim = context.expand( '${DataSource_OpenId#claim}' ) def description= context.expand( '${DataSource_OpenId#description}' ) def token_response = context.expand( '${Token#Response#$[\'id_token\']}' ) log.info ("For property "+omschrijving+" with numberA \""+numberA+"\" we got following token (expires on: "+ expirydate_plus1hour + "): " +token_response) testRunner.testCase.testSuite.project.setPropertyValue ("authorization_"+description, token_response) return expirydate_plus1hour And now the big kicker. The RequestFilter.filterRequest piece of code: //event creates custom header for ALL requests (overwriting the Authorization header if it was already provided in the test steps). import java.util.ArrayList; import java.util.List; import java.util.regex.*; def headers = request.requestHeaders // test step variabelen ophalen def requestname = request.getName() def numberA = context.getProperty("numberA") def numberB = context.getProperty("numberB") // based upon the operation we use in the test steps we will inject another authorization header. So the event listens to the compiled request and will insert the authorization header BEFORE it submits it. String operation = request.getOperation() // first operation is the actual get access & refresh tokens from the authorization service (oauth2 and openid). There is no authorization header needed for these operations. We use test suite _Authorization to dynamically call the authoriztion service and // write the results to an external excel file if ( operation == "RestResource: /authorization/oauth2/token"|| operation == "RestResource: /authorization/openid/token" ) { headers.remove("Authorization"); request.requestHeaders = headers; log.info ("teststep "+ requestname + " concerns operation " + operation + " . This is an authentication request oauth2 or openid, so we do not need to insert an authorization header.") } else // GENERIC - string "(403)", "(401)" or "(_negative_)" in de test step name - we do not pass authorization! This way we can test the actual authorzation, to see we get a 401 or 403 http response code if ( requestname.contains("(403)")|| requestname.contains("(401)")|| requestname.contains("(_negative_)") ) { headers.remove("Authorization"); request.requestHeaders = headers; log.info ("teststep "+ requestname + " authenticion testcase. So we do not insert Authorization heade (capital A). It is possible to manually insert an authorization header (no capital A, so we can distinguish between automatical insertion or manual)") } else // depending on the rest resource endpoint operations, I can insert specific authorization headers. For instance, below operations do not have an input parameter. To be able to generically determine which openid token to insert for each rest request test step // we include in the test step name square brackets with between it the number for which we want to get the openid token, for instance [12345 ] will go and insert the authorziation header token for number 12345, generated in the _Authentication test suite. if ( operation == "RestResource: /api/v1/abc/{id}"|| operation == "RestResource: /api/v1/def" ) { if ( requestnaam.contains("[") ) { Matcher openid = Pattern.compile("(?<=\\[).+?(?=\\])").matcher(requestnaam) def token = context.expand( '${#Project#authorization_'+openid[0]+'}' ) // log.info ("name of the test step is: " + requestnaam +" - with operation: " +operation) // log.info ("In the requstname we have \'[" +openid[0] + "]\', so we inject the authorization OpenId header like geneated in the _Authorization test suite: " +token) headers.remove("Authorization"); headers.put("Authorization", "bearer "+token); request.requestHeaders = headers; } else //Attention, in case of no "[" in the request name we do a default openid authorization { def token = context.expand( '${#Project#authorization_default}' ) log.info ("name of teststep is: " + requestnaam +" - with operation: " +operation) log.info (Default openid authorization, \${#Project#authorization_default} die resulteert in deze OpenId Authorization header: " +token) headers.remove("Authorization"); headers.put("Authorization", "bearer "+token); request.requestHeaders = headers; } } else // In below operation we will have an input parameter for the request. We will read that one and automatically inject the matching openid authorization header we got from the _Authorization test suite. if ( operation == "RestResource: /api/getdata/numberA/{numberA}"|| operation == "RestResource: /api/getotherdata/numberA/{numberA}" ) { def token = ""; if (numberA != null) { token = context.expand( '${#Project#OpenIdToken_'+numberA+'_soapui_default}' ) log.info ("Name of the teststep is: " + requestname +" - with operation: " +operation) log.info ("From the request we get parameter number " +numberA + " which results in following openid token: " +token) headers.remove("Authorization"); headers.put("Authorization", "bearer "+token); request.requestHeaders = headers; } else { log.info "No numberA found in the request parameter, so the result would probably be a bad request (400)"; throw new Exception("No numberA found in the request parameter, so the result would probably be a bad request (400)"); } } else // DEFAULT - we insert another oauth authorization header (not openid) valid for most of my operations { def authorization_oauth = context.expand('${#[_Authentication#_Authentication#DataSource_external file]#authorization_REST}'); headers.remove("Authorization"); headers.put("Authorization",authorization_oauth); request.requestHeaders = headers; log.info ("Name of test step is: " + requestname +" - with operation: " +operation) log.info ("Default event handler applied: Trigger event Authorization header with the general oauth2 token, retrieved from external file: "+authorization_oauth) }
