Make Smart Assertion more useful with added features
Smart Assertions are very useful when having lots of data that need to be asserted. However currently there are a few issues that make it more difficult then it needs to be: 1. No Search Facility that allows quickly to jump to an item. 2. No conditional flow where assertions can be managed depending on some response details. 3. No facility to deal with lists that are unsorted and can be in a different order each time. 4. No facility to convert the Smart Assertions to Groovy Script which would make it useful as a tool to get a basic script with all the assertions that can be further edited to deal with special cases. Our responses contain hundreds of pieces of data that need to be evaluated. The Smart Assertions are great for quickly generating the assertions of a response but since we have many lists which don't arrive in a specific order, this does not work very well for us. If the Smart Assertions could recognise lists and pre-order them before assertion this would solve this problem and save us a lot of time. We also have an issue with repeated properties like for example version numbers that we only want to change once and then have it used in many places. However using a property only works as long as the data doesn't change. When data changes and we have to make a "Load from Transaction" again, those properties have to be added again. It would be nice to have something like an "Update from Transaction" that notices changes but doesn't overwrite existing items that haven't changed and preserves the properties being used. A differential dialog might be useful here when updating where we can see the changes, modify them if necessary or have them applied. Another great feature would be a conversion from Smart Asserts to Script. Other test applications that we use for desktop applications already can do this and it would be nice if SmartAPI also would be able to generate a script from the Assertions. This would help with situations where we need to have lots of special flows , loops and transformations of data but don't want to always have to write each assert from scratch.Randomly generated values in requests and other ReadyAPI functions
Current situation ReadyAPI DataGenerator provides different random values (random integer, email, name, city, etc.) I like this function since it creates nice test data. However, the random values are only accessible via DataGenerator DataSource. Proposal It would be great if we could use them in: REST, SOAP, and other requests Variable setup Groovy scripts etc. In ReadyAPI the random values could be available via: expansion e.g. ${#Random#FirstName} Get data function adding a Random category in the top level Implementation proposal Wrapping Java Faker (https://github.com/DiUS/java-faker ) would do the task. Java faker can be configured to use localized data (e.g. english names, spanish names, etc.) The faker provides a lot of interesting data useful for testing, just to name few of them: Address Animal App Artist Avatar Aviation Book Business Company Color File Finance Food Name Nation Number PhoneNumber and many more, including some funny categories like ChuckNorris.Script library interoperability, javaDoc, refactoring, search & autocomplete in all scripts
I'm sorry, but to me the idea mentioned Intellisense-on-custom-groovy-librarybyrunzTHalmost 6 years ago (wow!) is still not implemented, even though labeled as such. In current version ReadyAPI 3.10.0 I still do not have the kind of intellisense I would like to see: - no hassling with .jar files, I want to have interoperability with all of the groovy files in my custom script library like defined in the ReadyAPI project. A restart of some sort I can understand, but when we need to start with importing files we know this is something that will only work locally and for that version. - PLEASE: Proper IDE capabilities, like auto-complete, auto-formatting, code error detections (missing curly bracket for instance), renaming/refactoring, search (and replace),... in ALL scripts (setup scripts, groovy tests steps, tear downs, script library,...). It is such a breeze when I use other (front-end) test automation tools to see that I can do those things, but then return to ReadyAPI and feel like an IT-guy from the 90's. - I'd like proper javaDoc documentations to know what each method does. Currently I can only see what parameter type(s) I need to provide, but not what this method actually does or what I can expect to be returned. Scripting is such an extra weapon to be used in ReadyAPI, it's a shame it cannot be made more powerful by Smartbear.Re-insert the Assertion HTTP
in version 3.9.0 there is a smartAssertion created that deprecates the HTTP assertions. As all of my testcases (a huge amount) have this assertion I'm not going to adapt them manually as this will take a huge effort to do so. No clue why Smartbear has deleted this assertion as I think quasi all of the testers are using this!? Please, re-install the HTTP assertions!Add an event for Report post-processing
The need In ReadyAPI there are nice options of reporting, however there is missing any function to do report post-processing. The post-processing needs are: to send a report via email put the report into a storage create a Jira item etc. Currently the report files are just placed in the output directory. On TestCase and TestSuite level there are Report script holders, but the purpose of the script is toextend the reporting options(it is executed before the reports are finished, not after), i.e. for report pre-processing. On the project level there is no such option. Solution proposal A good place to execute specific actions arethe events. But there is no "ReportListener.afterReport" event. Please add a new event ReportListener.afterReport which will be triggered after any report is finished by testrunner. Such event will be used by Groovy scripts to do the report post-processing.
Masking encrypted properties in ReadyAPI Test for Azure DevOps task instead of plain text
Hi, The latest version of ReadyAPI 3.20.0 seems to fix encrypted properties being displayed as plain text in logs within the ReadyAPI software itself but this does not translate when running the project from Azure DevOps using the "ReadyAPI Test for Azure DevOps task" as encrypted properties are still being exposed there. 1. The project password for the task is in plain text - This property should be masked as anyone who views the task would be able to download and access the encrypted project and its properties. 2. Encrypted properties are displayed in the logs/requests as plain text The password has been encrypted in the custom properties using the "selected properties" project encryption . However, encyrpted properties are being shown in the task logs when a request that uses the properties fail or when logged using groovy script as plain text. This display of encrypted properties as plain text is unacceptable especially when creating production tests so it would be benefical if encyrpted properties are not exposed. Thanks, Kayleigh
Ability to Find/Replace in any direction
The Find/Replace dialog in ReadyAPI requires you to search for a term in a particular direction (Forward or Backward) In most cases I don't know what direction the term I'm searching for is in so I'd rather search the whole file. This would be more intuitive if the default behaviour was to search the entire file for all matched items (i.e. does not require a direction)
TestEngine + Jenkins Plugin: Execute list of Project/Suite/Case SoapUI tests
In their current implementation 1.) the TestEngine needs to be provided a zip with all required files by the calling client 2.) the TestEngine Jenkins plugin will execute one Project/Suite/Case per Jenkins job Problem: This setup does not scale to execute many Project/Suite/Case SoapUI tests in a row Suggestion: ad 1.) a.) Provide TestEngine the ability to access/checkout GIT repos containing SoapUI Projects b.) Add an API to TestEngine that will take the Project/Suite/Case/... parameters and pick all required files from above local checkout area --> Like this no files need to be created/transfered to the TestEngine server What still needs to be done is the specification of the list of Project/Suite/Case to be executed. This could be provided by one or more config files maintained in the GIT repo(s) ad 2.) In case the TestEngine runs on the Jenkins slave(s), the Jenkins job could - checkout the GIT repo(s) - locate the config file(s) containing the list of /Project/Suite/Case SoapUI Tests in the checkout area (e.g. via parameter) - loop the config file(s) calling the TestEngine API with Project/Suite/Case/... parameters --> like this - The TestEngine is running as a server (contrary to testruner.sh) - The invocation of many Project/Suite/Case's will run with minimal overhead (presumably msecs instead of secs) - all required files are taken from a repo -> any updates propagate to TestEngine automatically Remark: This type of automated execution of SoapUI Test from different Project/Suite/Case needs to complemented with a suitable test report (e.g. csv file with Project/Suite/Case/Step, timeStamp, Status, ExecTime, ErrorMsg, ...
