Inconsistent Behavior between JIRA Test Automation UI and Zephyr Squad's Test Automation API

 

I am attempting to use the Zephyr Squad Test Automation API to upload test executions to the appropriate test cycles. Unfortunately, I have seen some pretty finicky behavior from the API itself. To clarify, we are using the Zephyr JIRA integration - Zephyr Squad. 

 

To begin, I sanity checked that my test result files were compliant with formatting standards by uploading them manually via the Test Automation UI in JIRA.  Plenty of test automation tasks were executed successfully, but with some specific workflows needing to be met. For cucumber formatted JSONs, it was necessary to use the Create and Execute button to get these test results to upload successfully - in a single step. Uploading test execution results would fail If I wanted to break up the creation and execution of the Test Automation task into two steps. For example, I was able to create the automation task with the same file from the previous workflow by first choosing the Create option, but when I chose to Execute the task, the task would hang with In Progress status.  

I'd like to point out that none of the test result files or Task configurations had changed between these two workflows. 

 

With JUNIT xml formatted test results, the issues above were not encountered. So I thought, "hmmm, perhaps it is better to use the JUNIT xml format to upload test results." 

 

 

 

After deciding that we can move forward with JUNIT xml formatted test reports, I attempted to try out the Zephyr Squad Test Automation API via Postman. I was able to successfully create Test Automation Tasks via this API endpoint : https://prod-vortexapi.zephyr4jiracloud.com/api/v1/automation/job/create. However, I was unable to successfully execute the newly created Test Automation Task via this endpoint: https://prod-vortexapi.zephyr4jiracloud.com/api/v1/automation/job/execute. When hitting the latter endpoint with the appropriate jobId, I receive an Internal Server Error.

 

 

I then attempted to execute the newly created Test Automation job via the JIRA UI instead - to see if I had any luck. Weirdly enough, the Test Automation job that was created (not executed) through the API was not able to execute and instead gave me this unexpected error:

 

Considering I was unable to successfully execute the job via the API call in the first place (see Internal Server Error), I should not be encountering this error message. Additionally, the status for this job remains as UNEXECUTED.

 

Can you please point me to what the root cause of these issues may be? This is pretty inconsistent behavior. 

 

Thanks