Ask a Question

server version: uploading test results to Zephyr Scale via the automation API

SOLVED
fede1
New Contributor

server version: uploading test results to Zephyr Scale via the automation API

Hello!

I'm using Zepyhr Scale Server and I would like to upload to Zephyr the results of my automation testsuite made with pytest. I've tried this POST request: 

post(url="https://{my-jira-host}/rest/atm/1.0/automation/execution/{projectKey}", auth=({my_username}, {my_password}), files={"file":open("test_results.zip","rb")})

 but it doesn't work because the response is "errorMessages":["Invalid Custom Format JSON file"]}'.

I'm uploading a zip file containing one xml file generated with 

pytest --junitxml=output/junitxml_report.xml

as it's explained here https://support.smartbear.com/zephyr-scale-cloud/docs/test-automation/pytest-integration.html

I've tried to make the same request with an API client (Postman) and the error is "Invalid ZIP file", even if I fail the authentication with a wrong username or even if I upload the xml file only.

Maybe someone does the same thing and could help me? I'm a newbie πŸ™‚ thanks!

1 ACCEPTED SOLUTION

Accepted Solutions
abioneperhobby
Occasional Contributor

Re: server version: uploading test results to Zephyr Scale via the automation API

Support team answered me that there is no junit impor funcionality in server version, but only in cloud version.

i implemented by myself parsing the xml and using server api to create-update test results.

View solution in original post

10 REPLIES 10
abioneperhobby
Occasional Contributor

Re: server version: uploading test results to Zephyr Scale via the automation API

Hi,

i'm facing the same issue with both the junit.xml file generated by pytest and the junit.xml generated by jest, and neither one works.

Someone has a working xml file?

 

MisterB
Community Leader

Re: server version: uploading test results to Zephyr Scale via the automation API

Just checking as I notice that the documentation link you posted is for the Cloud version but you say you're on the Server version.  The APIs might not be interchangeable between the versions.

 

Also, maybe some of these previous posts to the community might help: Search - SmartBear Community

 

 

fede1
New Contributor

Re: server version: uploading test results to Zephyr Scale via the automation API

thanks for the answer! yes you're right, I checked the cloud documentation because it's written how to make the integration with pytest, and in the server documentation it isn't reported. Is it impossible? but I used the server version API that I found here https://support.smartbear.com/zephyr-scale-server/api-docs/v1/

MisterB
Community Leader

Re: server version: uploading test results to Zephyr Scale via the automation API

Glad that's helped clear up part of the issue.  I've not used the APIs that much, and not for what you're trying to do but it seems to me that the API for server is generic and as long as you follow the syntax it should work.  Has it worked for you?

abioneperhobby
Occasional Contributor

Re: server version: uploading test results to Zephyr Scale via the automation API

i'm using the server documentation, 

maybe the problem can be with the endpoint?

 

following documentation https://support.smartbear.com/zephyr-scale-server/api-docs/v1/ it should to be something like

 

http://your-jira-host:port/your-jira-context/rest/atm/1.0/

but taking a look at the network in chrome i found a call with this endpoint

 

https://jira.namirial.com/rest/tests/1.0/testcase/search?archived=false&fields=id,key,projectId,name... 

 

and the api works using /rest/tests/1.0/testcase/search but not using /rest/atm/1.0/ as described

 

abioneperhobby_2-1629793875567.png

 

but if i point to

 

https://jira.namirial.com/rest/tests/1.0/automation/execution/CUSTOMPRJ

 

i obtain

 

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<status>
<status-code>404</status-code>
<message>null for uri: https://jira.namirial.com/rest/tests/1.0/automation/execution/CUSTOMPRJ</message>
</status>

 

instead if i point to https://jira.namirial.com/rest/atm/1.0/automation/execution/CUSTOMPRJ 

 

the endpoint is recognized but the api respond that there are a invalid custom format json file

 

abioneperhobby_3-1629794873243.png

 

i tried with

 

  • xml generated by pytest 
  • xml generated by jest
  • xml downloaded online

 

MisterB
Community Leader

Re: server version: uploading test results to Zephyr Scale via the automation API

As a test, you could use another rest/atm endpoint to make sure it's working for you.  This endpoint works for me:

 

https://jira.[YOUR INSTANCE].com/rest/atm/1.0/testrun/[YOUR TEST CYCLE]/testresults

 

If that's working for you, and the JSON file you're providing is in the correct format as mentioned in the documentation here (smartbeartm4j / tm4j-junit-integration β€” Bitbucket) then I'm sorry I can't help.  I've only used the APIs for GET functions and not very often.

abioneperhobby
Occasional Contributor

Re: server version: uploading test results to Zephyr Scale via the automation API

i make other tests and i understand that the api doesn't want a junit.xml file but a json file formatted as zephyr wants, as for example

{
   "version": 1,
   "executions":[
      {
         "source":"CalculatorSumTest.sumTwoNumbersAndPass",
         "result":"Passed",
         "testCase": {
            "key": "QA-T1"
         }
      },
      {
        "source":"CalculatorSumTest.sumTwoNumbersAndFail",
        "result":"Failed",
         "testCase": {
            "key": "QA-T2"
         }
      },
      {
        "source":"CalculatorSumTest.notMappedToTestCaseAndPass",
        "result":"Passed"
      },
      {
        "source":"CalculatorSumTest.mappedToTestCaseNameAndPass",
        "result":"Passed",
         "testCase": {
            "name": "Mapped to Test Case Name and Pass"
         }
      }
   ]
}

as described here https://bitbucket.org/smartbeartm4j/tm4j-junit-integration/src/master/ 

but here https://support.smartbear.com/zephyr-scale-cloud/docs/test-automation/pytest-integration.html documentation talk about uploading a simple xml file, not a json file, but using this api https://support.smartbear.com/zephyr-scale-cloud/api-docs/#operation/createJUnitExecutions that doesn't exists for the server version.

 

 

 

Can Zephyr Server version be updated to accept the createJUnitExecutions api?

 

MisterB
Community Leader

Re: server version: uploading test results to Zephyr Scale via the automation API

Sorry but I can only suggest raising a request with the SmartBear support team at this link: Login - Jira Service Management (atlassian.net)

 

But @josh42 might also be able to help?

josh42
Contributor

Re: server version: uploading test results to Zephyr Scale via the automation API

Hi guys,

I work with the server API quite extensively, but have not used the JUnit integration, yet. I can only refer you to the docs

https://support.smartbear.com/zephyr-scale-server/docs/test-automation/integrations/junit.html

https://support.smartbear.com/zephyr-scale-server/docs/test-automation/integrations/annotations-with...

 

One thing, I can clear up is the difference between the API endpoints using rest/atm/... and the ones using rest/tests/...

The atm ones are the public API endpoints of Zephyr Scale which are documented for public use.

The tests endpoints are the internal ones that ZS uses. So those are not meant for public use and thus are not documented publicly. They can be quite handy, if you are savvy enough to figure them out (because they return a lot more information - like internal ids and such - than the public endpoints) but you will always run the risk of running into undocumented changes in the API and not getting the same sort of support than you get for the official API (also, you can break stuff quite easily using these endpoints, so be careful).

 

As for the rest, I'm with @MisterB , raise a feature request for the server API and in the meantime contact support directly to have them help you figure out, why your JUnit results do not have the right format.

 

Cheers

Josh

cancel
Showing results forΒ 
Search instead forΒ 
Did you mean:Β