Ask a Question

Unable to Publish Cucumber Tests Results, using Jenkins (pluggin zephyr scale), to Zephyr Scale

SOLVED
ykowalko
New Contributor

Unable to Publish Cucumber Tests Results, using Jenkins (pluggin zephyr scale), to Zephyr Scale

Hi,

We have developed some tests automatized from the frameworks named "webdriverIO" and "Cucumber".

Our scenarios of testing are uploaled automaticly from the "Zephyr Scale" (https://......./jira/secure/Tests.jspa#/design?projectId=xxxxxx) with a jenkins job using the pluggin "zephyr scale" (like this:https://support.smartbear.com/zephyr-scale-server/docs/test-automation/integrations/configure-jenkin...) . And It's working.

After the running of the execution of the tests, we generated a "cucumber report" (in a json file) and "junit report" (in a xml file) adding in a folder of my repository GIT

From Jenkins, and the "Zephyr Scale pluggin", we want published these results/reports in the "Zephyr Scale" in JIRA. BUT It's not working

When we are running the build, we have this issue :

{code}

[Zephyr Scale] [ERROR] *There was an error trying to publish test results to Zephyr Scale*. Error details:
com.adaptavist.tm4j.jenkins.http.Tm4jJiraRestClient.processUploadingResultsResponse(Tm4jJiraRestClient.java:102)
com.adaptavist.tm4j.jenkins.http.Tm4jJiraRestClient.uploadCucumberFile(Tm4jJiraRestClient.java:36)
com.adaptavist.tm4j.jenkins.extensions.postbuildactions.TestResultPublisher.perform(TestResultPublisher.java:87)
com.adaptavist.tm4j.jenkins.extensions.postbuildactions.TestResultPublisher.perform(TestResultPublisher.java:67)
jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:752)
hudson.model.Build$BuildExecution.post2(Build.java:177)
hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:697)
hudson.model.Run.execute(Run.java:1932)
hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
hudson.model.ResourceController.execute(ResourceController.java:97)
hudson.model.Executor.run(Executor.java:429)
*There was an error with the Jira Instance(https://........./jira). Http Status Code: 500*
[Zephyr Scale] [ERROR] Tests results have not been sent to Zephyr Scale
ERROR: Build step failed with exception
java.lang.RuntimeException
at com.adaptavist.tm4j.jenkins.extensions.postbuildactions.TestResultPublisher.perform(TestResultPublisher.java:76)
at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:752)
at hudson.model.Build$BuildExecution.post2(Build.java:177)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:697)
at hudson.model.Run.execute(Run.java:1932)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Build step 'Zephyr Scale: Publish Test Results' changed build result to FAILURE
Build step 'Zephyr Scale: Publish Test Results' marked build as failure
Finished: FAILURE
{code}



+In my job jenkins I used these params+
> Manage Source Code/Git/Repositories/Repository URL = url of my git
> Manage Source Code/Git/Repositories/Credentials = my credentials
> Manage Source Code/Git/Branches to build/Branch Specifier = the branch where are the json cucumber report
> Action After the build/Zephy Scale: Publish Test Results/Jira Instance = url of my jira
> Action After the build/Zephy Scale: Publish Test Results/Project Key = the key of my project
> Action After the build/Zephy Scale: Publish Test Results/Test framework = Cucumber
> Action After the build/Zephy Scale: Publish Test Results/File path = the path of the cucumber reports (.json)

Do you know why the publish of my cucumber results in the zephyr scale JIRA pushed a server error?
Do you have an exemple of json file to send a result in Zephyr Scale?

In attachment my cucumber result file (but in .txt)

Note I use :

Zephyr Scale Server

- version 8.0.3

- without used  API Zephyr KEY



Thank you in advance for your help and advices
Best regards
Yannick

3 ACCEPTED SOLUTIONS

Accepted Solutions
osmolyar
New Contributor

Re: Unable to Publish Cucumber Tests Results, using Jenkins (pluggin zephyr scale), to Zephyr Scale

@ykowalko the issue is this output in the json report, which records an "element" the feature corresponding to the feature itself (keyword: Scenario, type: feature), and has an empty steps: [] array.  The Zephyr scale https://usjira.iscinternal.com/rest/atm/1.0/automation/execution/cucumber api (which I presume the Jenkins plugin uses) doesn't support empty steps arrays. The latest wdio-cucumber-js-json reporter has been fixed so you can download the latest version 4.1.4 and try again (https://github.com/webdriverio-community/wdio-cucumberjs-json-reporter/issues/68)

 

"tags": [],
"elements": [
{
"keyword": "Scenario",
"type": "feature",
"description": "",
"name": "PAYS - Russia - telecommandes header-main-menu - CTA",
"tags": [],
"id": "pays---russia---telecommandes-header-main-menu---cta;pays---russia---telecommandes-header-main-menu---cta",
"steps": []
},

 

You can also test the uploads separately via url --basic --user ****:******** -F "file=@login.zip" https://usjira.iscinternal.com/rest/atm/1.0/automation/execution/cucumber/DP?autoCreateTestCases=tru...

View solution in original post

osmolyar
New Contributor

Re: Unable to Publish Cucumber Tests Results, using Jenkins (pluggin zephyr scale), to Zephyr Scale

@ykowalko 

That api endpoint should be available on Zephyr Scale Server.  I'm not entirely sure that is what the Jenkins plugin uses (though that would make sense).  But it's the endpoint that we use to test uploads manually with Postman.

osmolyar_0-1634229701524.png

I was able to upload your tests to our instance by saving the text as a .json file and zipping (zipfile attached).  Perhaps the issue is that the file extension on your file is ".xml".  If the file extension is not ".json" in Postman, the error "Invalid zip file" is returned.  Maybe that is the error that Jenkins is reporting.

osmolyar_1-1634233491963.png

Note the tags arrays were removed from the json in the attached zip file since otherwise there's an error about no matching test key.

 

 

View solution in original post

ykowalko
New Contributor

Re: Unable to Publish Cucumber Tests Results, using Jenkins (pluggin zephyr scale), to Zephyr Scale

Hi
thank you for your feedback 
We will work with your advices
Best regards
Yannick

View solution in original post

5 REPLIES 5
ykowalko
New Contributor

Re: Unable to Publish Cucumber Tests Results, using Jenkins (pluggin zephyr scale), to Zephyr Scale

Note I use :

Zephyr Scale Server

- version 8.0.3

- without used  API Zephyr KEY

osmolyar
New Contributor

Re: Unable to Publish Cucumber Tests Results, using Jenkins (pluggin zephyr scale), to Zephyr Scale

@ykowalko the issue is this output in the json report, which records an "element" the feature corresponding to the feature itself (keyword: Scenario, type: feature), and has an empty steps: [] array.  The Zephyr scale https://usjira.iscinternal.com/rest/atm/1.0/automation/execution/cucumber api (which I presume the Jenkins plugin uses) doesn't support empty steps arrays. The latest wdio-cucumber-js-json reporter has been fixed so you can download the latest version 4.1.4 and try again (https://github.com/webdriverio-community/wdio-cucumberjs-json-reporter/issues/68)

 

"tags": [],
"elements": [
{
"keyword": "Scenario",
"type": "feature",
"description": "",
"name": "PAYS - Russia - telecommandes header-main-menu - CTA",
"tags": [],
"id": "pays---russia---telecommandes-header-main-menu---cta;pays---russia---telecommandes-header-main-menu---cta",
"steps": []
},

 

You can also test the uploads separately via url --basic --user ****:******** -F "file=@login.zip" https://usjira.iscinternal.com/rest/atm/1.0/automation/execution/cucumber/DP?autoCreateTestCases=tru...

View solution in original post

ykowalko
New Contributor

Re: Unable to Publish Cucumber Tests Results, using Jenkins (pluggin zephyr scale), to Zephyr Scale

Hi @osmolyar  ,
cc @smartBear team


Thank you very much for your feedback 

I upgraded my "wdio-cucumberjs-json-reporter" into "^4.1.4". My "steps: [] array" is not empty now, but when I tried to publish the tests result in Zephyr Scale Server (8.0.3) I have the same error if I use the Zephy Scale pluggin in Jenkins

It's working if I zip my result and run a curl command from the rest API like this:
curl --basic --user MYLOGIN:MOTDEPASS -F "file=@report.zip" https://MYURLJIRA/jira/rest/atm/1.0/automation/execution/cucumber/MYPROJECTKEY?autoCreateTestCases=f...

Do you have an idea of the issue with Zephy Scale pluggin in Jenkins ?

In attachment, the error message in the build, and a json result.

 

Thank you in advance
Regards

osmolyar
New Contributor

Re: Unable to Publish Cucumber Tests Results, using Jenkins (pluggin zephyr scale), to Zephyr Scale

@ykowalko 

That api endpoint should be available on Zephyr Scale Server.  I'm not entirely sure that is what the Jenkins plugin uses (though that would make sense).  But it's the endpoint that we use to test uploads manually with Postman.

osmolyar_0-1634229701524.png

I was able to upload your tests to our instance by saving the text as a .json file and zipping (zipfile attached).  Perhaps the issue is that the file extension on your file is ".xml".  If the file extension is not ".json" in Postman, the error "Invalid zip file" is returned.  Maybe that is the error that Jenkins is reporting.

osmolyar_1-1634233491963.png

Note the tags arrays were removed from the json in the attached zip file since otherwise there's an error about no matching test key.

 

 

View solution in original post

ykowalko
New Contributor

Re: Unable to Publish Cucumber Tests Results, using Jenkins (pluggin zephyr scale), to Zephyr Scale

Hi
thank you for your feedback 
We will work with your advices
Best regards
Yannick

View solution in original post

cancel
Showing results for 
Search instead for 
Did you mean: