cancel
Showing results for 
Search instead for 
Did you mean: 

How to add test execution result with statuses of steps from API

SOLVED
remixtedi
Occasional Contributor

How to add test execution result with statuses of steps from API

 

Hello, i am working to integrate my c# and MSTest solution to cucumber studio, we are using BDD format scenarios/tests and need to add test execution results with status for each step, but as you see from the screenshot, there is no such support for it... can you help me with that?

 

remixtedi_1-1614321246230.png

1 ACCEPTED SOLUTION

Accepted Solutions
remixtedi
Occasional Contributor

Re: How to add test execution result with statuses of steps from API

If i implement SpecNuts in my project, there will not be connection using uid's between scenario in .feature file and scenario in cucumber studio...

View solution in original post

8 REPLIES 8
cbliard
Staff

Re: How to add test execution result with statuses of steps from API

Hello,

 

You're right, there is no support in REST API to update individual test step results.

There are two ways to update them currently:

- manually through the interface

- automatically by sending a cucumber json result file. This is the only format we support that can update individual test step results.

 

You can push such results to an external test run.

Create an external test run from the test run creation page, then open it

cbliard_0-1614325703072.png

 

In the three dots menu, there is a "Push results" entry. Select it

cbliard_1-1614325768914.png

An information box will be displayed with information about how to push results in Cucumber JSON format (you have to select this format in the dropdown, as Junit XML is selected by default)

cbliard_2-1614325861644.png

 

You'll have to configure your test suite to generate this Cucumber Json file, and then try pushing it with curl as instructed. Once done, you should see the execution results with each step result correctly set.

 

Let us know if that worked.

remixtedi
Occasional Contributor

Re: How to add test execution result with statuses of steps from API

Hi Thanks for your reply,

Is there some instructions to configure my c# and mstest solution to generate results in cucumber messages format? but as i know external test results are not linked with scenarios in cucumber studio and this is not a solution for me... 😕

cbliard
Staff

Re: How to add test execution result with statuses of steps from API

Yeah you're right. When using external test runs, results are not tied to the scenarios you wrote. Sorry, I missed that it was important in your initial request.

 

To get the link between both, you need to create a test run from your scenarios, and then export those tests to code skeleton using hiptest-publisher. Upon doing that, you'll see that it generates tests in your language. Each test contains a uid that identify them on the CucumberStudio side.

 

After executing your tests, you'll have a result file that contains the test results and the uid. Pushing it back to CucumberStudio will update the results in the test run.

 

To be able to generate a cucumber json result file, you should use a gherkin based framework like SpecFlow. Is it an option in your case?

 

If exporting to C# directly, when a test fails, it's hard to know at which step it was failing because this information is not present most of the time: we only have the test name, and the steps are test implementation details. It looks like you are able to get this information though as you wanted to use the API to update tests at the step level. Can you describe how you have this information?

remixtedi
Occasional Contributor

Re: How to add test execution result with statuses of steps from API

hiptest-publisher is not an option for us because we have many test scenarios and complex solution, so generated files from cucumber studio (like specflow files) are not acceptable. we are using Specflow and we can detect every step and its status on the way or in the end of execution, so it there are some tools to generate automatically some result file in cucumber json format it would be great.

remixtedi
Occasional Contributor

Re: How to add test execution result with statuses of steps from API

Can you add support to add/update statuses of test steps? this will be great feature and we will move on your platform.

cbliard
Staff

Re: How to add test execution result with statuses of steps from API

I'm not very fluent in C# world. It looks like SpecNuts is an option: https://github.com/DonutReport/SpecNuts

Have you tried it already?

remixtedi
Occasional Contributor

Re: How to add test execution result with statuses of steps from API

If i implement SpecNuts in my project, there will not be connection using uid's between scenario in .feature file and scenario in cucumber studio...

View solution in original post

cbliard
Staff

Re: How to add test execution result with statuses of steps from API


@remixtedi wrote:

Can you add support to add/update statuses of test steps? this will be great feature and we will move on your platform.


So we have just added this ability. You can find the relevant documentation on our REST API documentation

To explain it briefly: along with the status, status-author and description attributes, you can also add a step-statuses attribute. It must be an array of string. Each string is a status. Possible values are ‘passed’, 'failed’, 'wip’, 'retest’, 'blocked’, 'skipped’, 'undefined’.
 
It then looks like this:
 
Request
 
Content-Type: application/json; charset=utf-8
Accept: application/vnd.api+json; version=1
access-token: xxxxx
client: xxxxx
uid: christophe.bliard@hiptest.net
 
{
  "data": {
    "type": "test-results",
    "attributes": {
      "status": "passed",
      "status-author": "Chris",
      "description": "Allright!",
      "step-statuses": ["passed", "undefined", "passed"]
    }
  }
}
 
Response
 
{
  "data": {
    "type": "test-results",
    "id": "218522447",
    "attributes": {
      "created-at": "2021-03-04T13:16:18.426Z",
      "updated-at": "2021-03-04T13:16:18.426Z",
      "last-author": "christophe.bliard@smartbear.com",
      "status": "passed",
      "description": "Allright!",
      "status-author": "Chris",
      "step-statuses": [
        "passed",
        "undefined",
        "passed"
      ],
      "test-snapshot-id": 17761339,
      "execution-environment-id": 178455
    },
    "links": {
      "self": "/test-results/218522447"
    },
    "relationships": {
      "test-snapshot": {
        "data": {
          "type": "test-snapshots",
          "id": "17761339"
        }
      },
      "build": {
        "data": {
          "type": "builds",
          "id": "1126746"
        }
      },
      "execution-environment": {}
    }
  },
  "included": [--truncated--]
}
 
And a screenshot
 
cbliard_0-1614879448753.png
Hope that helps 🙂
New Here?
Join us and watch the welcome video:
Labels