cancel
Showing results for 
Search instead for 
Did you mean: 

TestComplete integration with TestRail

Occasional Contributor

TestComplete integration with TestRail

Hi, 



I am using TestRail to manage my Test suites and I am using TestComplete to perform automation testing of my application.





TestComplete allows to set log defects to JIRA. But does it provide any kind of API or Plugin which can help me to integrate TestComplete execution result to TestRail ? I mean is there any way in which I can update my test case result directly from TestComplete ?



Thanks,

Chandresh Parmar
Thanks,
Chandresh Parmar | QA Engineer
22 REPLIES 22
Community Manager

RE: TestComplete integration with TestRail


Hi Chandresh,


 


Support for JIRA is implemented via TestComplete's Script Extension. It gets access to JIRA objects via its API. If TestRail provides REST API as well, I suppose you can extend the support and add the needed code to work with TestRail.


 

---------
Tanya Gorbunova
SmartBear Community Manager

Join the annual community event and share your stories to win gifts
Wintertainment2019
Occasional Contributor

RE: TestComplete integration with TestRail

Thanks for reply Tanya!



I will have to look into TestComplete's Script Extension first and then I will try to access TestRail objects.



Thanks,

Chandresh Parmar
Thanks,
Chandresh Parmar | QA Engineer
New Contributor

Re: TestComplete integration with TestRail

I implemented a solution (in JScript) using the .NET api bindings from the testrail website:

 

// Note: this functionality depends on 3rd party .NET libraries that needed to be compiled by you. We downloaded the testrail .NET api bindings
// from the testrail website, along with the Json.NET library. Using visual Studio 2010, we built the Gurock.TestRail library, referencing the Newtonsoft.Json
// library. We built x86 and x64 versions using .NET 3.5. In this project, the libraries are stored in a folder next to the project, and have 
// already been linked to the project via the CLR bridge.


testrail = {};

// Use these methods for updating test case statuses in TestRail
// Use the methods testrail.passCase, testrail.failCase, testrail.blockedCase, and testrail.retestCase to add case results.
// The methods require the run id and the case id
// A run id in test rail is formatted like 'R4683'. Remove the letter R from the beginning and just supply the integer.
// A case id in test rail is formatted like 'C866870'. Remove the letter C from the beginning and just supply the integer.
//
// Supplying the additionalFields json object is optional. The request fields can be found in the add_result method definition 
// of the api at http://docs.gurock.com/testrail-api2/reference-results
// e.g. {comment: '<A comment to be logged in the result record>', version: '<free form version info>'}

testrail.passCase = function(runId, caseId, additionalFields) {
    testrail.addStatusAndResultForCase(runId, caseId, additionalFields, 1);
};

testrail.failCase = function(runId, caseId, additionalFields) {
    testrail.addStatusAndResultForCase(runId, caseId, additionalFields, 5);
};

testrail.blockedCase = function(runId, caseId, additionalFields) {
    testrail.addStatusAndResultForCase(runId, caseId, additionalFields, 2);
};

testrail.retestCase = function(runId, caseId, additionalFields) {
    testrail.addStatusAndResultForCase(runId, caseId, additionalFields, 4);
};

testrail.runIdAndCaseIdFromTestId = function(testId) {
    test = testrail.getTest(testId);
    caseId = test.Item('case_id').Value_2.m_value;
    runId = test.Item('run_id').Value_2.m_value;
    return {runId: runId, caseId: caseId};
};


// Support functions for the above functions. These do not need to be called directly outside of this file.

testrail.addStatusAndResultForCase = function(runId, caseId, additionalFields, statusId) {
    additionalFields = additionalFields || {};
    additionalFields.status_id = statusId;
    testrail.addResultForCase(runId, caseId, additionalFields);
};

testrail.addResultForCase = function(runId, caseId, additionalFields) {
    dataObj = testrail.dataDictonary(additionalFields);
    testrail.sendPost("add_result_for_case/" + runId + "/" + caseId, dataObj);
};

testrail.getTest = function(testId) {
    result = testrail.sendGet('get_test/' + testId);
    return result;
};

testrail.sendPost = function(apiArgsAsString, dataDictionaryObj) {
    testrail.apiClient().SendPost(apiArgsAsString, dataDictionaryObj);
};

testrail.sendGet = function(apiArgumentString) {
    results = testrail.apiClient().SendGet(apiArgumentString);
    return results;
};

testrail.dataDictonary = function(jsonObj) {
    var dataD = dotNET.System_Collections.Hashtable.zctor();
    for (var key in jsonObj) {
        if (jsonObj.hasOwnProperty(key)) {
            dataD.Add(key, jsonObj[key]);
        }
    }
    return dataD;
};

testrail.apiClient = function() {
    var client = dotNET.Gurock_TestRail.APIClient.zctor("https://testrail.blahblah1234.com/testrail/"); 
client.User = "bojack.horseman@blahblah1234.com";
client.Password = "api_key_from_your_testrail_instance_instead_of_users_pwd";
return client;
};

//TODO: add error handling
//TODO: implement the add_results api call so we can do a bulk update instead of calling add_result_for_case for each case

Usage:

 

//USEUNIT TestRail

function testUpdateTestRail() {
testrail.retestCase(4683, 866867);
testrail.passCase(4683, 866868, {version: 'Build 10.0.0.1234'});
testrail.blockedCase(4683, 866869, {comment: 'Blocked due to JIRA-4321'}); testrail.failCase(4683, 866870, {comment: 'Failed. JIRA-5678 logged.', version: 'Build 10.0.0.1234'});
}
New Member

Re: TestComplete integration with TestRail

Can you tell me it your integration was successful?  I am looking at trying the same integration.

 

Were you able to get results from Test Complete to automatically update TestRail?

 

What was involved, what issues had to be overcome.

 

Thanks!

Frequent Contributor

Re: TestComplete integration with TestRail

Can I please get more information on how to get the library going?
Specially the notes in the script can you clarify the steps to do it?

// Note: this functionality depends on 3rd party .NET libraries that needed to be compiled by you. We downloaded the testrail .NET api bindings
// from the testrail website, along with the Json.NET library. Using visual Studio 2010, we built the Gurock.TestRail library, referencing the Newtonsoft.Json
// library. We built x86 and x64 versions using .NET 3.5. In this project, the libraries are stored in a folder next to the project, and have
// already been linked to the project via the CLR bridge.

Thank you
Community Hero

Re: TestComplete integration with TestRail

It's not a RESTful API. So you need to use the code they provide for binding to various languages and integrate it into TC using the CLR Bridge functionality. Which allows you to use external libraries.

 

The previous poster used the .NET bindings.

 

So start reading here: https://support.smartbear.com/testcomplete/docs/testing-with/advanced/using-external-functions/calli...

 

Which is what you'll need to use: http://docs.gurock.com/testrail-api2/bindings-dotnet

 

Now, I do notice that TestRail refer to that as V2 of their API. I have no idea if the above script was written using these bindings, or ones from an older version.

 

But above all, you need to understand how to use external libraries. Once you understand that, then you can start worrying about how you create and use one for TestRail. (And you need someone who knows their way around Visual Studio and .NET to make all this work.)

Frequent Contributor

Re: TestComplete integration with TestRail

Hi Colin,

Thank you for your reply. Yes I downloaded the API binding and the Json.NET dependency.

I managed to build the project including that dependency. Now I guess the only part missing would be making TestComplete use it.
I'll investigate the link and hopefully will work in the end.

Thanks again!

Frequent Contributor

Re: TestComplete integration with TestRail

Apparently there's an error when parsing the request.

 

So what I did is compiling the testrail integration with donNET Gurock.TestRail including Json90 library.

 

Then added the resulting dlls to the CLR Bridge using 64-bits architecture/appartment model = 'Does not matter' (as it seems to be the right thing to do although I'm not sure)

 

Then calling the function where parameters are correct and meaning (testRun, testCaseNumber):

function testUpdateTestRail() {
testrail.retestCase(13, 3486);
}

 

The error is what you see in the image.... Any clues/suggestions?

I guess the real question would be if CLR has been added correctly, which I assume yes as I can see the dotNET routines when typing and why isn't parsing Json correctly and how to debug it.

 

Thank you

 

 

 

Community Hero

Re: TestComplete integration with TestRail

CLR Bridge (which I've only used once) seems to be working fine. The error is coming from the TestRail method.

 

Looks like it simply doesn't like one of the characters in your file path. (Hence the "unexpected character" error. The actual TestRail API isn't one I'm familiar with ...) Without knowing which API endpoint you're hitting, and the JSON you're sending it, can't be certain what it doesn't like. But it looks like a file path? And you have back, rather than forward slashes in there? In which case, you may need to double them up as per: http://stackoverflow.com/questions/24828873/sending-a-filepath-in-json-string-to-php-codeigniter

 

Or switch it to using forward slashes.

 

Have you tried putting the JSON into something like: http://jsonlint.com/

 

... to check it's valid?