TestComplete integration with TestRail

[chandreshp]

Hi, 



I am using TestRail to manage my Test suites and I am using TestComplete to perform automation testing of my application.





TestComplete allows to set log defects to JIRA. But does it provide any kind of API or Plugin which can help me to integrate TestComplete execution result to TestRail ? I mean is there any way in which I can update my test case result directly from TestComplete ?



Thanks,

Chandresh Parmar

[TanyaYatskovska]

Hi Chandresh,

Support for JIRA is implemented via TestComplete's Script Extension. It gets access to JIRA objects via its API. If TestRail provides REST API as well, I suppose you can extend the support and add the needed code to work with TestRail.

[chandreshp]

Thanks for reply Tanya!



I will have to look into TestComplete's Script Extension first and then I will try to access TestRail objects.



Thanks,

Chandresh Parmar

[schef]

I implemented a solution (in JScript) using the .NET api bindings from the testrail website:

// Note: this functionality depends on 3rd party .NET libraries that needed to be compiled by you. We downloaded the testrail .NET api bindings
// from the testrail website, along with the Json.NET library. Using visual Studio 2010, we built the Gurock.TestRail library, referencing the Newtonsoft.Json
// library. We built x86 and x64 versions using .NET 3.5. In this project, the libraries are stored in a folder next to the project, and have 
// already been linked to the project via the CLR bridge.


testrail = {};

// Use these methods for updating test case statuses in TestRail
// Use the methods testrail.passCase, testrail.failCase, testrail.blockedCase, and testrail.retestCase to add case results.
// The methods require the run id and the case id
// A run id in test rail is formatted like 'R4683'. Remove the letter R from the beginning and just supply the integer.
// A case id in test rail is formatted like 'C866870'. Remove the letter C from the beginning and just supply the integer.
//
// Supplying the additionalFields json object is optional. The request fields can be found in the add_result method definition 
// of the api at http://docs.gurock.com/testrail-api2/reference-results
// e.g. {comment: '<A comment to be logged in the result record>', version: '<free form version info>'}

testrail.passCase = function(runId, caseId, additionalFields) {
    testrail.addStatusAndResultForCase(runId, caseId, additionalFields, 1);
};

testrail.failCase = function(runId, caseId, additionalFields) {
    testrail.addStatusAndResultForCase(runId, caseId, additionalFields, 5);
};

testrail.blockedCase = function(runId, caseId, additionalFields) {
    testrail.addStatusAndResultForCase(runId, caseId, additionalFields, 2);
};

testrail.retestCase = function(runId, caseId, additionalFields) {
    testrail.addStatusAndResultForCase(runId, caseId, additionalFields, 4);
};

testrail.runIdAndCaseIdFromTestId = function(testId) {
    test = testrail.getTest(testId);
    caseId = test.Item('case_id').Value_2.m_value;
    runId = test.Item('run_id').Value_2.m_value;
    return {runId: runId, caseId: caseId};
};


// Support functions for the above functions. These do not need to be called directly outside of this file.

testrail.addStatusAndResultForCase = function(runId, caseId, additionalFields, statusId) {
    additionalFields = additionalFields || {};
    additionalFields.status_id = statusId;
    testrail.addResultForCase(runId, caseId, additionalFields);
};

testrail.addResultForCase = function(runId, caseId, additionalFields) {
    dataObj = testrail.dataDictonary(additionalFields);
    testrail.sendPost("add_result_for_case/" + runId + "/" + caseId, dataObj);
};

testrail.getTest = function(testId) {
    result = testrail.sendGet('get_test/' + testId);
    return result;
};

testrail.sendPost = function(apiArgsAsString, dataDictionaryObj) {
    testrail.apiClient().SendPost(apiArgsAsString, dataDictionaryObj);
};

testrail.sendGet = function(apiArgumentString) {
    results = testrail.apiClient().SendGet(apiArgumentString);
    return results;
};

testrail.dataDictonary = function(jsonObj) {
    var dataD = dotNET.System_Collections.Hashtable.zctor();
    for (var key in jsonObj) {
        if (jsonObj.hasOwnProperty(key)) {
            dataD.Add(key, jsonObj[key]);
        }
    }
    return dataD;
};

testrail.apiClient = function() {
    var client = dotNET.Gurock_TestRail.APIClient.zctor("https://testrail.blahblah1234.com/testrail/"); 
    client.User = "bojack.horseman@blahblah1234.com"; 
    client.Password = "api_key_from_your_testrail_instance_instead_of_users_pwd"; 
    return client; 
}; 

//TODO: add error handling 
//TODO: implement the add_results api call so we can do a bulk update instead of calling add_result_for_case for each case

Usage:

//USEUNIT TestRail

function testUpdateTestRail() {
    testrail.retestCase(4683, 866867);
    testrail.passCase(4683, 866868, {version: 'Build 10.0.0.1234'});
    testrail.blockedCase(4683, 866869, {comment: 'Blocked due to JIRA-4321'});
    testrail.failCase(4683, 866870, {comment: 'Failed. JIRA-5678 logged.', version: 'Build 10.0.0.1234'});
}

[cbeal]

Can you tell me it your integration was successful?  I am looking at trying the same integration.

Were you able to get results from Test Complete to automatically update TestRail?

What was involved, what issues had to be overcome.

Thanks!

[leandropoblet]

Can I please get more information on how to get the library going?
Specially the notes in the script can you clarify the steps to do it?

// Note: this functionality depends on 3rd party .NET libraries that needed to be compiled by you. We downloaded the testrail .NET api bindings
// from the testrail website, along with the Json.NET library. Using visual Studio 2010, we built the Gurock.TestRail library, referencing the Newtonsoft.Json
// library. We built x86 and x64 versions using .NET 3.5. In this project, the libraries are stored in a folder next to the project, and have
// already been linked to the project via the CLR bridge.

Thank you

[Colin_McCrae]

It's not a RESTful API. So you need to use the code they provide for binding to various languages and integrate it into TC using the CLR Bridge functionality. Which allows you to use external libraries.

The previous poster used the .NET bindings.

So start reading here: https://support.smartbear.com/testcomplete/docs/testing-with/advanced/using-external-functions/calli...

Which is what you'll need to use: http://docs.gurock.com/testrail-api2/bindings-dotnet

Now, I do notice that TestRail refer to that as V2 of their API. I have no idea if the above script was written using these bindings, or ones from an older version.

But above all, you need to understand how to use external libraries. Once you understand that, then you can start worrying about how you create and use one for TestRail. (And you need someone who knows their way around Visual Studio and .NET to make all this work.)

[leandropoblet]

Hi Colin,

Thank you for your reply. Yes I downloaded the API binding and the Json.NET dependency.

I managed to build the project including that dependency. Now I guess the only part missing would be making TestComplete use it.
I'll investigate the link and hopefully will work in the end.

Thanks again!

[leandropoblet]

Apparently there's an error when parsing the request.

So what I did is compiling the testrail integration with donNET Gurock.TestRail including Json90 library.

Then added the resulting dlls to the CLR Bridge using 64-bits architecture/appartment model = 'Does not matter' (as it seems to be the right thing to do although I'm not sure)

Then calling the function where parameters are correct and meaning (testRun, testCaseNumber):

function testUpdateTestRail() {
testrail.retestCase(13, 3486);
}

The error is what you see in the image.... Any clues/suggestions?

I guess the real question would be if CLR has been added correctly, which I assume yes as I can see the dotNET routines when typing and why isn't parsing Json correctly and how to debug it.

Thank you

[Colin_McCrae]

CLR Bridge (which I've only used once) seems to be working fine. The error is coming from the TestRail method.

Looks like it simply doesn't like one of the characters in your file path. (Hence the "unexpected character" error. The actual TestRail API isn't one I'm familiar with ...) Without knowing which API endpoint you're hitting, and the JSON you're sending it, can't be certain what it doesn't like. But it looks like a file path? And you have back, rather than forward slashes in there? In which case, you may need to double them up as per: http://stackoverflow.com/questions/24828873/sending-a-filepath-in-json-string-to-php-codeigniter

Or switch it to using forward slashes.

Have you tried putting the JSON into something like: http://jsonlint.com/

... to check it's valid?