Forum Discussion

sokol's avatar
sokol
New Contributor
4 years ago

ReadyAPI Test for Azure DevOps manual test association

Hi I have executed test in Azure Devops pipeline and Ready API Test for Azure Devops extension. I have tried to connect my ReadyAPI test executed in pipeline in Azure Devops (using ReadyAPI) to my ...
  • khanchett's avatar
    khanchett
    4 years ago

    It's a rather involved process.  The gist of it is this:

    * Install and setup Visual Studio for MSTest

    * Write a MSTest test case that calls a command line to execute the ReadyAPI command line call.

      NOTE:  I've noticed anything printed to the console during the test run is shown as an attachment in the ADO test case results.  This can be helpful to relay info back to ADO.

    * Get this working through VS, then move on to Azure DevOps.

    * In Azure DevOps, Test Plans, create a test case and save. 

    * Grab the test ID and back to VS, open Test Exporer, right click on the test case to tie to the ADO test case and associate the test id and save.

    * In Azure DevOps create a separate CI-CD pipeline to run just test plan test cases.

    * In your new Test Plan, select the test case created and click "Run with Options" in the drop down.

    * In the next pop up you will select the middle option to run automated tests, connect it to the build # and the pipeline and Stage.

     

    There are a lot of little things in-between to tweak.  This is the general format that I have working for a Java Selenium automation framework and for a ReadyAPI automation framework.  

     

    Hope this helps.  To be clear, Microsoft doesn't clearly explain this in any way.  This is something I've come up with on my own and researching each step of the way in the process.  If you find something else that makes it easier please share!  

  • khanchett's avatar
    khanchett
    4 years ago

    First thing is you need to setup your tests environment in Visual Studio.  I went with MSTest, seemed to be older, for my purposes it worked well with ADO.  There are other options you can use, you can research online and see what works best for you.  So I would start there.  If you are not familiar with what you choose, first step would be to create a basic test case to get comfortable.  Once you have that then what I did for the test case is basically set it up like this:

     

    public void <MSTest Name here>()
{
     string parameter = " -P<Prametername>=2948";
     string commandline = "/c " + testrunner + testcase + environment + parameter + projectfile;
     System.Console.WriteLine("**Command Line: " + commandline); // Output to see values for debug

     System.Diagnostics.Process process = System.Diagnostics.Process.Start("CMD.exe", commandline);
     process.WaitForExit();
     int x = process.ExitCode;
     System.Console.WriteLine("~~~ " + testrunner);  // Output to see values for debug
     System.Console.WriteLine("Exit code: " + x); // Output to see values for debug
     Assert.IsTrue(x.Equals(0)); // Tells us if this is successful or not. 0=Pass
}

     

    I setup all these variables to the values I needed to pass in and make life easier.  (Using the "Launch TestRunner" in ReadyAPI helps here). 

     

    I would suggest you get this to work first and then worry about cleaning it up later.

     

    Hopefully you find this helpful.  If you run into anything please don't hesitate to reach out to me privately and I can give you more detail. ðŸ˜Š

     

    KHanchett

Dustin0's avatar
Dustin0
New Contributor

khanchett 

 

wanted to say thank you for the initial suggestion.

I went to try and implement something like this for C#, but most of the Test projects would not allow the running of the executable.  For example an MSTest UnitTest Project would say "A Project with an Output Type of Class Library cannot be started directly"

 

I was able to launch Ready API within a new C# project and can utilize a Process.Start command in Main method within the class. This gives me the launch of the executable that I can probably update to be more dynamic later.   But struggled to get the command line runner working within Test projects. 

 

If you have some capacity to PM a bit this coming week I'd love to go over your setup for getting an executable or batch file called within TestRunner

 

this was my source within a standard C# project that worked

 

static void Main(string[] args)
{
Console.WriteLine("Starting Ready API!");
ProcessStartInfo startInfo = new ProcessStartInfo();
startInfo.FileName = "S:\\Program Files\\SmartBear\\ReadyAPI-3.9.0\\bin\\testrunner.bat";
startInfo.Arguments = "-sRegression -j -f${WORKSPACE} \"-RJUnit-Style HTML Report\" -FXML -Eqa C:\\Users\\TestUser\\Downloads\\MY-readyapi-project";
startInfo.WindowStyle = ProcessWindowStyle.Normal;
Process.Start(startInfo);
}

khanchett's avatar
khanchett
Contributor
4 years ago

First thing is you need to setup your tests environment in Visual Studio.  I went with MSTest, seemed to be older, for my purposes it worked well with ADO.  There are other options you can use, you can research online and see what works best for you.  So I would start there.  If you are not familiar with what you choose, first step would be to create a basic test case to get comfortable.  Once you have that then what I did for the test case is basically set it up like this:

 

public void <MSTest Name here>()
{
     string parameter = " -P<Prametername>=2948";
     string commandline = "/c " + testrunner + testcase + environment + parameter + projectfile;
     System.Console.WriteLine("**Command Line: " + commandline); // Output to see values for debug

     System.Diagnostics.Process process = System.Diagnostics.Process.Start("CMD.exe", commandline);
     process.WaitForExit();
     int x = process.ExitCode;
     System.Console.WriteLine("~~~ " + testrunner);  // Output to see values for debug
     System.Console.WriteLine("Exit code: " + x); // Output to see values for debug
     Assert.IsTrue(x.Equals(0)); // Tells us if this is successful or not. 0=Pass
}

 

I setup all these variables to the values I needed to pass in and make life easier.  (Using the "Launch TestRunner" in ReadyAPI helps here). 

 

I would suggest you get this to work first and then worry about cleaning it up later.

 

Hopefully you find this helpful.  If you run into anything please don't hesitate to reach out to me privately and I can give you more detail. ðŸ˜Š

 

KHanchett