Forum Discussion

sokol's avatar
sokol
New Contributor
4 years ago
Solved

ReadyAPI Test for Azure DevOps manual test association

Hi

I have executed test in Azure Devops pipeline and Ready API Test for Azure Devops extension.

I have tried to connect my ReadyAPI test executed in pipeline in Azure Devops (using ReadyAPI) to my manual test cases to make more coverage result and see much more information such test case steps, description etc. 

 

Have you guys tried something like that? I have read a lot of information and can find only answer that it is not supported (e.g. Selenium test you can easily associate in VisualStudio to manual test cases prepared in AzureDevops, but I can't find info how to do it with ReadyAPI tests). 

 

I'm sure that someone maybe wrote some addon or maybe it is only a matter of configuration.

 

Please help, any good ideas or info from your experience are more than welcome.

 

Thanks in advance

Piotr

  • It's a rather involved process.  The gist of it is this:

    * Install and setup Visual Studio for MSTest

    * Write a MSTest test case that calls a command line to execute the ReadyAPI command line call.

      NOTE:  I've noticed anything printed to the console during the test run is shown as an attachment in the ADO test case results.  This can be helpful to relay info back to ADO.

    * Get this working through VS, then move on to Azure DevOps.

    * In Azure DevOps, Test Plans, create a test case and save. 

    * Grab the test ID and back to VS, open Test Exporer, right click on the test case to tie to the ADO test case and associate the test id and save.

    * In Azure DevOps create a separate CI-CD pipeline to run just test plan test cases.

    * In your new Test Plan, select the test case created and click "Run with Options" in the drop down.

    * In the next pop up you will select the middle option to run automated tests, connect it to the build # and the pipeline and Stage.

     

    There are a lot of little things in-between to tweak.  This is the general format that I have working for a Java Selenium automation framework and for a ReadyAPI automation framework.  

     

    Hope this helps.  To be clear, Microsoft doesn't clearly explain this in any way.  This is something I've come up with on my own and researching each step of the way in the process.  If you find something else that makes it easier please share!  

  • First thing is you need to setup your tests environment in Visual Studio.  I went with MSTest, seemed to be older, for my purposes it worked well with ADO.  There are other options you can use, you can research online and see what works best for you.  So I would start there.  If you are not familiar with what you choose, first step would be to create a basic test case to get comfortable.  Once you have that then what I did for the test case is basically set it up like this:

     

    public void <MSTest Name here>()
    {
         string parameter = " -P<Prametername>=2948";
         string commandline = "/c " + testrunner + testcase + environment + parameter + projectfile;
         System.Console.WriteLine("**Command Line: " + commandline); // Output to see values for debug
    
         System.Diagnostics.Process process = System.Diagnostics.Process.Start("CMD.exe", commandline);
         process.WaitForExit();
         int x = process.ExitCode;
         System.Console.WriteLine("~~~ " + testrunner);  // Output to see values for debug
         System.Console.WriteLine("Exit code: " + x); // Output to see values for debug
         Assert.IsTrue(x.Equals(0)); // Tells us if this is successful or not. 0=Pass
    }

     

    I setup all these variables to the values I needed to pass in and make life easier.  (Using the "Launch TestRunner" in ReadyAPI helps here). 

     

    I would suggest you get this to work first and then worry about cleaning it up later.

     

    Hopefully you find this helpful.  If you run into anything please don't hesitate to reach out to me privately and I can give you more detail. ğŸ˜Š

     

    KHanchett

7 Replies

  • If I am understanding your question correctly you are wanting to tie in your test plans to execute ReadyAPI test cases?  If this is the case I have figured out a way to do it with ReadyAPI and any other tests you can execute from a command line that isn't "Microsoft".

     

    The short of it is that you need to use Visual Studio to achieve it with Microsoft's VS Test task.  The good news is I believe it would even work with the Community version.  It's very bare bones of what you would need to do in Visual Studio.  I have it currently working with ReadyAPI and Java Selenium via Maven calls.

     

    I don't think Microsoft supports any of it except if you are using their products exactly the way they say to.  I've not found much help elsewhere.  My results are reported back for each test case and traced and seems to be working pretty smoothly.  

     

    If this is what you are looking for feel free to reach out to me directly.  I'm not sure there is a private way to do that here.  I would be more than happy to share my code and explanation.

     

    • sonya_m's avatar
      sonya_m
      SmartBear Alumni (Retired)

      Thank you for helping khanchett ! 

      The SmartBear Community supports private messaging, however, if the info you are willing to share is OK share publicly, please do - other people will be able to use your approach in the future!

       

      Hi sokol, is this something you are looking to implement? Looks like khanchett  can help!

      • khanchett's avatar
        khanchett
        Contributor

        It's a rather involved process.  The gist of it is this:

        * Install and setup Visual Studio for MSTest

        * Write a MSTest test case that calls a command line to execute the ReadyAPI command line call.

          NOTE:  I've noticed anything printed to the console during the test run is shown as an attachment in the ADO test case results.  This can be helpful to relay info back to ADO.

        * Get this working through VS, then move on to Azure DevOps.

        * In Azure DevOps, Test Plans, create a test case and save. 

        * Grab the test ID and back to VS, open Test Exporer, right click on the test case to tie to the ADO test case and associate the test id and save.

        * In Azure DevOps create a separate CI-CD pipeline to run just test plan test cases.

        * In your new Test Plan, select the test case created and click "Run with Options" in the drop down.

        * In the next pop up you will select the middle option to run automated tests, connect it to the build # and the pipeline and Stage.

         

        There are a lot of little things in-between to tweak.  This is the general format that I have working for a Java Selenium automation framework and for a ReadyAPI automation framework.  

         

        Hope this helps.  To be clear, Microsoft doesn't clearly explain this in any way.  This is something I've come up with on my own and researching each step of the way in the process.  If you find something else that makes it easier please share!