cancel
Showing results for 
Search instead for 
Did you mean: 

TestLeft support for UI-testing of "VisualStudio Extensions" - Ready to use? HowTo?

Highlighted
New Contributor

TestLeft support for UI-testing of "VisualStudio Extensions" - Ready to use? HowTo?

Hello. 

I installed TestLeft and I use the trial license.

I like to use the test suite for automated ui tests for VisualStudio Extensions.

My simple validation scenario is:

 

[1] start VisualStudio 2019 Preview experimental instance

[2] wait for started instance of [1]

[3] open "Create new project"-dialog   (i.e. File | New | Project...)

[4] select any project template

[5] click "Next"

[6] click "Create"

[7] wait for finished project creation

 

Do you have any manuals, howtos, hints to cover this scenario?

I played a little bit around and I still stuck at step [2] and [3].

2 REPLIES 2
Community Hero

Re: TestLeft support for UI-testing of "VisualStudio Extensions" - Ready to use? HowTo?

Hi,

 

According to https://support.smartbear.com/testleft/docs/general-info/supported-technologies/supported-applicatio..., you should be able to get what you need.

Note, that as Visual Studio does not provide debug information for itself, TestLeft will not be able to access its internals (except of .Net-based modules if any) and will get access to system-exposed windows only.

 

Also, I did not see any relation to Visual Studio Extension in your scenario. If your extension is .Net-based or provides debug info you should be able to access its windows and public methods/properties.

 

> I still stuck at step [2] and [3].

What exact problems do you have?

 

Regards,
Alex
[Community Expert Group]
____
[Community Expert Group] members are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Postings made by [Community Expert Group] members
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
[Community Expert Group] signature is used with permission by SmartBear Software.
http://smartbear.com/forums/f83/t86934/community-experts/
================================
New Contributor

Re: TestLeft support for UI-testing of "VisualStudio Extensions" - Ready to use? HowTo?

Thanks for your reply.

 

I just mentioned that I like to test VisualStudio Extensions because that is my main domain.

I can not test an VisualStudio Extension when the test tooling (i.e. TestLeft) does not support the application barebone. You have to know that most menus and toolbar are weaved dynamically during runtime (i.e. base on a xml-dialect called VSCT [Visual Studio Command Table, see https://docs.microsoft.com/en-us/visualstudio/extensibility/internals/visual-studio-command-table-do...]. We provide several commands with our VisualStudio Extension as well some individual toolwindows and several editor implementations, etc.pp.

 

Anyway, we like to have UI-testing for integration-tests (i.e. blackbox tests) with a "release" setup of our VisualStudio Extension. In this case we have a minimum set of debug information. We do some reflection and we use "Automation Interface"-approaches (e.g. EnvDTE.*) but 50-75% can be visual checks (e.g. screenshot compares, color checks, etc.) We do not need any VisualStudio-insides.

 

After said this, the easiest way to check if any test tooling allow UI-testing for VisualStudio is, to just clarify if the basic menu-walks work, e.g. go to Menu | File | New… to create a new project. When this does not work any effort to handle other calls will not work as well and the test tooling is not usable for our scenarios.

Currently I do not know how to traverse any dynamically weaved (vsct-based) UI-elements in VisualStudio, here for example, to create a new project (see attached screenshot).

TestLeftA.png

 

Currently I tried three ways to query the menu item "Project...":

 

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using SmartBear.TestLeft.TestObjects;
using SmartBear.TestLeft.TestObjects.WPF;

namespace TestLeftProject1
{
    [TestClass]
    public class TestLeftTest : UnitTestClassBase
    {
        #region Class initializers
        [ClassInitialize]
        public static void ClassInitialize(TestContext context)
        {
            UnitTestClassBase.InitializeClass(context);
        }

        [ClassCleanup]
        public static void ClassCleanUp()
        {
            UnitTestClassBase.FinalizeClass();
        }
        #endregion

        private IProcess _process = null;

        private IProcess RunVisualStudio()
        {
            if (_process == null)
            {
                _process = Driver.Applications.Run(
                @"C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\Common7\IDE\devenv.exe",
                @"/rootSuffix Exp");
            }

            var proc = System.Diagnostics.Process.GetProcessById(_process.ProcessId);
            if(proc.HasExited)
            {
                _process = null;
                _process = RunVisualStudio();
            }

            return _process;
        }
        
        [TestMethod]
        public void CreateProject_Test01()
        {
            IProcess process = RunVisualStudio();
            
            for (int i = 0; i < 300; ++i)
            {
                try
                {
                    IControl mainWindow = process.Find<IControl>(new WPFPattern()
                    {
                        ClrFullClassName = "Microsoft.VisualStudio.PlatformUI.MainWindow"
                    }, 2);

                    if (mainWindow == null)
                    {
                        System.Threading.Thread.Sleep(500);

                        continue;
                    }

                    mainWindow.Keys("ctrl+shift+n");
                }
                catch(Exception ex)
                {
                    Console.WriteLine(ex.Message);
                }
            }
        }

        [TestMethod]
        public void CreateProject_Test02()
        {
            IProcess process = RunVisualStudio();

            IControl ctrl = process.Find<IControl>(new WPFPattern()
            {
                ClrFullClassName = "Microsoft.VisualStudio.PlatformUI.MainWindow"
            }, 2).Find<IWPFMenu>(new WPFPattern()
            {
                WPFControlAutomationId = "MenuBar"
            }, 2).Find<IControl>(new WPFPattern()
            {
                ClrFullClassName = "Microsoft.VisualStudio.PlatformUI.VsMenuItem",
                WPFControlText = "_File"
            });

            ctrl.Click();

            var ctrlNew = ctrl?.Find<IControl>(new WPFPattern()
            {
                ClrFullClassName = "Microsoft.VisualStudio.PlatformUI.VsMenuItem",
                WPFControlText = "_New"
            });

            ctrlNew?.Click();

            var ctrlNewProject = ctrlNew?.Find<IControl>(new WPFPattern()
            {
                ClrFullClassName = "Microsoft.VisualStudio.PlatformUI.VsMenuItem",
                WPFControlText = "_Project..."
            });

            ctrlNewProject?.Click();
        }

        [TestMethod]
        public void CreateProject_Test03()
        {
            IProcess process = RunVisualStudio();

            IControl vsMenuItem = process.Find<IControl>(new WPFPattern()
            {
                ClrFullClassName = "System.Windows.Controls.Primitives.PopupRoot"
            }, 2).Find<IControl>(new WPFPattern()
            {
                ClrFullClassName = "Microsoft.VisualStudio.PlatformUI.VsMenuItem",
                WPFControlText = "_Project..."
            }, 3);

            vsMenuItem.Click();
        }
    }
}

 

 

 

New Here?
Join us and watch the welcome video:
Update to TestLeft 2.3
Labels
Join the September Hub-bub to show off, learn and win