Forum Discussion

allenj2020's avatar
allenj2020
Contributor
7 years ago

TestComplete scalability vs Appium

Hello all,

 

I'm a long time TestComplete desktop user, but we're developing a companion mobile app for the first time.

I'm in the early stages of researching automation options.

 

Appium seems to be the industry leader and is supported by many cloud based device farms for scripting, it looks like it can scale to any level needed and do parallel test execution, but the setup is complex and the learning curve is steep.

TestComplete Mobile is tempting as it will allow us to easily write new mobile tests using our existing test framework and knowledge, but I'm concerned about the long term scalability if we want to quickly run tests on multiple devices in the future.  At the moment it seems to be limited to just physical connected device(s)?

This will be fine in the short term, but I don't want to get a couple of years into a full implementation, decide we want to do parallel testing on multiple physical/virtual/cloud devices at once, and find out we can't do it.

 

Does anyone have any experience or comments on the scalability of TestComplete Mobile?

Will it support, or are there plans to support:

a) Multiple physical devices on the same machine.

b) Cloud based device farms.

b) Automated or semi automated device configuration (e.g. deploy the app and run the test).  If this can be done with external setup scripts that's OK.

d) Emulators (e.g. launched from Android Studio) - low priority but could be occasionally useful.  I'm not as interested in the Chrome simulation capability as it also requires a Web module license which we don't have.

 

Thanks for any input you may have!

Allen

 

 

  • AlexKaras's avatar
    AlexKaras
    7 years ago

    Hi Allen,

     

    > Will it support, or are there plans to support:

    Obvious note : this is a user forum and none of us is informed about development plans. You may contact Support directly via the https://support.smartbear.com/message/?prod=TestComplete form and wonder if they can provide any piece of information.

     

    Nevertheless:

    -- Multiple physical devices on the same machine are supported but in sequential execution mode (i.e. tests can be executed on only one device at a time);

    -- Cloud device farms are not supported. (Yet?). Though AWS cloud itself is supported - search https://support.smartbear.com/articles/testcomplete/ page for the 'Cloud' word - there are 5+ articles on the topic;

    -- Application deploy (installation) to device is supported;

    -- Android emulators are supported.

     

    P.S. With TestComplete it is possible and not complex to create tests that interact with both desktop and mobile. E.g. do something in mobile and process the result in desktop. This is something that obviously cannot be implemented with Appium.

     

  • AlexKaras's avatar
    AlexKaras
    Champion Level 3
    7 years ago

    Hi Allen,

     

    > Will it support, or are there plans to support:

    Obvious note : this is a user forum and none of us is informed about development plans. You may contact Support directly via the https://support.smartbear.com/message/?prod=TestComplete form and wonder if they can provide any piece of information.

     

    Nevertheless:

    -- Multiple physical devices on the same machine are supported but in sequential execution mode (i.e. tests can be executed on only one device at a time);

    -- Cloud device farms are not supported. (Yet?). Though AWS cloud itself is supported - search https://support.smartbear.com/articles/testcomplete/ page for the 'Cloud' word - there are 5+ articles on the topic;

    -- Application deploy (installation) to device is supported;

    -- Android emulators are supported.

     

    P.S. With TestComplete it is possible and not complex to create tests that interact with both desktop and mobile. E.g. do something in mobile and process the result in desktop. This is something that obviously cannot be implemented with Appium.

     

    • tristaanogre's avatar
      tristaanogre
      Esteemed Contributor
      7 years ago

      If you purchase a license to use the Environment Manager via TestComplete, then you have access to multiple hardware devices that you can execute tests on.  However, I DON'T know if that will work for custom mobile applications.  I think it only supports web applications displayed on mobile devices.

    • allenj2020's avatar
      allenj2020
      Contributor
      7 years ago

      Thanks Alex, that's good information about the device support.  

       

      I'm already doing parallel execution of the desktop application tests with multiple test machines, it would be interesting to push the boundaries and see if having 2-3 physical devices or emulators on each host could be made to work.

       

      Our application will be a pure Android and iOS app, not web based, so I don't think the cross browser testing ability will be helpful.

       

      I would still be curious to hear from any users doing multi-device testing with TestComplete Mobile.

      But based on the information so far it sounds promising enough to move ahead with a trial.

       

      Thanks.

      Allen

      • AlexKaras's avatar
        AlexKaras
        Champion Level 3
        7 years ago

        Any time, Allen :)

         

        I remember that nginoza did a lot of mobile testing with TestComplete. Maybe he can provide some comments as well...