Forum Discussion

speedmaster's avatar
speedmaster
Occasional Contributor
12 years ago

How is it coming from LoadRunner? ...

I'm a longtime LR user, about 10 years. Anyone else here make the jump from LR to LoadComplete? How did you find it?



Does LoadComplete work with anything other than Web? We do some performance testing via Ctrix and other Windows client-server apps.



Thanks very much in advance,

Chris

4 Replies

  • punekar's avatar
    punekar
    Occasional Contributor
    Chris -



    It's great to see you here on these forums. I've had to recently make the transition from LR to LC in the last six months. 



    My key observations are as follows:




    The key thing to realize is that this tool is not LoadRunner. That's why it's an order of magnitude (almost 20x) cheaper than HP LR.


     


    Having worked with LoadComplete in 2 different firms, I find their USP is product support. They are quick to provide fixes, most of them inside of 2 days. The support staff also works collaboratively to devise solutions to the specific issues you have. Above all, their product development team and Director are reachable in case of issues. When I compare this to my experiences with HP, it is night and day.


     


    Once you understand that this is a maturing platform, you will be able to build scripts rapidly.


     


    Some other notes:


     


    1. The tool is more black box than white box. In the sense, their scripts are stored in a proprietary format as are the result files. This has caused me some irritation in being able to edit them outside of their editor interface. Hopefully, this will change. I've had conversations about putting their scripts in an open format and logging results to databases instead of JSON objects in flat files. 


     


    2. Their regular expression support is robust. However, they can only catch one expression in a variable. I've asked for the ability to capture an array of matches as opposed to just the one. Again, it's not clear in what release this feature will show up.


     


    3. Scripting support is not there at the moment. This means any custom modification of variables, string manipulation has to happen outside the tool. We have devised a mechanism to get around this limitation and so far it has worked fine. However, I have had a lengthy discussion with the Product Director on this and have been promised full scripting in the coming quarters.



    Sergei - are we still good for Q2 2013 for scripting?


     


    4. The relative ease with which scripts can be recorded/replayed is good. It compares well to other tools on the market on this score.


     


    5. Parameterization is supported in a variety of input formats including databases.


     


    6. Correlation is managed by regular expressions that you specify in the editor. It takes a few tries to get this right, especially if it's been a while since you last wrote POSIX-compliant regexps.


     


    7. There is support to monitor server machines of most types.


     


    8. Currently only HTTP/S is supported ( with parameterizable NTLM authentication credentials). So Citrix, Client/Server is out of scope for now. 



    If the tool moves from a black-box design to a more open extensible design, people like us could extend this tool to do work across more protocols. The last conversation I had with the product folks, I said exactly the same thing - that is, you can't fully anticipate what features the users will want and respond at that speed. Instead make the tool extensible and let the users "roll their own" on the platform. 


     


    9. The Controller/Generator setup is similar to that of Loadrunner. Same ports etc.






    10. Nomenclature differs from LoadRunner.


     


    Read "Test" as LoadRunner "Scenario"


    Read "Scenario" as LR "Script"


    Read "Station" as LR "Generator"




    Cheers

    Suresh



  • Anonymous's avatar
    Anonymous
    Chris
    This is from one of the SmartBears, so I hope that other users will comment on experience transitioning from LoadRunner.

    Regarding your second question. LoadComplete does not work with anything other than Web, but we do have a package based on TestComplete for load testing Windows client-server apps.
    If you are interested in details, please contact me directly at
    sergei dot sokolov at smartbear dot com

    Cheers
  • Anonymous's avatar
    Anonymous
    Suresh

    thanks for the extensive notes. There are my comments regarding some:



    1. We don't intend to make the entire "test script" (in LR terminology) available to users at this point. That may change with time, but no projections at present, regrettably. That said, regarding #3:



    3. We do plan to provide scripting support specifically for data manipulation. Yes, Q2 still sounds about right.



    6. There is a lot of auto data correlation that happens under the hood. Regex / internal variables are a supplement for that.



    8. More importantly. We do not plan to support Citrix or Client/Server in LoadComplete. However, there is a pretty efficient approach that can be implemented with TestComplete/ABS that we currently provide as custom deployment. Please contact me off line for details. We will likely publish a technical article on that subject.



    Cheers



  • punekar's avatar
    punekar
    Occasional Contributor
    Sergei -



    Thanks for taking time to respond.



    You will have to change your mind sooner or later about exposing the scenario/script format to users. Mercury/HP tried to keep this under wraps for the longest time and didn't succeed. People reverse engineered it and figured out the mechanics. Today everyone knows about the LCC precompiler and their custom object format. It helped for a more "aware" user base who would comprehend runtime issues with controller/generator communication etc.



    Key takeaway - Information, and openness, is better than a black box philosophy.



    Also, the results format today has to absolutely change. You have to populate it into either CSV or database. There is really no reason why users should not be in control of their own data. All you have to do is output raw numbers to CSV to make it immediately useful. We can then import it into spreadsheets and do our own reporting from there onwards.



    Please tell me there is a plan to ensure this is happening in a future release.



    About support for other protocols - why not include support for a compiled virtual user? Let us write our code in Visual C++, compile to object format, and have LoadComplete simply invoke that compiled code at runtime.