Load testing with VSTS and LoadRunner

I've not been able to post for a while - been knee deep in work and life, while standing on my head...

Anyway, I've been working with the QA team at my workplace on load testing some of our applications headed for Production over the last few days, getting a taste of both LoadRunner and VSTS.

LoadRunner is *the* heavy duty tool supporting loads of protocols, including SAP and Siebel (amongst others), in addition to the basic requirement of HTTP. We were testing an ASP.NET application which came a cropper when ViewState values would not get replayed, but until then recording a test, replaying it and configuring a load was fairly simple. Debugging is not A-grade AFAIK - but breakpoints and analysis of variables is provided. We tested this off a 10 day trial license, which restricts load to 10 virtual users; the real thing is *said* to be expensive (claims unverified).

On to Visual Studio Team System (VSTS) - Microsoft have done a good job making setting up a test easy, but the bias towards testing Microsoft stuff is definitely visible (e.g. in browser support). Setting up a test was easy, as was parameterizing it. LoadRunner has lots of features to generate data, but VSTS 2005 required some effort to bind to a data source. Thankfully, VSTS 2008 ('Orcas') will have data binding to CSV/XML files in-built; VSTS does provide for extensibility via Request/Test Plugins, which can be used to provide data. I did hit an issue where if an asp:Button has its UseSubmitBehavior switched off, the script records the Cancel button's existence on the page as well, leading to the subsequent HTTP request not being handled correctly by the server. Tests were conducted on a 180-day trial license, without any feature restrictions.

The comparison plays out like this - for basic web apps (definitely ASP.NET apps), VSTS should suffice. While not being cheap, VSTS does not cost as much as LoadRunner. However, LoadRunner is a far more mature product and versatile with it - having support for ERPs and protocols like CITRIX and RDP. The VSTS feature of rigging Unit Tests and Web Tests into Load Tests was pretty impressive and the .NET language + debugging features of the VS IDE outstripped LoadRunner by some distance.

Final word: select what you need for your specific requirement; but for the majority of small-medium business applications, VSTS should meet your need.