How much detail to include in a performance test report?

I’m a member of a LinkedIn group called “Performance Testing“. This morning a member of the group from BlazeMeter posted a link to their blog article about how much detail to put into a performance test report.

http://blazemeter.com/blog/what-include-load-test-report-technical-vs-management-reports

 

Historically I’ve always put large amounts of detail into performance test reports, but over the last 12 months or so I’ve started to reduce the amount of content. This allows me to produce what the customer generally wants in a shorter time frame.

In most performance tests customers tend to want to know the answer to one question:
Will my application perform well under expected user load?

This can generally be answered with “yes” or “no”, although occasionally the answer is “maybe”. (I tend to use traffic lights to show this at a high level.)

Often much more information than this is simply wasting time. In the last year or so I’ve started to produce reports  in Powerpoint that can be easily referred to in conference calls or webinars, be presented to clients at their site and can be re-used internally by my client’s project managers when they want to pass on information to their own internal customers.

I have found that by including less high-level detail and including embedded spreadsheets, charts or other documents allowing technical readers to “drill down” to the detail; I can keep all the potential readers of my reports happy.

I’ve attached a PDF “mock up” of a performance test report based on a test that I ran for a client earlier this year. I’d be interested to hear any comments from other testers about what works for them.

pdf-icon
Sample Test Report

First update to LoadRunner 11.50

When I launched LoadRunner to continue my investigations of the new version this morning, I was prompted with an automatic upgrade dialogue box. I’m really pleased that HP has decided to continue with automatic updates for LoadRunner which I first observed with LoadRunner 11.0 (Patch 4).

LoadRunner 11.5 Automatic Update to Patch 1
The update for LoadRunner 11.5 took about 7 minutes after the initial download of the update patch and took my version number from 11.50.2216.0 to 11.50.2552.0.

LoadRunner 11.5.2216.0 to 11.50.2252.0

Updated LoadRunner Date/Time sample script

A couple of days ago, I read Scott Moore’s interesting article about using multipliers to increment LoadRunner date/time variables by multiples of more than one day.

This made me look at my own date/time sample script, mentioned in an earlier post, and incorporate Scott’s suggestions in there. I added some sample code to check to see what day it was and calculate the number of days until next Monday.

Next Monday LoadRunner

Click the icon below to download the script containing this code together with other examples of date/time manipulation in LoadRunner.
zip
DateTimeSamples

 

For those of you who like to see “code” in posts, here is the “Next Monday” code in full…

//What date is next Monday?
lr_save_datetime(“%A”, DATE_NOW, “Today”);

if (strcmp (lr_eval_string(“{Today}”),”Monday”)==0)
{
lr_save_datetime(“%d/%m/%y”, DATE_NOW+(ONE_DAY*7), “DDMMYY”);
lr_output_message(“Today is Monday, next Monday’s Date is %s”,lr_eval_string(“{DDMMYY}”));
}
if (strcmp (lr_eval_string(“{Today}”),”Tuesday”)==0)
{
lr_save_datetime(“%d/%m/%y”, DATE_NOW+(ONE_DAY*6), “DDMMYY”);
lr_output_message(“Today is Tuesday, next Monday’s Date is %s”,lr_eval_string(“{DDMMYY}”));
}
if (strcmp (lr_eval_string(“{Today}”),”Wednesday”)==0)
{
lr_save_datetime(“%d/%m/%y”, DATE_NOW+(ONE_DAY*5), “DDMMYY”);
lr_output_message(“Today is Wednesday, next Monday’s Date is %s”,lr_eval_string(“{DDMMYY}”));
}
if (strcmp (lr_eval_string(“{Today}”),”Thursday”)==0)
{
lr_save_datetime(“%d/%m/%y”, DATE_NOW+(ONE_DAY*4), “DDMMYY”);
lr_output_message(“Today is Thursday, next Monday’s Date is %s”,lr_eval_string(“{DDMMYY}”));
}
if (strcmp (lr_eval_string(“{Today}”),”Friday”)==0)
{
lr_save_datetime(“%d/%m/%y”, DATE_NOW+(ONE_DAY*3), “DDMMYY”);
lr_output_message(“Today is Friday, next Monday’s Date is %s”,lr_eval_string(“{DDMMYY}”));
}
if (strcmp (lr_eval_string(“{Today}”),”Saturday”)==0)
{
lr_save_datetime(“%d/%m/%y”, DATE_NOW+(ONE_DAY*2), “DDMMYY”);
lr_output_message(“Today is Saturday, next Monday’s Date is %s”,lr_eval_string(“{DDMMYY}”));
}
if (strcmp (lr_eval_string(“{Today}”),”Sunday”)==0)
{
lr_save_datetime(“%d/%m/%y”, DATE_NOW+(ONE_DAY*1), “DDMMYY”);
lr_output_message(“Today is Sunday, next Monday’s Date is %s”,lr_eval_string(“{DDMMYY}”));
}