UNIX performance monitoring with LoadRunner

On my current project, I need to monitor UNIX system performance. I’m currently waiting for a new version of application code to test, so I thought that I’d look into some of the UNIX performance counters that I’m going to monitor.

I’m very familiar with Windows performance monitoring but less so with UNIX, so I thought that I’d try to compare UNIX performance counters with Windows performance counters. When monitoring the performance of a UNIX system using LoadRunner you need to enable RSTATD on the server being tested.

Once you’ve enabled RSTATD it’s a simple matter to tell your LoadRunner controller to monitor the UNIX ststistics for the server under test. All you need is it’s IP address. Once you add the server to the list of monitored servers iN LoadRunner Controller you se a number of counters which LoadRunner can monitor.

 

download

 

Many of these counters are broadly equivalent to Windows performance counters. The table below describes the equivalent Windows counter and describes it’s purpose. I’ve tried to define the UNIX counters and where applicable I’ve described the Windows and UNIX counters.

 

UNIX counter Windows Counter Description
Average Load* N/A The sum of the number of processes waiting in the run queue plus the number currently executing.
Collision rate N/A The total number of network collisions/sec
Context switch rate System – Context Switches/sec The rate at which processors switch from executing one thread to another. High switch rates can indicate performance problems as servers juggle multiple running applications.
CPU utilisation %Processor Time The percentage of elapsed time that the process spends executing non-idle threads.
Disk traffic %Disk time The percentage of elapsed time that the disk(s) are  busy servicing read or write requests.
Incoming packets error rate Packets received errors The number of packets received containing errors that precvent them from being delivered to a higher OSI layer protocol.
Incoming packets rate Packets received/sec The number of packets received on the network interface
Interrupt rate Interrupts/sec Average rate at which the processor receives and services hardware interrupts. Processes generate an interrupt when they finish a task and need to report that fact to the CPU.
Outgoing packets error rate Packets outbound errors The number of packets that can’t be transmitted due to errors
Outgoing packets rate Packets sent / sec The rate at which packets are sent on the network interface
Page-in rate Pages Input/sec The rate at which pages are read from disk to resolve hard page faults. Hard page faults occur when a process refers to a page in virtual memory which is not in it’s working set or available elsewhere in physical memory and has to be read from disk.
Page-out rate Pages Output/sec The rate at which memory pages are written to disk to free up space in physical memory.
Paging rate Paging rate The rate at which pages are read from disk or written to disk.  This is the sum of Pages Input/sec and Pages Output/sec.
Swap-in rate N/A The number of pages read into memory per second
Swap-out rate N/A The number of pages written out of memory per second
System mode CPU utilization Processor – %Priviledged time The percentage of elapsed time that the processor spends executing user threads (i.e. running applications)
User mode CPU utilization Processor – %User time The percentage or elapsed time that the processor spends executing priviledged or system mode threads.

 

*Average Load is not clearly defined and it is important to understand that it represents a moving average over time rather than a snapshot of actual performance. Because Load Average is the sum of two performance statistics (queue length and the number of processes currently executing) it is hard to use it to compare the performance of dissimilar systems. An Average Load which would be acceptable on one system might be unacceeptable on another. The best description that I’ve found for Average Load was “Linux Load Average – Not Your Average Average”, written by Neil Gunther of Performance Dynamics and it can be found at this URL. http://luv.asn.au/overheads/NJG_LUV_2002/luvSlides.html

Thanks to Tim Nichols of CheckPoint Technologies for his questions which encouraged me to clarify the description of Average Load.

Using MySQL instead of VTS for LoadRunner

I have two requirements for some testing which I’m currently undertaking and MySQL seems to be able to help me in both cases.

  • Firstly, I want to create data in one script and use it in subsequent scripts. When the data is “in use” I want other vUsers to use an alternate piece of data. When a piece of data has been “used” I want to flag it as such and prevent it being used in subsequent iterations.
  • Secondly, I want to write detailed information about transactions including a description of the transaction, the time it executed and the response time to a database. This will allow more detailed analysis than is normally possible using LoadRunner Analysis and allow a database of performance stats for multiple tests run over a period of time to be built up.
I know that it is possible to use Virtual Table Server (VTS) for this, but VTS has a number of limitations.
  • VTS can only read in data one row at a time. Queries must then be written in LoadRunner using C code to determine whether the data is suitable for use. This is inefficient when compared to SQL queries which execute on the server and can return data directly to the vUser.
  • VTS is not officially supported by HP (and it wasn’t supported by Mercury). MySQL isn’t supported either, but MySQL is well documented on the Internet and there is an established user community as well as large numbers of people familiar with SQL syntax.
  • VTS is an in-memory database and needs to import data before a test and export data to file before it is shut down. This makes it very easy to accidentally close VTS before data is saved and lose your test data. MySQL and other databases automatically save the database tables when they shut down which reduces the risk of this occurring.
A PDF document describing the process for using MySQL with LoadRunner, together with LoadRunner library files, the MySQL DLL and sample scripts which read and write to MySQL are available in my GitHib library.
Please feel free to comment below if you find this useful.

 

LoadRunner function write to and read data from external file

On a client site recently, I had a requirement to encrypt a small amount of data which needed to be re-used later in a LoadRunner script. The encryption was done by OpenSSL; whilst it was easy to use OpenSSL to encrypt the data in question, it proved extremely difficult to incorporate the OpenSSL functions into LoadRunner.

As a work-around, I wrote a function which writes output to an external file with a unique filename. LoadRunner can then use the external OpenSSL executable by using the system function to call an external DOS command, if the output from this command is written to an output file, the script can then read the data back in from the output file and use it later in the script.

This seems quite convoluted but it works!!

The function can be downloaded in a working sample script.