Towards faster .NET: Tuning Tips

In our previous post we discussed some peculiarities of ASP.NET performance tuning. Here’s the second part for you.

System Resources

When you need to measure how many system resources your application consumes, you need to pay particular attention to the following:

  • Processor. Processor utilization, context switches, interrupts and so on.
  • Memory. Amount of available memory, virtual memory, and cache utilization.
  • Network. Percent of the available bandwidth being utilized, network bottlenecks.
  • Disk I/O. Amount of read and write disk activity. I/O bottlenecks occur if read and write operations begin to queue.

The next sections describe the performance counters that help you measure the preceding metrics.

Processor

To measure processor utilization and context switching, you can use the following counters:

  • Processor\% Processor Time

Threshold: The general figure for the threshold limit for processors is 85 percent.

Significance: This counter is the primary indicator of processor activity. High values many not necessarily be bad. However, if the other processor-related counters are increasing linearly such as % Privileged Time or Processor Queue Length, high CPU utilization may be worth investigating.

  • Processor\% Privileged Time

Threshold: A figure that is consistently over 75 percent indicates a bottleneck.

Significance: This counter indicates the percentage of time a thread runs in privileged mode. When your application calls operating system functions (for example to perform file or network I/O or to allocate memory), these operating system functions are executed in privileged mode.

  • Processor\% Interrupt Time

Threshold: Depends on processor.

Significance: This counter indicates the percentage of time the processor spends receiving and servicing hardware interrupts. This value is an indirect indicator of the activity of devices that generate interrupts, such as network adapters. A dramatic increase in this counter indicates potential hardware problems.

  • System\Processor Queue Length

Threshold: An average value consistently higher than 2 indicates a bottleneck.

Significance: If there are more tasks ready to run than there are processors, threads queue up. The processor queue is the collection of threads that are ready but not able to be executed by the processor because another active thread is currently executing. A sustained or recurring queue of more than two threads is a clear indication of a processor bottleneck. You may get more throughput by reducing parallelism in those cases.

You can use this counter in conjunction with the Processor\% Processor Time counter to determine if your application can benefit from more CPUs. There is a single queue for processor time, even on multiprocessor computers. Therefore, in a multiprocessor computer, divide the Processor Queue Length (PQL) value by the number of processors servicing the workload.

If the CPU is very busy (90 percent and higher utilization) and the PQL average is consistently higher than 2 per processor, you may have a processor bottleneck that could benefit from additional CPUs. Or, you could reduce the number of threads and queue more at the application level. This will cause less context switching, and less context switching is good for reducing CPU load. The common reason for a PQL of 2 or higher with low CPU utilization is that requests for processor time arrive randomly and threads demand irregular amounts of time from the processor. This means that the processor is not a bottleneck but that it is your threading logic that needs to be improved.

  • System\Context Switches/sec

Threshold: As a general rule, context switching rates of less than 5,000 per second per processor are not worth worrying about. If context switching rates exceed 15,000 per second per processor, then there is a constraint.

Significance: Context switching happens when a higher priority thread preempts a lower priority thread that is currently running or when a high priority thread blocks. High levels of context switching can occur when many threads share the same priority level. This often indicates that there are too many threads competing for the processors on the system. If you do not see much processor utilization and you see very low levels of context switching, it could indicate that threads are blocked.

Memory

To measure memory utilization and the impact of paging, you can use the following counters:

  • Memory\Available Mbytes

Threshold: A consistent value of less than 20 to 25 percent of installed RAM is an indication of insufficient memory.

Significance: This indicates the amount of physical memory available to processes running on the computer. Note that this counter displays the last observed value only. It is not an average.

  • Memory\Page Reads/sec

Threshold: Sustained values of more than five indicate a large number of page faults for read requests.

Significance: This counter indicates that the working set of your process is too large for the physical memory and that it is paging to disk. It shows the number of read operations, without regard to the number of pages retrieved in each operation. Higher values indicate a memory bottleneck.

If a low rate of page-read operations coincides with high values for Physical Disk\% Disk Time and Physical Disk\Avg. Disk Queue Length, there could be a disk bottleneck. If an increase in queue length is not accompanied by a decrease in the pages-read rate, a memory shortage exists.

  • Memory\Pages/sec

Threshold: Sustained values higher than 500 indicate a bottleneck.

Significance: This counter indicates the rate at which pages are read from or written to disk to resolve hard page faults. Multiply the values of the Physical Disk\Avg. Disk sec/Transfer and Memory\Pages/sec counters. If the product of these counters exceeds 0.1, paging is taking more than 10 percent of disk access time, which indicates that you need more RAM.

  • Memory\Pool Nonpaged Bytes

Threshold: Watch the value of Memory\Pool Nonpaged Bytes for an increase of 10 percent or more from its value at system startup.

Significance: If there is an increase of 10 percent or more from its value at startup, a serious leak is potentially developing.

  • Server\Pool Nonpaged Failures

Threshold: Regular nonzero values indicate a bottleneck.

Significance: This counter indicates the number of times allocations from the nonpaged pool have failed. It indicates that the computer’s physical memory is too small. The nonpaged pool contains pages from a process’s virtual address space that are not to be swapped out to the page file on disk, such as a process’ kernel object table. The availability of the nonpaged pool determines how many processes, threads, and other such objects can be created. When allocations from the nonpaged pool fail, this can be due to a memory leak in a process, particularly if processor usage has not increased accordingly.

  • Server\Pool Paged Failures

Threshold: No specific value.

Significance: This counter indicates the number of times allocations from the paged pool have failed. This counter indicates that the computer’s physical memory or page file is too small.

  • Server\Pool Nonpaged Peak

Threshold: No specific value.

Significance: This is the maximum number of bytes in the nonpaged pool that the server has had in use at any one point. It indicates how much physical memory the computer should have. Because the nonpaged pool must be resident, and because there has to be some memory left over for other operations, you might quadruple it to get the actual physical memory you should have for the system.

  • Memory\Cache Bytes

Threshold: No specific value.

Significance: Monitors the size of cache under different load conditions. This counter displays the size of the static files cache. By default, this counter uses approximately 50 percent of available memory, but decreases if the available memory shrinks, which affects system performance.

  • Memory\Cache Faults/sec

Threshold: No specific value.

Significance: This counter indicates how often the operating system looks for data in the file system cache but fails to find it. This value should be as low as possible. The cache is independent of data location but is heavily dependent on data density within the set of pages. A high rate of cache faults can indicate insufficient memory or could also denote poorly localized data.

  • Cache\MDL Read Hits %

Threshold: The higher this value, the better the performance of the file system cache. Values should preferably be as close to 100 percent as possible.

Significance: This counter provides the percentage of Memory Descriptor List (MDL) Read requests to the file system cache, where the cache returns the object directly rather than requiring a read from the hard disk.

Disk I/O

To measure disk I/O activity, you can use the following counters:

  • PhysicalDisk\Avg. Disk Queue Length

Threshold: Should not be higher than the number of spindles plus two.

Significance: This counter indicates the average number of both read and writes requests that were queued for the selected disk during the sample interval.

  • PhysicalDisk\Avg. Disk Read Queue Length

Threshold: Should be less than two.

Significance: This counter indicates the average number of read requests that were queued for the selected disk during the sample interval.

  • PhysicalDisk\Avg. Disk Write Queue Length

Threshold: Should be less than two.

Significance: This counter indicates the average number of write requests that were queued for the selected disk during the sample interval.

  • PhysicalDisk\Avg. Disk sec/Read

Threshold: No specific value.

Significance: This counter indicates the average time, in seconds, of a read of data from the disk.

  • PhysicalDisk\Avg. Disk sec/Transfer

Threshold: Should not be more than 18 milliseconds.

Significance: This counter indicates the time, in seconds, of the average disk transfer. This may indicate a large amount of disk fragmentation, slow disks, or disk failures. Multiply the values of the Physical Disk\Avg. Disk sec/Transfer and Memory\Pages/sec counters. If the product of these counters exceeds 0.1, paging is taking more than 10 percent of disk access time, so you need more RAM.

  • PhysicalDisk\Disk Writes/sec

Threshold: Depends on manufacturer’s specification.

Significance: This counter indicates the rate of write operations on the disk.

Network I/O

To measure network I/O, you can use the following counters:

  • Network Interface\Bytes Total/sec

Threshold: Sustained values of more than 80 percent of network bandwidth.

Significance: This counter indicates the rate at which bytes are sent and received over each network adapter. This counter helps you know whether the traffic at your network adapter is saturated and if you need to add another network adapter. How quickly you can identify a problem depends on the type of network you have as well as whether you share bandwidth with other applications.

  • Network Interface\Bytes Received/sec

Threshold: No specific value.

Significance: This counter indicates the rate at which bytes are received over each network adapter. You can calculate the rate of incoming data as a part of total bandwidth. This will help you know that you need to optimize on the incoming data from the client or that you need to add another network adapter to handle the incoming traffic.

  • Network Interface\Bytes Sent/sec

Threshold: No specific value.

Significance: This counter indicates the rate at which bytes are sent over each network adapter. You can calculate the rate of incoming data as a part of total bandwidth. This will help you know that you need to optimize on the data being sent to the client or you need to add another network adapter to handle the outbound traffic.

  • Server\Bytes Total/sec

Threshold: Value should not be more than 50 percent of network capacity.

Significance: This counter indicates the number of bytes sent and received over the network. Higher values indicate network bandwidth as the bottleneck. If the sum of Bytes Total/sec for all servers is roughly equal to the maximum transfer rates of your network, you may need to segment the network.

  • Protocol related counters:
    • Protocol_Object\Segments Received/sec
    • Protocol_Object\Segments Sent/sec

Protocol_Object can be TCP, UDP, NetBEUI, NWLink IPX, NWLink NetBIOS, NWLink SPX, or other protocol layer performance objects.

Threshold: Application-specific.

Significance: Protocol-related counters help you narrow down the traffic to various protocols because you might be using one or more protocols in your network. You may want to identify which protocol is consuming the network bandwidth disproportionately.

  • Processor\% Interrupt Time

Threshold: Depends on processor.

Significance: This counter indicates the percentage of time the processor spends receiving and servicing hardware interrupts. This value is an indirect indicator of the activity of devices that generate interrupts, such as network adapters.

In the next part of this article we’ll discuss how and what to measure in more detail.

You might also like