Server Performance Issues:Bottlenecks and Enhancements

1. Server Performance Issues:Bottlenecks and Enhancements By Yemi Ogunrombi IS8030 - Integrated Computing Systems

2. Introduction • The subject of server performance is very broad and a major on-going issues in the use of IS/IT systems. • Generally performance issues are detected usually when users experience slow response from their IT/IS systems. This can be due to one or more reasons.

3. Introduction • Commonly most server systems run either Unix, Novell Netware, NT. The focus in this presentation is on Unix • Depending on the infrastructure of the organization, there can be one or more servers with a number of users connected via varying types of network topologies.

4. Monitoring your System • There are two main strands to understanding a system • The first entails producing a regular collection of reports on the machine usage • The second entails taking a user viewpoint. • What applications do the users run? • Could a job be deferred until overnight, or is it important for the user to obtain the output within the next hour? • Understanding the business requirement is important at this stage

5. Monitoring your system • In Unix there are 7 major resource types to be monitored and tuned for performance • CPU • Memory, Virtual Memory • Disk Sub-system, Interface, Controller • Communication Lines • I/O • Network Time • Applications Programs

6. First Considerations • When investigating a possible performance problem: • First, determine whether the system is under stress or only lightly loaded. (The sar -q, uptime and ruptime commands will show averages of system load) • If your system is working hard, you then need to identify which processes are the most demanding by getting the percentage of the CPU and physical memory used by each process (using the top or ps commands)

7. First Considerations • Second, determine how hard the CPU is working (iostat will report the percentage of time that your CPU is idle) • If the CPU is lightly loaded, the CPU will spend some time being idle. • If the system is stressed, and the CPU has significant idle time, then there is performance problem. The CPU may be waiting whilst the system is paging, or there may be I/O or network bottlenecks.

8. Memory Performance • A loaded system with significant CPU idle time could be caused by memory shortages. (Memory usage can be monitored by running vmstat or sar -wpgr). • If there is a high level of swapping and paging, then the system is short of memory and is moving whole processes to disk • When a Unix Server is configured, you allocate some part of your disk space to be used for swapping and Paging

9. Memory Performance • Possible Solutions (The heart of your Unix operating system where your operating system configuration is defined is the Kernel) • Remove configuration defined for unused devices • Configure your system for the maximum number of concurrent users that will be using the system. The Kernel has internal tables. The size of these tables depends on the maximum number of users the system can support.

10. Memory Performance • Possible Solutions • Minimize buffer cache if this will not adversely impact your I/O performance (the buffer cache is the area of temporary storage for data moving to or from the disks and is used to optimize disk access by minimizing transfers) • Share Paging and Swapping activities evenly across the system by distributing swap space over as many disks as possible • Probably the easiest solution is to just buy more memory

11. Disk Performance • Problems with I/O subsystem can severely affect system performance • Memory problems resulting in high paging and swapping will increase I/O traffic, therefore it is essential to solve memory problems before I/O issues. • Issues might be due to unbalanced disk activity, therefore there is need to redistribute your I/O activities more evenly among the disks

12. Disk Performance • Reorganization of filesystems to ensure that heavy activities are on the fastest disk drive and uses faster controllers • Large critical files should be places on a file system with a large blocksize. Commonly accessed files should not be placed in deeply nested sub-directories • Disk fragmentation causes poor I/O performance

13. Disk Performance • The number of disks that are used should be looked into. If a 10GB of disk space is required, it is better to have five 2GB disks than one 10GB disk. With more disks, you have more spindles spinning and thus more data can be written to the disks at the same time. • How these multiple disks are arranged is also important. • The RAID technology helps to address some of the disk problems such as disk stripping and parallelism

14. Tuning the Application • The way the application code is written is one of the primary and common causes of performance • To detect this, it is important to track which jobs are resource intensive and run a trace on the process/transaction • If the process is inevitably resource intensive, then if it can be run as a batch during low machine usage times

15. Tuning the Application • Improve or rewrite the code • This is common in database applications where creating an index on a large table could cause querying it to change from half an hour to seconds

16. CPU Performance • With the advent of faster processors, typically the CPU itself is no longer the problem • The amount of level 2 cache can have significant effects on performance

17. CPU Performance • However, if the CPU has near 100% utilization, but users still have poor response times then the following alternatives should be considered • Reschedule CPU intensive jobs • Limit the CPU time a process can use with the limit command • Separate the users and application unto separate servers • Upgrade the system

18. Network Performance • If previous check does not solve the problem, the issue could be the network. (use netstat, netpmon, nfssat shows network with input and output packets and number of collisions) • It might be necessary to reorganize the network • There might be slow network components, cards, cables etc • If communications is over national telecommunications systems, then that could contribute to the performance

19. Conclusion • Performance issues might be due to one or more bottlenecks • It is important that software applications are tested for potential performance issues and tuned before implemented. • RAID technology is advised for disk arrangement • Thorough planning can help avoid a lot of performance issues

20. References • www.circle4.com/jaqui/papers/webunuk.html • www.tssp.co.uk/documents/supplements/unixsyssper.htm • www.cssnw.com/CPUPERF.htm • www.metron.co.uk/tips/unix.html • www.networkcomputing.com/shared/server3.html