Cse 221 Project – Winter 2003 System Performance Measurements on RedHat Linux 0 and Windows xp 1 Project Members

Download 205.68 Kb.

Page	8/8
Date	29.01.2017
Size	205.68 Kb.
	#11568

1 2 3 4 5 6 7 8

6 Arithmetic Operations

This test measures the number of integer or float operations that can be done in a second. Another test compares the execution time of integer operations with and without optimizations.

6.1 Integer and Float Operations

6.1.1 Metric Description

Integer and float arithmetic operations of addition, subtraction, multiplication and division were tested in this experiment.

Each operation was executed in a “for” loop with the operation being performed once per iteration. N iterations of these loops were timed. These timings include the looping overhead (mainly an increment and a compare operation), which is common to all the tests, and hence can be ignored if only a pure comparison is being made. O4 compiler optimization was used to ensure that while the iterations executed, all values stayed in the registers. This would isolate the test from the overheads of memory accesses.
The loops were structured as shown below, with all variables declared as integers (or floats). N is always declared as an integer. Several runs were taken and the average execution time was used to calculate the approximate number of iterations executed per second.

/*** Sample Arithmetic Operation Test. ***/
N=100000000;

/* Integer Addition */

GetTime(1);

for(i=0;i

acc = i + inc;

difftime = GetTime(2);

printf("Int Add %lf MOps/sec\n",N/(difftime*1e+6));

Figure 31 : Code Segment for Arithmetic Operations

6.1.2 Results and Analysis

Figure 32 : Graphs for Arithmetic Operations

These numbers show us that overall, integer operations are faster than float operations. Operations speed up with compiler optimization, with the exception of integer division. Windows and Linux gave comparable numbers for all these tests.

7 Linux Vs Windows

In this section, the performance of Linux and Windows are compared for the same measurements of memory access, disk access and processes. Please note that Windows measurements were done on ‘cygwin’ environment which might attribute to some unexpected results. The same compiler was used for compiling the programs for the test. One thing that can be noted for memory and disk performance is that Windows seems to use the cache more effectively than Linux.

7.1 Memory

Memory – Cache Vs RAM Vs Swap Disk

As excepted there is a considerable drop in bandwidth for both memory read and write for both Linux and Windows when the size of the allocated memory is nearly the same as the cache size. In both cases of read and write the bandwidths of Windows drops much earlier than Linux.

Figure 33 : Graphs comparing Linux and Windows memory access bandwidths
Code Plus Data – Different Levels of Optimizations

Effects of Code Optimization on Cache Vs RAM memory access times

In our test for comparing memory access times between the cache, RAM and the disk, and for estimating the cache-resident code size, different levels of compiler optimization showed different trends. This is shown in the following plots.

Figure 34 : Graphs for Code plus data with different levels of optimization on Linux

Figure 35 : Graphs for Code plus data with different levels of optimization on Windows
From these plots it can be seen that without compiler optimization, there is no apparent difference in performance between the cache and the RAM. With any level of optimization, this divide is apparent. At lower levels of optimization, the performance curves are noisier than those for higher levels of optimizations. This is an indication of the fact that the code gets more compact with higher levels of optimization, thus resulting in less contention between the cache and the RAM and hence smoother trends.

Figure 36 : Graphs comparing Linux and Windows with O3 optimization level

The same compiler was used on both Linux and Windows and a rather consistent feature observed is that the Windows curves were smoother at the cache size and did not start decreasing much before the cache size, as was observed for Linux.

Looping and Sub-Looping

Figure 37 : Graphs comparing the Sub-Looping between Linux and Windows

These graphs show that performance of Linux and Windows are comparable for this experiment. The main difference is that the performance of Linux degrades before the chunk size is equal to the size of the cache. This could be due to the presence of other resident code in the Linux cache.

7.2 Disk IO

open-close-read-write
The performance of synchronous and asynchronous reads are different for Linux and Windows.
In the case of synchronous reads, Linux performs better than Windows. The bandwidth of Linux drops way beyond the size of the cache while there is drop in bandwidth for Windows at this point.
Windows performs better than Linux for asynchronous reads. The bandwidth for Linux drops considerably before the size of the cache, while Windows bandwidth drops around this point.

Figure 38 : Graphs comparing disk read bandwidths for Linux and Windows

The reads using synchronous I/O definitely show better performance.

Figure 39 : Graphs comparing disk write bandwidths for Linux and Windows

The performance for synchronous and asynchronous writes are comparable for Linux and Windows.

The bandwidth for synchronous writes drops for Windows at the cache size while that for Linux drops before the cache size. Windows performs better than Linux for synchronous writes at any size. Beyond the size of the cache, the performance of Windows continues to prove while that of Linux goes down.
The performance of asynchronous writes for Linux and Windows are comparable except when there is a drop in bandwidth for Windows at the cache size.
Sequential Vs Random

The performance of Sequential access for Linux is better than Windows for chunk sizes less than 64 KB. Beyond this size the sequential access bandwidths for both Linux and Windows are comparable.

The random access is similar for both Linux and Windows for all chunk sizes. For chunk sizes less than 64 KB, Linux performs better than Windows and vice versa for chunk sizes greater than 64KB.

Figure 40 : Graphs comparing Sequential and Random disk access between Linux and Windows
Sequential and Random For Page Read Ahead
The break for page-read ahead is observed only for Linux and not on Windows. This could be an indication that the page-read ahead limit is a feature of each operating system.

Figure 41 : Graphs for comparing the disk accesses for Page Read Ahead between Linux and Windows

7.3 Process

Inline Vs Function Vs Recursion

Figure 42 : Graphs comparing the times for different call types on Linux and Windows

For the different call types, Linux performs better than Windows. This can be attributed partly to the ‘cygwin’ environment on which it is know that the fork implementations are not optimal.

8 Conclusion

This project has been an exercise in designing metrics to test certain features of an operating system. We have probed into some of the mechanisms that control memory access, disk I/O, and process management and have tried to interpret our observations based on what we know about the corresponding operating system algorithms. We have also seen differences in behaviour while comparing performances with and without compiler optimization and have shown that in some cases, a careful choice of compiler optimization, can result in better performance. All these tests have been at the user level and we have tried to correlate our observations with documented details of the operating system, to see how much they are actually visible to a user program.

Appendix 1

While working on this project we came across a few interesting/surprising things that we did not know about earlier.

Inline implementations of code run faster than implementation by function call, without compiler optimization. With compiler optimization however, sometimes function call implementations can perform better than inline implementations.
In simple ‘for’ loops with integer loop indices, the data type of the limiting variable is a significant factor in the performance of the loop. Also sometimes, higher levels of compiler optimization are not as reliable as one would expect – we saw this while trying to use a global variable and a high level of compiler optimization, and although the global variable was getting updated in the code, it did not get updated in the memory and gave numerically wrong results.
RedHat Linux version 8.0 does not have an explicit file cache. It uses the RAM as a file cache, and stores data as dirty pages – writing to disk only periodically, or when a process requests more memory than is currently available as clean pages in the RAM.
On Linux, a cron job runs that triggers ‘updatadb’ every few hours. This is a version of ‘locate’ which organized file metadata and rearranges files on disk to maximize file access speeds and minimize fragmentation on disk.
If a process does repeated I/O – large number of repeated disk I/O system calls without any buffering (reading byte by byte), the process can get trapped in the kernel mode where it is just waiting for hardware I/O. This sends the process into a ‘disk sleep’ state. This could be related to the ‘standby spindown timeout’ setting via ‘hdparm’ which could send the process into this disk sleep mode. If a process went into this state – it took extremely long to get out of it. Even a small amount of buffering was enough to get around this.
Zombie processes are created if forked children processes are explicitly terminated and not ‘waited’ for by the parent. We ran into this when we accidently forked processes and after terminating them explicitly, did not have a ‘wait’ in the parent. This showed us that the maximum number of processes that can be forked is approximately 1200.
Working with the Cygwin UNIX Environment on Windows is the first time we have probed the system issues of a virtual machine implementation. From the documentation on the Cygwin web-site, we learnt the basic ideas behind how Cygwin emulates a UNIX shell on top of Windows, and what features of Linux have been ported efficiently and what not. Cygwin emulates a kernel by creating a shared address space for all its processes and this maintaining file and process information the way Linux does. Process creation meant the usage of part of this shared space with metadata stored to emulate the file system. Forking is expensive and uses a non copy-on-write policy, with a large number of explicit parent-child context switches. Regarding performance – processes are said to perform slower on Cygwin than on Linux, but it is not sure whether this can be attributed to the inefficiencies of Cygwin or the difference in the Unix and Windows Operating Systems.

Some additional performance tests that we could think of but have not implemented are as follows.

A test to compare context switch times and process execution times, with sets of processes having different controlled priorities. This could have been implemented by forking processes and forcing them to multiplex by performing blocking communication between each other via a pipe, and then observing what happens when their priorities are changed and controlled using kernel calls like ‘nice’.
A test that uses shared memory with multiple process accessing overlapping regions. This if designed properly, could be used to probe the system and observe the effect of how shared memory is handled via locks and monitors.
Multithreaded implementations of a process could have been compared to other implementation schemes. This could have been a part of the comparison between inline, function call, recursive and forked implementations of a simple nested computation.
A test to measure or observe the effects of interrupts.

References

[1] Maurice J. Bach, “The Design of the Unix Operating System”

[2] Andrea Arpaci-Dusseau and Remzi Arpaci-Dusseau, “Information and Control in Gray Box Systems” ,Proc, Eighteenh ACM Symposium on Operating Systems Principles, pp, 43-56,October 2001.

[3] J.K.OusterHout,”Why Aren’t Operating Systems Getting Faster as Fast as Hardware ?”,Proc, USENIX summer conference,pp.247-256, June 1990.
[4] Frigo, Leiserson,Prokop,Ramachandran,”Cache Oblivious Algorithms”, MIT Laboratory for Computer Science
[5] http://cygwin.com/usenix-98/cygwin.html
[6] Windows Performance Monitor
[7] RedHat Linux 8.0 manual pages

System Performance Measurements March 14, 2003 - -

Download 205.68 Kb.

Share with your friends:

1 2 3 4 5 6 7 8