This section contains information on the SPEC SFS 2014 benchmark directory structure, running the benchmark, and interpreting the benchmark metrics output generated in the summary results file.
4.1SFS Benchmark Directory Structure
The following is a quick overview of the benchmark’s directory structure. Please note that the variable “$SPEC” used below represents the full path to the install_directory, where the benchmark is installed.
The directory contains the SPEC SFS 2014 benchmark Makefile. The SPEC SFS 2014 benchmark uses the UNIX “Makefile” structure to build tools, compile the benchmark source into executables, and to clean directories of all executables. However, note that pre-built binaries for several operating systems, and therefore compilation should not be required in most cases.
The benchmark binaries for the specific environment being used are located in the “$SPEC/bin” directory if the user has built the binaries using the Makefile provided..
The benchmark pre-built binaries for various operating systems are located in subdirectories under the “$SPEC/binaries” directory.
The benchmark documentation is located under the “$SPEC/docs” directory.
The benchmark log and results files created during a benchmark run are located in the “$SPEC/results” directory.
4.2Pre-Compiled SPEC SFS 2014 Benchmark Binaries
Included in this benchmark release are pre-compiled versions of the benchmark for various operating systems at various levels. If it becomes necessary for the user to compile a version of the benchmark source for testing, a generic UNIX makefile is provided in the benchmark top level directory ($SPEC).
The makefile may be modified or supplemented in a performance neutral fashion to facilitate the compilation and execution of the benchmark on operating systems not included within the benchmark distribution. To build the software simply type: make
The Visual Studio workspace files are also provided should one need to rebuild the Windows executables. The workspace files are located in the $SPEC, and /libwin32 subdirectories. The SPEC SFS 2014 benchmark can be built with Visual Studio C++ 2010 Express.
The following is a list of the vendors and their respective operating system levels for which the benchmark workloads have been pre-compiled and included with the benchmark distribution.
FreeBSD 8.2, 10.0
Mac OS X (Tiger, Leopard, and Mavericks)
Windows 2008R2, Windows 2012, Windows 7, Windows 8.
This section briefly describes the usage of the Python based SfsManager provided with the SPEC Solution File Server (SFS) SPEC SFS 2014 benchmark suite. The SfsManager is used to run the benchmark. The results obtained from multiple data points within a run are also collected in a form amenable for ease of use with other result formatting tools.
This section does not cover the complete Client-Server environment setup in detail. It touches only on the portions currently handled by the SfsManager. For information on how to set up and run the SFS suite the reader is advised to refer to the section on configuring the SPEC SFS 2014 environment above.
A new Python SfsManager program exists to handle the execution of the SPEC SFS 2014 benchmark. The SfsManager was implemented to allow the same manager to run on Windows and UNIX. The manager uses SSH (Unix) and WMI (Windows) to communicate between the clients.
Example of SUT Validation
Before each load point, the client validates that it can perform all of the POSIX level operations that will be used during the benchmark. If the validation fails, then the benchmark will terminate, with errors collected in the log files.