Draft statement of work


Performance of the System



Download 0.66 Mb.
Page29/34
Date28.01.2017
Size0.66 Mb.
#9693
1   ...   26   27   28   29   30   31   32   33   34

9.0Performance of the System


Each of the Sequoia benchmarks has been carefully chosen and developed to represent a particular subset and/or specific characteristic of the expected ASC workload on the Dawn and Sequoia systems, which consists of solving complex scientific and engineering problems using a variety of computational techniques.

Sequoia benchmarks serve three purposes in the Sequoia procurement. First, the benchmarks provide the Offeror with the opportunity to provide LLNS concrete data (in its RFP response) associated with the performance, reliability and scalability of the proposed systems on programmatically important applications. Offeror’s proposal response should include information of many kinds, including a report on the results of running Sequoia benchmarks on existing Offeror hardware, simulators and extrapolations to future proposed systems. In this role, the benchmarks play an essential role in the proposal evaluation process. Second, the benchmarks will be used as an integral part of the systems SWL tests (see Section 8.3.3) in order to assess the proposed systems ability to meet or exceed Sequoia performance, scalability and stability technical requirements. Third, these benchmarks will be used to assess continuous performance improvement of compiler and other critical software technologies during development and after acceptance over the lifetime of the Dawn and Sequoia systems.

While performance is an important consideration in any computer acquisition, for the ASC Program it matters greatly how improved performance is achieved. See Section 9.4.1 for a discussion of allowed modifications to the Sequoia benchmarks.

The Sequoia benchmarks described below may be executed by the Offeror and results presented in its proposal response for the purpose of measuring the execution performance and compiler capabilities of existing systems that may be proposed. In addition, these benchmark results from existing systems may be used by Offeror to extrapolate and/or estimate the benchmark performance on future proposed systems. The general requirements and constraints outlined below apply to all of the benchmark codes. Additional requirements and/or constraints found in individual benchmark readme files apply to that individual benchmark.



Although all of the benchmark results are considered important and will be carefully analyzed by LLNS during proposal evaluation, LLNS understands that Offerors are working with limited resources. The benchmarks are divided into three tiers to give Offerors the relative priority of the benchmarks. The Sequoia Marquee Benchmarks, also known as the Tier 1 codes, are designated as TR-1 requirements and Offeror may report results in the response evaluation as described in the proposal evaluation attachment. Tier 2 codes are designated as TR-2 requirements, while the Tier 3 codes are designated as TR-3 requirements. All of the Tier 3 codes are “micro kernels” that are provided as a courtesy to Offerors who wish to report results from prototype node or simulators of processor or node that are not yet available as hardware that are proposed as offerings for the Sequoia procurement. “Micro kernels” also serve as compiler challenges, especially for threading and the exploitation of Vector or SIMD hardware.

ASC Sequoia Benchmarks







Language

Parallelism

Description

Tier

Code

F

Py

C

C++

MPI

OpenMP

Pthreads




1

UMT

X

X

X

X

X

X




Marquee performance code. Single physics package code. Unstructured-Mesh deterministic radiation Transport

1

AMG







X




X

X




Marquee performance code. Algebraic Multi-Grid linear system solver for unstructured mesh physics packages

1

IRS







X




X

X




Marquee performance code. Single physics package code. Implicit Radiation Solver for diffusion equation on a block structured mesh

1

SPhot

X










X

X




Marquee performance code. Single physics package code. Monte Carlo Scalar PHOTon transport code

1

LAMMPS










X

X







Marquee performance code. Full-system science code. Classical molecular dynamics simulation code (as used)

2

Pynamic




X




X

X







Subsystem functionality and performance test. Dummy application that closely models the footprint of an important Python-based multi-physics ASC code

2

CLOMP







X







X




Subsystem functionality and performance test. Measure OpenMP overheads and other performance impacts due to threading

2

FTQ







X










X

Fixed Time Quantum test. Measures operating system noise

2

IOR







X







X




Interleaved or Random I/O Benchamrk. IOR is used for testing the performance of parallel filesystems using various interfaces and access patterns.

2

Phloem MPI Benchmarks







X




X







Subsystem functionality and performance tests. Collection of independent MPI Benchmarks to measure the health and stability of various aspects of MPI performance including interconnect messaging rate, latency, aggregate bandwidth, and collective latencies under heavy network loads.

2

Memory Benchmarks







X







X




Memory Subsystem functionality and performance tests. Collection of STREAMS and STRIDE memory benchmarks to measure the memory subsystem under a variety of memory access patterns

3

UMTMk

X



















Threading compiler test and single core performance

3

AMGMk







X







X




Sparse matrix-vector operations single core performance and OpenMP performance

3

IRSMk







X













Single core optimization and SIMD compiler challenge

3

SPhotMK

X



















Single core integer arithmetic and branching performance

3

CrystalMK







X













Single core optimization and SIMD compiler challenge.

Table 9 3: ASC Sequoia Benchmarks are categorized into three tiers of importance.


Download 0.66 Mb.

Share with your friends:
1   ...   26   27   28   29   30   31   32   33   34




The database is protected by copyright ©ininet.org 2024
send message

    Main page