FY13 Cosmic Frontier Experimental Research Program – Lab Review Argonne National Laboratory Background Material Program Status & Plans
A.2 HACC: Hardware/Hybrid Accelerated Cosmology Code
Download 390.84 Kb.
|
- Navigate this page:
- Publications
- Selected Invited Presentations
10.A.2 HACC: Hardware/Hybrid Accelerated Cosmology CodeThe computational cosmology effort was initiated at Argonne in May 2011. Since then, a team of researchers from HEP, ALCF, and MCS Divisions led by Salman Habib and KatrinHeitmann has continuously developed HACC, the Hardware/Hybrid Accelerated Cosmology Code simulation framework. HACC is a portable, high-performance N-body solver that can exploit diverse supercomputing architectures at full scale; it also includes a comprehensive set of analysis tools. Aside from conventional parallel supercomputers, HACC has been ported successfully to IBM BlueGene/Q (BG/Q) systems and to heterogeneous platforms such as GPU accelerated systems. On Sequoia, LLNL’s BG/Q system (twice as large as Mira, the BG/Q system at Argonne), HACC has achieved nearly 14 petaflops, in a record-setting benchmark run with 3.6 trillion simulation particles. This achievement placed the team among the Gordon Bell Award finalists at SC12 [1](this award is given in recognition of outstanding achievements in high performance computing), and was highlighted in a DOE SC/NNSA press release (http://www.nnsa.energy.gov/mediaroom/pressreleases/sequoia112812). The GPU enhanced version of HACC is now running at scale on all of Titan, Oak Ridge National Laboratory’s supercomputer, currently second-fastest worldwide, and has been once again selected as a Gordon Bell finalist at SC13 [2] (the award will be decided in November). Figure 10.A.2: Zoom-in visualization of the density field in a 1.07 trillion particle, 4.225 Gpc box size simulation with 6 kpc force resolution. This HACC simulation is currently running on 32 BG/Q racks, as part of a current INCITE allocation. The image, taken during a late stage of the evolution (z=0.7), illustrates the global dynamic range covered, ~106, although the finer details are not resolved by the visualization. Over the last year, partially supported by a multi-institutional DOE HEP/ASCR SciDAC-3 project in computational cosmology (PI: Salman Habib) we haveincorporated new physics capabilities into HACC, namely dynamical dark energy models and massive neutrinos. Simulations exploring these new additions have been carried out as part of the Early Science Program (ESP) on Mira and of an Advanced Leadership Computing Challenge (ALCC) project that focused primarily on neutrinos. The results are currently being analyzed. The ALCC awards computing allocations to projects that specifically support science of interest to the DOE with an emphasis on high-risk, high-payoff simulations in areas directly related to agency missions. During the acceptance phase of Mira, the world’s largest cosmological simulation was initiated, evolving almost 1.1 trillion particles spanning a (4.225Gpc)3volume on 32 of Mira’s 48 racks. A snapshot at an intermediate redshift is shown in Figure 10.A.2. In order to complete this simulation, Habib and collaborators successfully competed for time under the INCITE program (Innovative and Novel Computational Impact on Theory and Experiment) for the current calendar year. A new INCITE proposal for three years that would start in early 2014 has been submitted; the selections will be announced in November 2013. INCITE and ALCC proposals are reviewed by science domain experts for scientific excellence as well as by computer scientists for computational readiness and are highly competitive. In our proposal, a total of 200M CPU hours per year on both Mira and Titan have been requested. This allocation will enable the simulation efforts outlined in the previous sections to support major DOE dark energy and CMB experiments. In addition to the successful ports to Mira and Titan, other new features have been implemented into HACC to ensure best performance on very large scales, including locally adaptive time stepping, improved load balancing, a new I/O scheme including efficient compression algorithms, and new analysis capabilities. A comprehensive description of HACC is given in Ref. [3]. Publications
Selected Invited Presentations
Directory: hep hep -> Simulation of Cosmic Muons in the atlas tile Calorimeter hep -> Have high stiffness hep -> Minutes of the hep-scd board of supervisors meeting Download 390.84 Kb. Share with your friends:
|