Simulation-based engineering and science

de Leeuw, N.H., C.J. Nelson, C.R.A. Catlow, P. Sautet, and W. Dong. 2004. Density-functional theory calculations of the adsorption of Cl at perfect and defective Ag(111) surfaces. Phys. Rev. B, 69:045419.

Delgado-Buscalioni, R., P.V. Coveney, G.D. Riley, and R.W. Ford. 2005. Hybrid molecular-continuum fluid models: Implementation within a general coupling framework. Philos. Trans. R. Soc. A-Math. Phys. Eng. Sci., 363:1975-1985.

Lancaster, R.W., P.G. Karamertzanis, A.T. Hulme, D.A. Tocher, D.F. Covey, and S.L. Price. 2006. Racemic progesterone: predicted in silico and produced in the solid state. Chem Commun, 4921-4923.

Lancaster, R.W., P.G. Karamertzanis, A.T. Hulme, D.A. Tocher, T.C. Lewis, and S.L. Price. 2007. The polymorphism of progesterone: Stabilization of a 'disappearing' polymorph by co-crystallization. Journal of Pharmaceutical Sciences 96:3419-3431.

Nekovee, M. 2007. Worm epidemics in wireless ad hoc networks. New J Phys, 9:189.

Price, S.L. 2008. From crystal structure prediction to polymorph prediction: Interpreting the crystal energy landscape. Phys. Chem. Chem. Phys., DOI: 10.1039/b719351c [in press].

Sushko, P.V., A.L. Shluger, and C.R.A. Catlow. 2000. Relative energies of surface and defect states: Ab initio calculations for the MgO(001) surface. Surface Science 450:153-170.

To, J., A.A. Sokol, S.A. French, and C.R.A. Catlow. 2007. Formation of active sites in TS-1 by hydrolysis and inversion. J. Phys. Chem. C 111:14720-14731.

Site: University of Cambridge Centre for Computational Chemistry

Email: df246@cam.ac.uk

Prof. Michiel Sprik

Email: ms284@cam.ac.uk

Dr. David Wales

Email: dw34@cam.ac.uk

Dr. Ali Alavi

Email: asa10@cam.ac.uk

Theoretical Chemistry at Cambridge traces its origins back to the appointment of Prof. Lennard-Jones in 1933. One of his students, John Pople, was the 1998 Nobel Prize winner in chemistry for developments in electronic structure theory. The present head of the centre, Prof. Frenkel, has just taken over from Prof. Hansen, who had been leader of the sector for the past decade. The Centre presently has 6 academic staff, 2 Royal Society Fellows, a number of distinguished emeritus faculty, and approximately 30 graduate students and postdoctoral fellows. It is housed in contiguous recently renovated space within the main chemistry building.

The research activities of the academic staff span most of modern theoretical chemistry, with significant overlap into areas of condensed matter physics and biology. Prof. Frenkel is a leading developer and applier of condensed phase molecular simulation methods, with present focus on bio-molecular recognition and motor action. Prof. Sprik performs condensed phase simulations using ab initio molecular dynamics, with present focus on modeling redox chemistry. Dr. Wales is interested broadly in the characterization and exploration of potential energy surfaces, including applications to protein folding and glass formation. Dr. Alavi works on fundamental problems in electronic structure theory, including a novel formulation of quantum Monte Carlo in determinant space. Dr. Althorpe works on quantum dynamical methods and applications, and Dr. Vendruscolo works on computational biology with a focus on protein folding and misfolding.

Cambridge University has a large computer cluster, with more than 2300 processors, which are available on a recharge basis ($0.14 per cpu hour). As a result of the recharge system, most researchers within the center utilize their own clusters (some of which have over 200 processors). Some projects are not computer-intensive and just make use of desktop computing resources.

The WTEC visit to the Centre was very short, but during the discussion, the following issues concerning present trends in simulation-based engineering and science were raised:

• 
  The Centre is able to recruit first-rate students and postdocs, which is crucial for its continued success. The WTEC team’s hosts emphasized that they are far more often limited by brain power in their research than by computer power.
  
• 
  For those aspects of research that depend upon computing, the team’s hosts felt that one area in which they were particularly fortunate at present is having a first-rate computer officer within the center to provide dedicated support.
  
• 
  There are significant funding challenges at present. To run a larger group requires a “patchwork quilt” of multiple grants and a strong focus on grant writing. The direct support of graduate students by the department has diminished greatly (about a factor of 4 less today than it was a decade ago) to now amount to only about 0.1 student per faculty member.
  
• 
  Several sector members work on or use large codes, whose size is beyond the ability of small groups to develop or even maintain. This means that close associations are needed with either an active open source development effort or an active commercial effort. The number of British “CCPs” (computational chemistry projects), which have been one mechanism for developing community codes in the UK, seems to be diminishing.
  
• 
  Some groups produce large amounts of data in their research. At present they take the view that such data is disposable, on at least some timeframe, and therefore do not make any special efforts to manage or maintain it. It can be far more cheaply regenerated in the future if that should be required.
  

Prof. Raymond E. Goldstein, Schlumberger Chair in Complex Physical Systems

Dr. Stephen Eglen

Dr. Ron Horgan

Dr. E.P.S. Shellard

Prof. A. Iserles

The Department of Applied Mathematics and Theoretical Physics (DAMTP) at the University of Cambridge was founded as a department in 1959 by George Batchelor, but its lineage from the 17^th century remarkably forms the summary of the history of applied mathematics and theoretical physics: Newton, Stokes, Clerk, Maxwell, Rayleigh, Babbage, Eddington, Dirac, G.I. Taylor, Jeffreys, and Lighthill. The first Lucasian Professor was Sir Isaac Newton, and this oldest professorship in mathematics (established 1663) is currently held be Prof. Stephen Hawking. Today, the DAMTP research areas include astrophysics, geophysics, fluid and solid mechanics, mathematical biology, quantum information, high-energy physics, relativity, and cosmology. The SBES theme runs through a common core of activities in applied and computational analysis. These groups are loosely defined; many staff contribute to more than one area as well as participating in a number of collaborative/multidisciplinary activities at the Center for Atmospheric Science, Center for Micromechanics, Institute for Theoretical Geophysics, Institute for Aviation and the Environment, Center for Quantum Computation, Cambridge Computational Biology Institute, COSMOS Supercomputer, BP Institute, Millennium Mathematics Project, and the Cambridge eScience Center. DAMTP staff members are also active participants in the High-Performance Computing (HPC) Facility.

The department has 50 academic staff (faculty), 70 research staff, 30 support staff, and 100 PhD students. At the undergraduate level (years 1–4), 900 students are enrolled in the Mathematics Tripos (shared with pure math). DAMTP is also responsible for the mathematics curriculum of 600 students enrolled in the natural sciences. The computational biology program (1-year M.Phil. degree) has 25 students. The department is housed in a spacious grounds and a new set of buildings (completed in 2002) at Cambridge’s Centre for Mathematical Sciences, along with the physical sciences library, the Center for Math Sciences, and the Isaac Newton Institute.

Computing Hardware

The operation of the HPC facility is noted in the report on the Cambridge University Centre for Computational Chemistry site visit. But the WTEC visiting team’s visit to DAMTP was an opportunity to learn about the history of the decision-making of the faculty in the cosmology group when they opted to purchase a high-end SGI/Altix SMP (shared memory) machine. The SMP/shared memory architecture has proven to be very popular with users, and time on this machine (COSMOS) is in very high demand.

After a succinct overview of the department by Prof. Haynes, the bulk of the WTEC team’s visit to DAMTP was devoted by presentations representing the SBES activities of several selected but representative research groups. Brief summaries of and comments on those presentations follow.

These research activities continue the department’s strong tradition in fluid mechanics of biological systems, most notably the legacy of Sir James Lighthill. The WTEC team saw experimental, computational, and theoretical elastohydrodynamic studies of microorganism swimming, but also learned of new efforts in elucidating the evolutionary transition to multicellularity. Free-boundary problems in the precipative growth of stalactites represented current research in multiscale physics. The use of a state-of-the art experimental apparatus in microscopy and micromanipulation represented a significant but welcome break from the well-known DAMTP tradition of fluid mechanics experiments on a shoestring budget.

For solid mechanics, Prof. Haynes presented the computer simulation results (animation file provided) for crack propagation as an example of multiscale SBES modeling with molecular scale resolution at the crack tip (CASTEP) and continuum methods in the far field.

This presentation discussed the current research themes of this group: computational molecular biology, cancer biology, gene expression (data generation, storage, analysis), disease dynamics, and computational neuroscience, as well as a new education initiative, the M.Phil. in computational biology. The most important quest of the “post genomic” era (“sequencing of the human genome was the easy part”), is understanding the organizational principles or how the biological parts work together. The presentation showed the visualization of microarray data (provided by Prof. Simon Tavaré and illustrating the multidisciplinary collaboration with his Cambridge Computational Biology Institute), e.g., to examine molecular signals for cancer. In computational neuroscience, Prof. Eglen described the Blue Brain project, a joint effort between IBM and Brain Mind Institute. This project hopes to elucidate the workings of the brain with computer models running on a BlueGene/L. The immediate task is the simulation of the workings of one neocortical column, which involves simulating 100,000 neurons. A million-fold increase in computational power can be applied towards the buildup of a model of the entire brain. The presentation then turned to some activities of the EPSRC’s “life sciences initiative” and the pilot e-Science grant CARMEN (code analysis, repository, and modeling for e-Neurosciences, at UK₤4.5 million). The UK also contributes UK₤500,000 over three years to the International Neuroinformatics Coordinating Facility (INCF). Prof. Eglen concluded with a list of major issues facing SBES in biology: the diversity and (un)reliability of experimental data, choice of model organisms, difficulty and cost of code maintenance for large scale simulations, and the need for improved visualization techniques.

(This report author makes the following observations on the topic.) Lattice-QCD computations in particle physics involve multidimensional integrals after inversion of poorly conditioned matrices, and slowly converging sums. The computational resolutions of physical interest take on the order of 1000 years at Tera-FLOP speeds, and thus to no surprise, members of this community show up on the “top ten users” lists for HPC facilities. The pending transition to petascale computing is of great importance to this community: results obtained on the order of one year for current targets in lattice resolution scales. The experiences of this community are of importance to our SBES study as the numerical methods share common algorithmic roots with molecular simulations for material science research.

(This report author makes the following observations on the topic.) A major goal of cosmology is to explain the present state of the universe in terms of the processes that happened early in its history. The “Big Bang” model, well known to the general public thanks to science outreach efforts, makes a number of predictions that can be tested with a combination of astronomical observations and computer simulations. Although these SBES efforts are far removed from the three primary thrusts of the WTEC SBES study, significant advances in HPC hardware and algorithms are motivated and generated by this community. A good part of the presentation and subsequent discussion focused on recent experiences (at Cambridge and beyond) with shared memory SMP in the form of a significant investment in the SGI Altix (COSMOS). The COSMOS facility was founded in 1997 and evolved through several generations of supercomputers, with the most recent upgrade being to the Altix in 2003. These experiences are relevant to possible future directions for the “Track 2” and “Track 3” machines of the NSF Office of Cyberinfrastructure HPC roadmap.

This research group complements the strong tradition in DAMTP of constructing solution methods for specific computational problems that arise from applications of mathematics with the more general machinery of numerical analysis, i.e., investigation of algorithms for fundamental mathematical calculations that are common across many applications and theoretical work on mathematical issues that are vital to the success of algorithms. In the discussion on emerging trends relevant to SBES, Prof. Iserles noted the increasing importance of understanding highly oscillatory solutions to PDEs.

Historically, DAMTP has possessed a singular tradition of excellence in applied mathematics that extends to its SBES applications. DAMTP today has not lost any of its traditional strengths but has gained new capabilities in the form of multidisciplinary collaborations with new institutes in the mathematical, physical, and biological sciences on the University of Cambridge campus, strong partnerships with leading multinational companies (BP, Rolls Royce, Schlumberger, and others), and leadership roles in EU framework projects. At the risk of wearing out the term “multiscale,” it must be mentioned that DAMTP has state-of-the-art facilities at multiple scales: from the new building and grounds to experimental apparati and instruments in the laboratory. From our past observations and expectations that algorithmic advances are perhaps even more important than hardware advances in the push to multicore petascale computing, DAMTP can be expected to play an important role in shaping the SBES landscape in the petascale era.

Department of Applied Mathematics and Theoretical Physics. 2005? Department Guide. Cambridge: DAMTP.

Cosmos website: http://www.damtp.cam.ac.uk/cosmos.

Markram, H. 2006. The Blue Brain project. Nature Reviews Neuroscience 7:153-160.

Site: University of Cambridge Theory of Condensed Matter Group

Cavendish Laboratory

J.J. Thomson Avenue

Cambridge CB3 OHE, UK

http://www.tcm.phy.cam.ac.uk/
Date Visited: February 26, 2008
WTEC Attendees: P. Cummings (report author), K. Chong, M. Head-Gordon, S. Kim
Hosts: Professor Mike C. Payne, Head, Theory of Condensed Matter Group

Email: mcp1@cam.ac.uk

Professor Richard J. Needs, Theory of Condensed Matter Group

Email: rn11@cam.ac.uk

Website: http://www.tcm.phy.cam.ac.uk/~rn11/

BACKGROUND

The Theory of Condensed Matter Group is housed in the Cavendish Laboratory, home of the Physics Department, at the University of Cambridge. The focus of the group is to predict the properties of materials from first principles—i.e., in one form or another, solving the Schrödinger equation. The group members refine and develop new calculational tools and apply them to problems in physics, chemistry, materials science, and biology. Several widely used codes, including the first-principles total energy pseudopotential code CASTEP (Cambridge Sequential Total Energy Package, http://www.castep.org), have originated within this group.

CASTEP is an interesting case in point with respect to code development in the European environment. Because it is developed within the UK by academic and government laboratory employees (at Daresbury Laboratory), it is available free of charge to UK academics. For other users, CASTEP is licensed to Accelrys Software, Inc. (http://www.accelrys.com), which sells CASTEP as part of Materials Studio. This means that CASTEP, which is free to academic and government laboratory users in the UK, costs real money for U.S. academic researchers.

The WTEC delegation met with two members of the Theory of Condensed Matter group, Professors Mike Payne and Richard Needs. Mike Payne is currently Head of Theory of the Condensed Matter Group and has worked on first-principles total energy calculations since 1985; he is the principle author of CASTEP. He was awarded the 1996 Maxwell Medal and Prize by the Institute of Physics and gave the 1998 Mott Lecture. He is responsible for many of the technical developments that have led to the widespread adoption of the total energy pseudopotential technique and has pioneered the application of this technique to a wide range of scientific problems in physics, chemistry, materials science, earth sciences and, most recently, biology.

Richard Needs has worked on a wide range of complex systems such as surfaces, interfaces, defects, and clusters, mainly studying structural properties, including phase transitions and excitation energies. He has used a variety of computational techniques, including density functional theory methods, many-body perturbation theory, and quantum Monte Carlo methods. In recent years he has been developing continuum fermion quantum Monte Carlo methods and applying them to problems in condensed matter. He and his group have developed the CASINO quantum Monte Carlo code that is now used in a number of groups around the world.

R&D ACTIVITIES

Richard Needs described his main research focuses as solving the many-body Schrödinger equation using statistical methods, predicting and understanding the optoelectronic and physical properties of nanoparticles and solids, applications to ultracold atoms, and development of the CASINO quantum Monte Carlo code (http://www.tcm.phy.cam.ac.uk/~mdt26/casino2.html). Quantum Monte Carlo methods are stochastic wave-function-based approaches that include direct treatment of quantum many-body effects. They are computationally more demanding than density-functional theory (DFT) approaches; however, they serve as benchmarks against which the accuracy of other techniques may be compared (Foulkes et al. 2001). CASINO is available free of charge to academic and nonprofit users. One of the recent focuses of the Needs group, highlighted in the presentation to the WTEC team, has been using DFT methods and random search to discover new structures for materials at high pressure, including silane (SiH₄) and hydrogen (Pickard and Needs 2006, 2007). Some of these structures have now been found experimentally.

Mike Payne focused on the program ONETEP (Order-N Electronic Total Energy Package) (Haynes et al. 2006) (http://www.onetep.soton.ac.uk), which he is developing with coworkers Peter Haynes, Chris-Kriton Skylaris, and Arash A. Mostofi. ONETEP achieves order-N scaling (i.e., computational cost that scales linearly, in the number of atoms) through the use of non-orthogonal generalised Wannier functions to represent electron density instead of orthogonal extended wavefunctions used in molecular orbital methods. DFT methods typically scale as the cube of the system size,, and molecular orbital methods as -. An example of calculating the ground state structure of DNA exhibited linear scaling in the number of atoms in the DNA; other examples are provided by Skylaris et al. (2008).

The other research area highlighted by Payne was multiscale modeling of dynamics of materials, specifically crack propagation in graphine sheets. The technique used is called learn on the fly (LOTF) (Csanyi et al. 2004, 2005). Rather than construct a new hybrid Hamiltonian that combines different models (as is frequently done in other hybrid approaches that combine first-principles and atomistic molecular dynamics simulation), the LOTF method uses a unique short-ranged classical potential parameters whose parameters are continuously tuned to reproduce the atomic trajectories at the prescribed level of accuracy throughout the system.

The Theory of Condensed Matter Group is clearly among the forefront groups in materials simulation, both within the UK and internationally, with several signature codes to its credit (CASTEP, ONETEP, and CASINO).

When asked what were the conditions that have made it possible in the UK (and Europe) to develop these codes while this has not happened in the United States, the surprising opinion offered was that the traditional scarcity of resources within the UK and Europe has made it almost obligatory for groups to collaborate on code development. Collaboration of this nature is endemic in the European system and is actively funded. By contrast, in the U.S. materials modeling community, there is much pressure within promotion and tenure processes to actively discourage collaboration until a faculty member reaches a senior level, by which time the U.S. faculty member has to consider the future careers of his/her students and post-doctoral researchers, again mitigating against collaboration.

Looking to the future, Payne and Needs did not view the number of people coming into the field as big an issue of concern as the declining programming skills of incoming graduate students, being addressed to some degree by the educational programs at centers for scientific computing established at the universities of Warwick (http://www2.warwick.ac.uk) and Edinburgh (http://www.epcc.ed.ac.uk), and by workshops hosted by the Collaborative Computational Projects (CCPs, http://www.ccp.ac.uk) at Daresbury Laboratory. Their view is that the Engineering and Physical Sciences Research Council (http://www.epsrc.ac.uk) is no longer funding code development; in particular, the CCPs are now funded only for code maintenance and training, not for code development. A long-term problem for the development of materials modeling codes is that the expectation of funding agencies and university administrators is for “sound-bite science,” publishable in the highest-profile journals. Software engineering is incremental by nature, and these “sound-bite science” expectations mitigate against it. Finally, it is too early to tell whether the reorganization of the UK science laboratories under the new Science and Technology Facilities Council (http://www.scitech.ac.uk/) will be beneficial for the development of materials modeling codes.

In summary, Payne and Needs see future challenges for continuing efforts to build large codes, unless there is broad recognition of the value that these codes bring to the scientific enterprise.