Simulation-based engineering and science

Eni SpA is one of the largest integrated energy companies in the world, focusing on finding, producing, transporting, transforming, and marketing oil and gas. Eni has the biggest market capitalization in Italy, with overall sales exceeding €86 billion, profits in excess of €9 billion, and employing more than 75000 people worldwide. Eni is composed of seven major divisions: Exploration & Production, Gas & Power, Refining & Marketing, Petrochemical, Engineering & Construction, and Corporate, as well as several affiliated and subsidiary companies, such as Snamprogetti SpA, Snam Rete Gas, Saipem, and Polimeri Europa.

Eni R&D activities are housed in the Refining & Marketing Division, and the strategic development activities are part of the Corporate Division. Eni Research & Development is one of the major centers of excellence for industrial research in Europe, with annual R&D expenditures of approximately €1.4 billion. It works in the entire oil and gas technology chain and in renewable sources, while safeguarding the environment, with a sustainable development perspective. The focus areas for technological innovation are: upstream oil and gas; natural gas conversion and bituminous sources; downstream gas; downstream oil—refinery and petrochemical processes; downstream oil—products development (fuels, lubricants and specialties); advanced energy systems and renewable sources (photovoltaic); and environment and sustainability.

Eni Research & Development operates through all stages of innovation, from technology monitoring and scenario studies to applied research, as well as from the development and evaluation of technologies to the commercial exploitation of R&D results. Examples of research projects involving modeling and simulation include design of nanomaterials for catalysts and solar cells (nanowires), simulation of recovery process, noise transmission, and environmental impact studies in the Health, Safety, and Environment domain. A recent project involving simulation of product shape selectivity by zeolites produced an efficient nonquantitative screening method for the selection of a suitable alkylation catalyst. Snamprogetti, which is a former subsidiary of Eni, focuses on simulation and modeling for structural analysis, CFD, and environmental issues, such as migration of contaminants in porous surfaces inject of acid gases, and biogenetic evolution of sedimentary processes by simulation of mass transport in porous media. They primarily use conventional continuum codes, such as TOUGH2 from Lawrence Berkeley National Lab. Eni also has a large R&D investment in renewable energy resources, particularly solar energy through a recent collaboration with MIT. They also have collaborations with PNL and use the codes developed there.

Eni researchers who hosted the WTEC team highlighted some of their key issues in modeling and simulation:

• 
  Accessibility of simulation tools. Currently the simulation tools available in industry are used by experts in computational methods, not people who are domain experts. In many cases, these two groups do not overlap. There is a severe need to develop simulation tools that can be used by domain experts, possibly with help or tailoring from the simulation experts.
  
• 
  Validation. Eni researchers use data from internal company sources and/or literature. Most validation efforts are initiated using published data, which are then followed up with internal company data or new experiments (this does not happen often). The researchers very rarely use field data in model validation.
  
• 
  Multiphysics, multiscale codes. The Eni researchers identified this as a critical issue for the company. For example, they are interested in simulation and prediction of molecular behavior in industrial processes (molecular parameters that mainly influence product formation). Currently they have a small research project with Politechnico Milano to integrate Fluent and Ansys together. Another project focuses on COMSOL, which is a multiphysics framework developed by a Swedish company.
  
• 
  Knowledge management. This was identified as another critical issue for Eni. It has some efforts underway in the area of analytical chemistry where IT tools have been used to exchange, archive, and retrieve knowledge generated in different projects across the organization.
  
• 
  Decision-making and enterprise-level modeling. The R&D group is pretty far from the decision-makers and does not participate actively in the use of simulation models in decision-making. They provide feedback to the decision-makers when asked for more data or higher fidelity predictions. The materials and process-level modeling efforts are not tightly coupled with system or enterprise-level models.
  

• 
  The company does not generally get funding from the Italian government for research projects. It has some EU funding, but not specifically in simulation and modeling activities.
  
• 
  Eni has direct relationships with universities, where the company provides funding for projects. Current collaborations with universities include Politechnico Milan, Politechnico Turin, University of Turin (molecular modeling), CNR Pisa, Bologna (modeling), University of Padova, MIT (solar energy), and Bresia. Typical funding levels in university-affiliated projects are in the €30~40,000 range. The Italian government does not have any program to put industry and universities together.
  
• 
  Eni typically owns all intellectual property developed under projects funded by the company.
  
• 
  The research group had a unit devoted to computing when part of Eni Research. In a new organizational structure, a centralized computing service does not exist; each project works on its own computational needs and resources. Typical computations are done on small clusters in the range of ~10 processors. There is some sharing of computational resources across projects. They do not use national super-computing resources, which are owned/managed by a consortium, due to the costs. The group agreed that improvements in raw computing power, algorithms, and implementation would greatly help their projects.
  
• 
  The WTEC team’s hosts noted that there is a gap in the educational background of incoming researchers/staff. They stressed that domain experts are not trained in simulation tools or don’t understand the tool capabilities, and the simulation experts are very often not experts in the domain.
  

Eni is one of the largest petrochemical enterprises in the world. Its simulation and modeling activities are being conducted primarily in the Research & Development group. The key issues identified by Eni researchers revolve around model validation, accessibility of simulation tools, knowledge management, and development of new multiphysics, multiscale modeling methods.

Site: ETH (Swiss Federal Institute of Technology) Zürich

http://www.ethz.ch/index_EN
ETH-Zentrum (Main Campus)

Institute of Computational Science (ICoS), http://www.icos.ethz.ch/

Computational Science and Engineering (CSE), http://www.rw.ethz.ch/index_EN

Institute of Fluid Dynamics, http://www.ifd.mavt.ethz.ch/
ETH-Hönggerberg Campus

Department of Materials: http://www.mat.ethz.ch/

Institute for Building Materials, http://www.ifb.ethz.ch/
Computational Physics for Engineering Materials http://www.ifb.ethz.ch/comphys/

Institute for Theoretical Physics, http://www.itp.phys.ethz.ch/
Date Visited: February 28, 2008
WTEC Attendees: S. Glotzer (author), L. Petzold, C. Cooper, J. Warren, V. Benokraitis
Hosts: ETH-Zentrum

Professor Dr. Leonhard Kleiser, Institute of Fluid Dynamics, Department of Mechanical and Process Engineering, ETH Zurich

Prof. Petros Koumoutsakos, Chair, Institute of Computational Science

Tel: +41 1 632 52 58; Fax: +41 1 632 17 03

E-Mail: petros@ethz.ch

Professor Dr. Christoph Schwab, Seminar of Applied Mathematics

E-Mail christoph.schwab@sam.math.ethz.ch.

Prof. Dr. Ralf Hiptmair, Seminar of Applied Mathematics

Email: hiptmair@sam.math.ethz.ch

Prof. R. Jeltsch, Seminar of Applied Mathematics

Email: rolf.jeltsch@sam.math.ethz.ch

Prof. Dr. Domenico Giardini, Director of the Swiss Seismological Service, Chair of Seismology and Geodynamics, Swiss Seismological Service

ETH-Hönggerberg / HPP P 6.1, CH-8093 Zurich, Switzerland

Tel: +41-44-633-2610; Secretariat: +41-44-633-2605

Email: d.giardini@sed.ethz.ch

ETH-Hönggerberg

Prof. Dr. Hans Jürgen Herrmann, Institut f. Baustoffe (IfB) (Institute for Building Materials), HIF E 12 Schafmattstr. CH-8093 Zürich, Switzerland

Tel: +41 44 633 27 01; E-Mail: hans@ifb.baug.ethz.ch

Prof. Dr. Matthias Troyer, Computational Physics, Institute for Theoretical Physics

HPZ E 7 Schafmattstr. 32, CH-8093 Zürich, Switzerland

Tel: +41 44 633 25 89; Fax: +41 44 633 11 15

E-mail: troyer@itp.phys.ethz.ch

Markus Reiher, Associate Professor for Theoretical Chemistry, Laboratory of Physical Chemistry HCI G 229, Wolfgang-Pauli-Str. 10, 8093 Zürich, Switzerland

Tel: +41 44 633 43 08; E-mail: markus.reiher@phys.chem.ethz.ch

The WTEC panel is additionally grateful to Prof. Dr. Peter Chen, ETH Vice President for Research, and his office for helping with the team’s visit

ETH is one of the premier universities in Europe, on par with the very top universities in the United States. The panel met with faculty from a number of departments, including physics, chemistry, mathematics, computer science, and several engineering departments, and had an extensive and fruitful discussion. Of particular relevance to SBE&S are campus-wide programs in Computational Science and Engineering (CSE), including the Computational Collaboratory, the Institute for of Computational Science and Engineering, and the Computational Science and Engineering degree program.

The Computational Collaboratory at ETH provides an umbrella that brings together SBE&S research across ETH. The StrategischeErfolgsPositionen program of ETHZ (2001–2007) in CSE provided the opportunity to develop this strategic area at ETHZ with the aim towards international leadership and appeal. The Computational/Collaborational Laboratory in CSE (CoLab) is the central part of this program. Established in ETHZ in 2002, the scientific program is defined by the CoLab Steering Committee. Key elements include a visiting faculty program, a post-doctoral program and summer workshops.

The Institute of Computational Science at ETH (ICoS) “entails interdisciplinary research, tackling complex scientific and engineering problems under the unifying concept of computation. Key research topics include Visual Computing, Bioinformatics and Computational Biology, Machine Learning and Bioinspired Computation, Parallel Computing, Large Scale Linear Algebra, Agent Based Modeling and Simulation, Advanced Symbolic Computation and Multiscale Modeling and Simulation. Faculty and Staff in ICoS are involved in projects spanning a wide range of applications in Systems Biology, Virtual and Augmented Reality, Human-Computer Interface, Pattern Recognition, Nanotechnology, Engineering, Art and Entertainment and Finance” (http://www.icos.ethz.ch/cse/about/index).

The Interdisciplinary Curriculum in Computational Science and Engineering provides Bachelors and Masters degrees to students at ETH. According to the program website, “The CSE Bachelor Program (which is new) consists of two years of studies (4 semesters, second and third year) and is based on knowledge acquired in a first year of basic studies at ETH Zürich or elsewhere. The basic exams after the first year are counted for 60 credit points (ECTS). In the following two years the students have to acquire 120 ECTS, half of them in mandatory Basic Courses, the other half mainly in mandatory Core Courses and in eligible Fields of Specialization and Elective Courses and with a Bachelor Thesis, respectively.” Furthermore, the “CSE Master Program at ETH Zürich (which started in 2005) consists of one year of studies (2 semesters) followed by a Master Thesis. The Master Program is based on the CSE Bachelor Program and its objective is to prepare students for a successful professional career in research in industry and business and/or on a university level. The Master students have to acquire 90 ECTS mainly in mandatory Core Courses, in eligible Fields of Specialization and Elective Courses as … for the Bachelor Program and with a Term Paper and a Master Thesis, respectively. Students have to take both Core Courses and pass the exams.”

SBE&S Research

SBE&S research at ETH spans nearly all disciplines and in many areas represents the state-of-the-art in the field. Examples of the research areas of some of our hosts follow.

Leonhard Kleiser’s main research areas are turbulent flows and laminar-turbulent transition. Turbulent and transitional flows are investigated by direct numerical simulations. Here no turbulence models are employed, but the basic equations of fluid dynamics are numerically integrated on large computers whereby all relevant length and time scales of the flow must be resolved. Such investigations contribute to a better understanding of the fundamental phenomena and mechanisms of transition and turbulence, to the conceptual exploration of methods for flow control, and to the development of improved models for practical calculation methods.

Petros Koumoutsakos’ research activities are in the areas of particle methods, machine learning, biomimetics, biologically inspired computation, and the application of these techniques to problems of interest in the areas of Engineering and Life Sciences. The common patterns that he researches are in multiscale modeling, simulation, design and optimization, and high-performance computing as applied to problems in nanotechnology, fluid mechanics, and life sciences. Research problems in his group range from the design of nano syringes, to tumor-induced angiogenesis and aircraft wakes.

Christoph Schwab’s research interests include computational finance—efficient pricing under multiscale stochastic volatility models; high-dimensional finite elements for elliptic problems with multiple scales; tensor product approximation & anisotropic Besov regularity of elliptic PDEs; FEM for elliptic problems with stochastic data; design of an hp-adaptive FE code for general elliptic problems in 3D; computational finance—Lévy models in finance, and numerical analysis and computation; numerical solution of operator equations with stochastic data; simulations for high current arc plasmas; sparse tensor product methods for radiative transfer; and computational finance—derivative pricing with additive driving processes.

Hans Jürgen Herrmann works on problems in granular materials, including dunes, Apollonian packings, density waves, fragmentation, stratification, segregation, sedimentation, dissipative gases, the shape of a sand pile, the dip under the heap, nonlinear elasticity of packings and shear bands; SOC on small-world lattices and the brain, compactification, fibers, cellular automata, and complex networks; percolation, kinetic gelation, cluster-cluster aggregation, traffic, mineral dendrites, superplasticity, Potts models, fracture, growth phenomena and geometrical critical phenomena.

Matthias Troyer is a leader in the development of new algorithms for quantum Monte Carlo for fermionic systems. His main research fields in computational physics are the development of generic parallel simulation algorithms for strongly interacting quantum systems, the investigation of quantum phase transitions, and the simulation of strongly correlated electron systems.

Markus Reiher’s current research interests comprise topics from relativistic quantum chemistry, bioinorganic and coordination chemistry, theoretical spectroscopy, and the foundations of chemistry.

Computational Science & Engineering (CSE) was launched at ETH as a unique educational program in 1997, with an emphasis on both research and training in computational science and engineering. A CSE curriculum was developed, and since its inception there have been three chairs of CSE at ETH Zurich. Existing courses were combined into an educational program, and thus the program was built at virtually no cost. The program was initially launched as a Master’s program, and has now been extended downward to the Bachelors degree. In 2008 the first freshman students will begin with a major in CSE. Nearly 30 faculty members participate in the program. The host department is mathematics, which handles the administration. Some of the courses were borrowed from Electrical Engineering (students will take calculus and linear algebra, for example, with the Electrical Engineering students). As an alternative to the CSE degree program, students may do a specialization in CSE (Master’s) while enrolled in a traditional degree program.

At the time of the WTEC visit, the ETH/CSE program had 40 undergraduates and 28 Master’s students. In computer science, approximately 20 students take a Master’s with a specialization in CSE, rather than taking a Master’s degree in CSE directly. A report on the CSE educational program is available online at http://www.cse.ethz.ch. Rolff Jeltsch and Walter Gander started the program. Many graduate students and postdocs take the senior level undergraduate CSE courses, particularly parallel programming. There is a new class in CS on high-performance computing. The number of students has increased steadily each year, because there is an increasing computing content in many of the traditional engineering disciplines. The WTEC team’s hosts indicated there has been radical change in the culture of both Engineering Departments and industry in the direction of more and more computation. Even the Mathematics Department has become more open to computing and how it can influence the direction of research. Computing is today more explicitly involved in experimental laboratories, for example, in the design of experiments.

Faculty at both campuses of ETH are more interested in and open to computing today than previously, due in part to new hires, increased awareness of computational science, and a change in industrial culture. Students see that they can get good jobs if trained in computational science and engineering. There has also been a major shift towards virtual experimentation in many fields, due to the increased predictive and explanatory capabilities of SBE&S.

The Computational Collaboratory, which supplemented the CSE Education program, was funded through 2007. Postdocs in the Collaboratory were supported half by the CSElab (http://www.cse-lab.ethz.ch/), and half by individual faculty. This was a highly successful program, but it is no longer supported financially by outside funds.

The WTEC team’s ETH hosts commented that the ETH CSE Master’s program was a pioneering program in Europe. The Bachelor’s program also is now copied elsewhere in Europe. For example, there is a Bachelor’s CSE program in Stuttgart based on the ETH Zurich model. Students from Physics and Chemistry are taking the undergraduate CSE courses even though they are not CS majors. This has also had a big impact. When asked which type of students the computational faculty prefer to work with, they responded: “For research, the physics students; for engineers or software developers, the CSE majors. A potential problem is that the students’ backgrounds are very broad, but not deep. There are courses at the computer centers on how to use the supercomputers, that one can easily send their students to. However, students taking the CSE degree oftentimes compute too much and do not think enough about the underlying physics or mathematics.”

ETH provides flexibility in how faculty members support computational clusters. One group (Reiher) has a 370 core computer cluster. While some faculty described good experiences with the campus computer center, others prefer to have their computers more accessible, since they have found that the service is not responsive. ETH has a budget for internal equipment and supports the acquisition of shared computer clusters by faculty. Funds are awarded competitively, with 60% success rate for internal proposals. Individuals can get 250,000 Swiss Francs every 4–5 years for computers; 10% cost-sharing is required. WTEC team members noted that this program seems unique to ETH.

Simulation codes used are both black box and home grown, depending on the group.

Discussion/Other Comments

Members of the first group (ETH-H) from the Department of Mathematics graciously provided written answers to questions provided in advance by the WTEC panel, and the rest of the group also provided verbal answers during our highly fruitful and engaging discussion. Both the written and verbal answers are included below.

One very important need for SBE&S is the development of much better techniques for programming. For example, MPI was never designed for the ways it is currently being used. It was designed for low-level usage with the intent that higher-level programming languages, or wrappers, would invoke these commands. The expansion of code size with MPI usage is tremendous. Such expansive code quickly becomes extremely difficult to maintain and buggy. Better programming languages and mechanisms, debuggers, and analyzers lag far behind hardware.

ETH Informatik has sidetracked its programming language/compiler work. Its only professor for compiler design is now doing other, unrelated things.

Memory remains the hardware bottleneck: DRAM is more than 100 times slower than CPU performance. Faster, but equally cheap and power-efficient, memory would be very helpful.

Lots of performance at low power would be available if we could easily program multicore/GPU devices. Electric power scales roughly with the third power of the clock, whereas it scales linearly with the number of cpus.

A 1-2 order of magnitude performance in computing power would help in astrophysics, chemical dynamics, and materials, but again we're ignoring something important. Problem conditioning (say the condition number of matrices) increases at least as fast as the problem size, usually much faster. Single-precision arithmetic soon becomes inadequate. Higher-precision arithmetic would be very helpful.

Typically, floating point arithmetic units take about 1.5 percent of chip area. So 128-bit arithmetic would help alleviate large-scale problem conditioning with only a modest increase in real estate.

Start-ups are an interesting question. There is much more of this here than is usually known. Consulting is very popular among advanced ETH students.