SDSC Achieves Record Performance in Seismic Simulations with Intel

Article Content

Example of hypothetical seismic wave propagation with mountain topography using the new EDGE software. Shown is the surface of the computational domain covering the San Jacinto fault zone between Anza and Borrego Springs in California. Colors denote the amplitude of the particle velocity, where warmer colors correspond to higher amplitudes. Image courtesy of Alex Breuer, SDSC.

Researchers at the San Diego Supercomputer Center (SDSC) at the University of California San Diego have developed a new seismic software package with Intel Corporation that has enabled the fastest seismic simulation to-date, as the two organizations collaborate on ways to better predict ground motions to save lives and minimize property damage.

The latest simulations, which mimic possible large-scale seismic activity in the southern California region, were done using a new software system called EDGE, for Extreme-Scale Discontinuous Galerkin Environment. The largest simulation used 612,000 Intel^® Xeon Phi™ processor cores of the new Cori Phase II supercomputer at the National Energy Research Scientific Computing Center (NERSC), the primary scientific computing facility for the Office of Science in the U.S. Department of Energy (DOE).

SDSC’s ground-breaking performance of 10.4 PFLOPS (Peta FLoating-point Operations Per Second, or one quadrillion calculations per second) surpassed the previous seismic record of 8.6 PFLOPS conducted on China’s Tianhe-2 supercomputer. Through efficient utilization of the latest and largest supercomputers, seismologists are now able to increase the frequency content of the simulated seismic wave field.

Obtaining higher frequencies is a key to predict ground motions relevant for common dwellings in conducting earthquake research. SDSC and Intel researchers also used the DOE’s Theta supercomputer at the Argonne National Laboratory as part of the year-long project.

“In addition to using the entire Cori Phase II supercomputer, our research also showed a substantial gain in efficiency in using the new software,” said Alex Breuer, a postdoctoral researcher from SDSC’s High Performance Geocomputing Laboratory (HPGeoC) and lead author of the paper, to be presented in June at the International Super Computing (ISC) High Performance conference in Frankfurt, Germany. “Researchers will be able to run about two to almost five times the number of simulations using EDGE, saving time and reducing cost.”

A second HPGeoC paper submitted and accepted for the ISC High Performance conference covers a new study of the AWP-ODC software that has been used by the Southern California Earthquake Center (SCEC) for years. The software was optimized to run in large-scale for the first time on the latest generation of Intel data center processors, called Intel^® Xeon Phi™ x200.

These simulations, also using NERSC’s Cori Phase II supercomputer, attained competitive performance to an equivalent simulation on the entire GPU-accelerated Titan supercomputer. Titan is located at the DOE’s Oak Ridge National Laboratory and has been the resource used for the largest AWP-ODC simulations in recent years. Additionally, the software obtained high performance on Stampede-KNL at the Texas Advanced Computing Center at The University of Texas at Austin.

Intel^® Parallel Computing Center at SDSC

Both research projects are part of a collaboration announced in early 2016 under which Intel opened a computing center at SDSC to focus on seismic research, including the ongoing development of computer-based simulations that can be used to better inform and assist disaster recovery and relief efforts.

The Intel Parallel Computing Center (Intel PCC) continues an interdisciplinary collaboration between Intel, SDSC, and SCEC, one of the largest open research collaborations in geoscience. In addition to UC San Diego, the Intel PCC at SDSC includes researchers from the University of Southern California (USC), San Diego State University (SDSU), and the University of California Riverside (UCR).

The Intel PCC program provides funding to universities, institutions, and research labs to modernize key community codes used across a wide range of disciplines to run on current state-of-the-art parallel architectures. The primary focus is to modernize applications to increase parallelism and scalability through optimizations that leverage cores, caches, threads, and vector capabilities of microprocessors and coprocessors.

“Research and results such as the massive seismic simulation demonstrated by the SDSC/Intel team are tremendous for their contributions to science and society,” said Joe Curley, senior director of Code Modernization Organization at Intel Corporation. “Equally, this work also demonstrates the benefit to society of developing modern applications to exploit power-efficient and highly parallel CPU technology.”

Such detailed computer simulations allow researchers to study earthquake mechanisms in a virtual laboratory. “These two studies open the door for the next-generation of seismic simulations using the latest and most sophisticated software,” said Yifeng Cui, founder of the HPGeoC at SDSC and director of the Intel PCC at SDSC. “Going forward, we will use the new codes widely for some of the most challenging tasks at SCEC.”

The multi-institution study which led to the record results includes Breuer and Cui; as well as Josh Tobin, a Ph.D. student in UC San Diego’s Department of Mathematics; Alexander Heinecke, a research scientist at Intel Labs; and Charles Yount, a principal engineer at Intel Corporation.

The titles of the respective presentations and publications are “EDGE: Extreme Scale Fused Seismic Simulations with the Discontinuous Galerkin Method” and “Accelerating Seismic Simulations using the Intel Xeon Phi Knights Landing Processor”. The work was supported by the National Science Foundation (NSF), SCEC, and the Intel PCC initiative. Intel, Xeon, and Xeon Phi are trademarks or registered trademarks of Intel Corporation in the U.S. and other countries.