SDSC’s ‘Comet’ Supercomputer Blazing Trails via Science Gateways

Article Content

Just six months after coming online, Comet, the new petascale supercomputer at the San Diego Supercomputer Center (SDSC) at the University of California, San Diego, is already blazing new paths of discovery, thanks in part to its role as a primary resource for an assortment of science gateways that provide scientists across many research domains with easy access to its computing power.

Simply described, science gateways provide web browser access to applications and data used by specific research communities. Gateways make it possible to run the available applications on supercomputers such as Comet, so results come quickly, even with large data sets. Browser access offered by gateways allows researchers to focus on their scientific problem without having to learn the details of how supercomputers work and how to access and organize the data needed.

“It’s possible to support gateways across many disciplines because of the variety of hardware and support for complex, customized software environments on Comet,” said Nancy Wilkins-Diehr, an associate director of SDSC and co-director of Extended Collaborative Support Services for the National Science Foundation’s XSEDE (eXtreme Science and Engineering Discovery Environment) program, an advanced collection of integrated digital resources and services that include Comet as a national resource for U.S. academic researchers. “This is a great benefit to researchers who value the ease of use of high end resources via such gateways.”

For the most recent quarter ended September 30, there were 3,310 gateway users across all XSEDE systems, according to data compiled by Wilkins-Diehr. There were 64,377 research jobs run by all gateways across all systems during the quarter, and 86 percent of them were run on either Comet or SDSC’s data-intensive Gordon supercomputer.

“That’s a notable level of usage for a new machine,” said SDSC Deputy Director Shawn Strande, who also is Comet’s program manager. “We anticipate that Comet will reach an active research community of more than 10,000 users, mostly via gateways. Our goal for Comet is to speed up as many researchers as possible, rather than supporting a handful of heroic calculations, so we configured it to serve as one of the most productive HPC systems available to the academic research community, just as its predecessor, Trestles, was.”

In recent years the most popular science gateway in XSEDE has been CIPRES, which stands for CyberInfrastructure for Phylogenetic RESearch (see sidebar). Typically, about 200 CIPRES jobs are running simultaneously on Comet and another 100 on Gordon.

“The scheduling policy on Comet allows us to make big gains in efficiency because we can use anywhere between one and 24 cores on each node,” said Mark Miller, a bioinformatics researcher with SDSC and principal investigator of the CIPRES gateway. “When you are running 200 small jobs 24/7, those savings really add up in a hurry.”

Currently, 30 science gateways are available via XSEDE’s resources, each one designed to address the computational needs of a particular community such as computational chemistry, phylogenetics, or the neurosciences. SDSC itself has delivered 77 percent of all gateway cycles since the start of the XSEDE project in 2011.

Supported by an NSF grant worth almost $24 million including hardware and operating funds, Comet is designed to meet the needs of what is often referred to as the ‘long tail’ of science – the idea that the large number of modest-sized computationally-based research projects represent, in aggregate, a tremendous amount of research that can yield scientific advances and discovery. A video about Comet can be viewed here.