Overview of the SciDAC-2 Program

The goal of Office of Science Scientific Discovery through Advanced Computing (SciDAC-2) program is to create the software and infrastructure to help scientists effectively utilize the next generation of supercomputers to tackle complex scientific challenges.

Argonne's Participation in SciDAC-2

The Mathematics and Computer Science Division at Argonne National Laboratory is leading or participating in more than two dozen SciDAC-2 projects in all three areas funded by the program: Centers for Enabling Technologies, Science Applications, and SciDAC Institutes.

Centers for Enabling Technologies -- meeting the specific needs of SciDAC science applications researchers as they move toward petascale computing. The centers will specialize in applied mathematics, computer science, distributed computing, or visualization.

Scientific Applications -- accelerating simulation-based science through partnerships among applications domains, computer science, and applied mathematics

Science Institutes -- helping a broad range of researchers, through hands-on workshops and tutorials, to take advantage of the increasing capabilities of supercomputing centers nationwide and to foster the next generation of computational scientists

Centers for Enabling Technologies

1. Center for Enabling Distributed Petascale Science

Argonne PI: Ian Foster
Collaborating Institutions: Fermi National Accelerator Laboratory, Lawrence Berkeley National Laboratory, University of Wisconsin-Madison, University of Southern California

The challenge: Both simulation science (e.g., climate, computational chemistry, fusion, astrophysics) and experimental science (e.g., high energy physics, nuclear physics, light sources, fusion) are poised to produce enormous quantities of data. However, this data is only useful if it can be effectively accessed and analyzed-tasks that are incredibly challenging due to not only the sheer size of the data but also the distributed environment in which data is stored and analysis occurs.

2. Scaling the Earth System Grid to Petascale Data

Argonne PIs: Ian Foster, Kate Keahey, Veronika Nefedova
Collaborating Institutions: Lawrence Berkeley National Laboratory, Lawrence Livermore National Laboratory, Los Alamos National Laboratory, NCAR, NOAA, Oak Ridge National Laboratory, University of Southern California

The challenge: The Earth System Grid (ESG) currently has over 2,300 registered users and manages 140 TB of data. In coming years, ESG faces significant challenges as the size, complexity, and number of climate datasets grow dramatically. In particular, the ESG needs to be extended to support the major Intergovernmental Panel on Climate Change assessment in 2010.

3. Towards Optimal Petascale Simulations (TOPS)

Argonne PIs: Lois McInnes
Collaborating Institutions: Lawrence Berkeley National Laboratory, Lawrence Livermore natinal Laboratory, Los Alamos National Laboratory Columbia University, University of California at Berkeley, University of California at San Diego, University of Colorado at Boulder, University of TExas at Austin

The challenge: Multiscale, multirate scientific and engineering applications in the SciDAC portfolio possess resolution requirements that are practically inexhaustible and demand execution on the highest-capability computers available, which will soon reach the petascale. At their current scalability limits, many applications spend a vast majority of their operations in solvers, due to solver algorithmic complexity that is superlinear in the problem size, whereas other phases scale linearly. Furthermore, the solver may be the phase of the simulation with the poorest parallel scalability, due to intrinsic global dependencies. This project focuses on relieving that bottleneck.

4. Center for Technology for Advanced Scientific Component Software

Argonne PIs: Lois McInnes, Kate Keahey, Jay Larson, Boyana Norris
Collaborating Institutions: Lawrence Livermore National Laboratory, Los Alamos National Laboratory, Oak Ridge National Laboratory, Pacific Northwest National Laboratory, Sandia National Laboratories, Binghamton University, Indiana University, University of Maryland, University of Utah, Tech-X Corporation

The challenge: Scientists face ever-increasing challenges in creating, managing, and applying simulation software to scientific discovery. These challenges, which arise from the growing complexity of the scientific problems and the rapid advances and increasing diversity in hardware platforms, impact researchers' productivity throughout the life cycle of their scientific software. This project will extend the software component methodology, with emphasis on improving usability, and will build a component ecosystem to provide more off-the-shelf components.

5. Scientific Data Management Center for Enabling Technologies

Argonne PIs: Bill Gropp, Rob Ross, Rajeev Thakur
Collaborating Institutions: Lawrence Berkeley National Laboratory, Lawrence Livermore National Laboratory, Oak Ridge National Laboratory, North Carolina State University, Northwestern University, University of California at San Diego, University of Utah

The challenge: With the increasing volume and complexity of data produced by ultra-scale simulations and high-throughput experiments, understanding the science is often hampered by the lack of comprehensive, end-to-end data management solutions ranging from initial data acquisition to final analysis and visualization. Improvements are needed to allow for more interactivity and fault tolerance when managing scientists' workflows, for better parallelism and feature extraction capabilities in their data analytics operations, and for greater efficiency and functionality in users' interactions with local parallel file systems and remote storage.

6. Center for Scalable Application Development Software

Argonne PIs: Peter Beckman
Collaborating Institutions: University of California at Berkeley, University of Tennessee at Knoxville, University of Wisconson at Madison, William Marsh Rice University

The challenge: Technical challenges are associated with effective utilization by applications of emerging multicore architectures and development of shared software infrastructures that enable the scientific community to incrementally construct programming support technologies that are portable across a broad range of high-end computer architectures.

Scientific Applications

1. Modeling Multiscale, Multiphase, Multicomponent Subsurface Reactive Flows Using Advanced Computing

Argonne PI: Barry Smith
Collaborating Institutions: Los Alamos National Laboratory, Oak Ridge National Laboratory Pacific Northwest National Laboratory, University of Illinois at Urbana-Champaign

The challenge: Predictive modeling of subsurface reactive flows is a daunting task because of the wide range of spatial scales involved - from the pore to the field scale - ranging over more than six orders of magnitude, and the wide range of time scales involved - from seconds or less to millions of years. With uniform grids, large 3D field scale continuum models employing billions of nodes can only resolve features on the order of meters and cannot capture phenomena at much smaller scales on the order of millimeters or less. This work is aimed at developing the next-generation massively parallel, multiphase, multicomponent reactive flow and transport code based on the successful prototype code PFLOTRAN.

2. Scalable and Extensible Earth System Model for Climate Change Science

Argonne Co-I: Rob Jacob, Ray Loy
Collaborating Institutions: Brookhaven National Laboratory, Lawrence Berkeley National Laboratory, Lawrence Livermore National Laboratory, Los Alamos National Laboratory, NCAR, Oak Ridge National Laboratory, Pacific Northwest National Laboratory, Sandia National Laboratories, State University of New York at Stony Brook

The challenge: The challenge for this SciDAC project is to transform an existing, state-of-the-science, third-generation global climate model, the Community Climate System Model (CCSM3), to a first-generation Earth system model that fully simulates the coupling between the physical, chemical, and biogeochemical processes in the climate system.

3. A Data Domain to Model Domain Conversion package (DMCP) for Sparse Climate-Related Process Measurements

Argonne PI: Rao Kotamarthi; Richard Coulter, Rob Jacob
Collaborating Institutions: University of Chicago

The challenge: No single methodology can be used with data collected at the spatial scale of the Atmospheric Radiation Measurement Program Climate Research Facility sites or for specific AmeriFlux locations, to derive suitable grid average or column mean values of measured variables for model evaluation and data assimilation in climate models. Such a tool would generate statistical error estimates of the mean quantities when averaged from the observation grid to the model grid, as well as correlations in errors across space and time. The clear challenge is to devise and implement a novel approach for generating data ensembles in order to develop climatologically aware methods for processing ACRF and other spatially sparse datasets.

4. Low-Energy Nuclear Physics National HPC Initiative: Building a Universal Nuclear Energy Density Functional

Argonne PIs: Ewing Lusk, Jorge More', Steven Pieper, Robert Wiringa
Collaborating Institutions: Oak Ridge National Laboratory,

The challenge: The structure and reactions of nuclei are key components of our world. They are crucial for understanding the extreme astrophysical environments where nuclei are formed as well as present and future applications for energy and defense. We propose to create a unified theory of nuclear structure and reactions grounded in fundamental theory by developing a Universal Nuclear Energy Density Functional (UNEDF) to predict nuclear properties and reactions with unprecedented accuracy and clearly-defined uncertainties. This will require theoretical, algorithmic, and computational developments that will take advantage of new computer resources including petascale architectures.

5. Framework Application for Core-Edge Transport Simulations (FACETS)

Argonne PI: Jay Larson
Collaborating Institutions: Colorado State University, Columbia University, General Atomics, Indiana University , Lawrence Livermore National Laboratory, Oak Ridge National Laboraotyr, Para Tools, Inc., Princeton Plasma Physics Laboratory, Tech-X Corporation, University of California - San Diego

The challenge: Critical to the success of the U.S. fusion program is the ability to conduct whole-device modeling. This ability requires the development of a multiphysics parallel framework that is flexible enough to facilitate switching from one model to another for a given aspect of the physics and is able to couple existing core and edge simulations with transport and wall interactions.

Science Institutes

1. Combinatorial Scientific Computing and Petascale Simulations (CSCAPES)

Argonne PIs: Paul Hovland, Boyana Norris, Jean Utke
Collaborating Institutions: Ohio State University, Old Dominion University, Sandia National Laboratories

The challenge: A key challenge for scientific computing is obtaining high performance for these advanced applications on such complicated computers. To address this challenge, the CSCAPES Institute will provide new capabilities in load balancing, new automatic differentiation capabilities, and new sparse matrix software tools.

2. Performance Engineering Research Institute

Argonne PI: Paul Hovland, Dinesh Kaushik, Boyana Norris
Collaborting Institutions: Lawrence Berkeley National Laboratory, Lawarence Livermore National Laboratory, Oak Ridge National Laboratory, Rice University, University of California at San Diego, University of Maryland, University of Nort Carolina, University of Southern Californai. University of Tennessee

The challenge: Achieving good performance on petascale computing systems will grow ever more challenging due to enormous scale and increasing complexity in both architecture and application. The ideal performance tool will analyze a scientific application, both as source code and during execution generate a space of tuning options, and search for a near-optimal performance solution. To achieve this vision, numerous challenges remain, including enhancement of automatic code manipulation tools, automatic runtime parameter selection, and automatic communication optimization.

3. SciDAC Institute for Ultrascale Visualization

Argonne PI: Robert Ross

Collaborting Institutions: Ohio State University, Sandia National Laboratories, University of California at Davis, University of Tennessee at Knowxille, University of Virginia

The challenge: Understanding the science behind ultra-scale simulations and high-throughput experiments requires extracting meaning from data sets of hundreds of terabytes or more. Parallel visualization is the most plausible path to understanding data at this scale; however, existing parallel visualization tools have limited functionality, are not portable or scalable to our largest systems, or are not readily adapted to new applications.