At the beach, the waves provide soothing white noise. But in scientific laboratories, waves play a key role in weather forecasting and climate research. Like the atmosphere, the ocean is typically one of the largest and most computationally demanding components of Earth system models, such as the U.S. Department of Energy's Energy Very Large-Scale Earth System Model, or E3SM.
New solver algorithms for the MPAS-Ocean model significantly enhance climate studies by reducing calculation times and improving accuracy. This breakthrough integrates Fortran and C++ programming and is a step forward in efficient and reliable climate modeling.
A breakthrough in ocean modeling
Most modern ocean models focus on two types of waves: barotropic systems, where the waves propagate faster, and barotropic systems, where the waves propagate slower. To help solve the challenge of simulating both models simultaneously, a team from the Department of Energy's Oak Ridge National Laboratory, Los Alamos National Laboratory, and Sandia National Laboratories developed a new solver algorithm that reduces the overall run time of E3SM's ocean circulation model, the Multi-Scale Ocean Prediction Model (MPAS-Ocean), by 45 percent.
The researchers tested their software on the Summit supercomputer at ORNL's Oak Ridge Leadership Computing Facility, a user facility of the Department of Energy's Office of Science, and on the Compy supercomputer at Pacific Northwest National Laboratory. They conducted their primary simulations on the Cori and Perlmutter supercomputers at Lawrence Berkeley National Laboratory's National Energy Research Scientific Computing Center, and their results were published in the International Journal of High Performance Computing Applications.
Innovations in climate modeling calculations
Trilinos, an open source software database ideal for solving scientific problems on supercomputers, is written in the C++ programming language. Earth system models like E3SM are typically written in Fortran, so the research team leveraged ForTrilinos, a related software library that incorporates Fortran interfaces into existing C++ software packages, to design and customize a new solver that focuses on pressure waves.
"One useful feature of this interface is that we can use every component of the C++ package in Fortran, so we don't need to translate anything, which is very convenient," said lead author Hyun Kang, a computational earth systems scientist at ORNL.
Improvements to MPAS-Ocean
Researchers at ORNL and Los Alamos National Laboratory published a paper in the Journal of Advances in Modeling Earth Systems that improves MPAS-Ocean. ForTrilinos-enabled solvers now overcome the remaining shortcomings of the solvers in previous studies, especially when users run MPAS-Ocean with a small number of compute cores to solve a given problem.
MPAS-Ocean's default solver relies on explicit subcyling technology, which uses many small time intervals or time steps to calculate the characteristics of anisotropic flow waves while performing baroclinic mode decomposition calculations without destabilizing the model. If one advances a baroque linear wave and a pneutropism wave with time steps of 300 seconds and 15 seconds respectively, the pneutropism calculation would need to complete 20 times more iterations to maintain the same speed, which requires a lot of computing power.
In contrast, the new isotropic system solver is semi-implicit, meaning it is unconditionally stable, so researchers can use the same number of large time steps without sacrificing accuracy, saving significant time and computing power.
A community of software developers has spent years optimizing various climate applications in Trilinos and Fortrilinos, so the latest MPAS-Ocean solver that leverages this resource outperforms hand-crafted solvers, enabling other scientists to accelerate their climate research efforts.
"If we had to code each algorithm individually, it would require more effort and expertise," Kang said. "But with this software, we can run simulations instantly and faster by incorporating optimization algorithms into the program."
Future improvements and impacts
Although current solvers still have scalability limitations on high-performance computing systems, their performance is excellent when the number of processors reaches a certain level. This disadvantage exists because the semi-implicit approach requires all processors to communicate with each other at least 10 times per time step, which reduces the performance of the model. To overcome this obstacle, the researchers are currently optimizing processor communication and porting the solver to the GPU.
In addition, the research team also updated the time-stepping method of the baroclinic mold decomposition algorithm to further improve the efficiency of MPAS-Ocean. With these advances, researchers aim to make climate predictions faster, more reliable, and more accurate—an important upgrade for ensuring climate security, enabling timely decision-making, and high-resolution forecasts.
"This barometric model solver enables faster calculations and more stable integration of various models, especially MPAS-Ocean," said Kang. "Extensive use of computing resources requires large amounts of power and energy, but by speeding up this model, we can reduce energy consumption, improve simulations, and more easily predict climate change impacts decades and even millennia into the future."
References: "MPAS-ocean implicit pressure mode solver using a modern Fortran solver interface" by Hyun-GyuKang, Raymond STuminaro, Andrey Prokopenko, SethR Johnson, Andrew GSalinger, and Katherine J Evans, November 17, 2023, "International Journal of High Performance Computing Applications".
doi:10.1177/10943420231205601
Compiled source: ScitechDaily