High-Performance Computing Center Stuttgart
A research team led by Dr. Kathrin Schulte at the University of Stuttgart’s Institute of Aerospace Thermodynamics (ITLR) is among the user groups that have been navigating this transition. For more than 25 years, ITLR researchers have used HLRS supercomputers to run a code called FS3D (Free Surface 3D). A large, modular code with many applications, FS3D uses direct numerical simulation to study fundamental physical processes involving droplets. Such simulations are relevant in numerous fields, from weather prediction, to the design of fuel injectors in engines, to the spraying of paint during manufacturing.
In mid-2024 HLRS held a hands-on code porting workshop for its user community. During the workshop, the ITLR team met with HLRS user support staff as well as hardware experts from Hewlett Packard Enterprise (HPE, Hunter’s manufacturer) and AMD to identify the simplest and most effective strategies for offloading parts of FS3D to GPUs.
When it was initially developed many years ago, FS3D was optimized for a vector computing architecture. Thus, the scientists had experience using the programming model OpenMP for parallel computing using shared memory. This offered the team certain advantages in preparing for Hunter, as array-based vector computing has technical similarities to the concept behind GPU processors. After discussing the advantages and disadvantages of different programming models with HPC experts at HLRS, the team opted to use OpenMP in the GPU implementation of FS3D.
“It would have been possible to use any of a number of other models, although choosing unwisely would have meant needing to rewrite even more code,” Schulte explained. “Because we had no experience with GPUs, we were not really qualified to make a decision. HLRS supported us in choosing a path.”
Despite the experience with vector computing and OpenMP, significant work was still needed to prepare FS3D for Hunter. The team met regularly throughout 2024, communicating often with HLRS user support staff and HPE technical experts for guidance. When they encountered bugs or identified features in FS3D that were not supported in the new programming environment, having quick access to system experts helped them to resolve problems and develop workaround strategies. “When a problem arose with a compiler, for example, it wasn’t like we got stuck for weeks. They helped us to find solutions quickly. We appreciated the constant support from all sides — HLRS, HPE, and AMD,” Schulte remarked.
Before Hunter was fully installed, HPE provided a small cluster at HLRS that is based on the same architecture. This enabled the ITLR team and others in HLRS’s user community to begin testing the ported versions of their codes. With these resources FS3D was ready for GPUs by the time Hunter went online in February 2025.
Additional optimization of FS3D is still needed to improve its performance, and so the ITLR team continues to work with staff at HLRS, HPE, and AMD to develop it further. By doing so now, the ITLR team will be in a good position to use HLRS’s next-generation system — called Herder — which will offer a dramatic jump in performance when it arrives in 2027. These capabilities will enable Schulte and her lab to achieve results faster and to investigate new kinds of phenomena in droplet dynamics.
— Christopher Williams
A member of the Gauss Centre for Supercomputing, HLRS is one of three German national centers for high-performance computing.
