Industrial HPC Users Face Unique Challenges

Keyvisual image main

A workshop at HLRS identified issues that need to be addressed if commercial companies are to take full advantage of increased computing power.

Although the High-Performance Computing Center Stuttgart (HLRS) dedicates the majority of its computing resources to large-scale academic research projects, clients from industry also form an important segment of its users. Working with private technology companies from many fields, HLRS is a partner for industrial R&D, particularly for the dynamic community of small and medium-sized enterprises (SMEs) in the German state of Baden-Württemberg.


It's clear that high-performance computing (HPC) offers valuable opportunities that can make companies more efficient and competitive in the global marketplace. At the same time, however, it's also evident that even when using comparatively modest computing resources, industrial HPC users have unique needs and face different challenges than their counterparts in academia. The first Industrial HPC User Roundtable, hosted at HLRS on December 8, 2017, set out to identify exactly what those special challenges are, as well as how HPC centers like HLRS could help businesses to overcome them.

The full-day event brought together more than 30 HPC users from companies of different sizes and in different industries to discuss the challenges they share. Organized by SICOS-BW, an independent organization physically based at HLRS that helps SMEs to access HPC resources, the day's presentations and open discussion indicated that the HPC industry is not necessarily evolving in ways that best meet its industrial users' needs.

During the event five speakers, all of them experienced industrial HPC-users, provided insights into how HPC is being used in industry. HLRS experts also gave presentations explaining how the center supports industry today in using its HPC systems and provided an outlook on emerging technologies in supercomputing.

Difficulties in hardware, software, and workflow integration

A recurring concern expressed by attendees was that consolidation and profitability goals among HPC system manufacturers threatens to "McDonalds-ize" supercomputing. As certain architectures and solutions become standardized, and as HPC hardware manufacturers develop business models that incorporate software and HPC service provision, some engineers in industry are beginning to fear they could lose the flexibility and custom solutions they require.

One area that could go in this direction is cloud supercomputing, currently an appealing model for small companies that can't justify the expense of building their own HPC infrastructure and data storage capabilities. If cloud providers support only certain software packages, however, or if they do not make it easy for users to move data between different platforms, they could end up being of limited use.

Another frequently mentioned problem at the meeting was the current state of commercial software packages for simulating and modeling complex systems. Many codes that are essential tools in industry were written for single-core machines and do not easily scale up for efficient use in parallel computing environments. This often puts software users at companies in the uncomfortable position of needing to become software developers to reprogram the code for use on HPC systems.

Several participants in the meeting also pointed out that commonly used licensing models from software manufacturers are often prohibitively expensive for smaller companies.

Considering the complicated, time-consuming processes companies require to bring products to market, streamlining workflows that incorporate simulation is also important. This is particularly the case when companies do not have an in-house HPC system or expertise in HPC. Participants in the meeting pointed out that when going to external providers of computational resources, it is also critical that providers facilitate data transfer efficiently, ensure reliable data security, and are sensitive to issues related to data ownership and liability.

Industry-focused HPC training is a key

Multiple participants pointed out that even when engineers receive computer science training in a university, it quickly becomes outdated once they graduate and the field continues to evolve. This means that there is a large and growing need in industry for continuing education that enables industrial HPC users to update their skills in using the latest technologies.

SICOS-BW's Markus Klietmann described plans at HLRS to expand its continuing education training program to address the specific needs of industrial users. Using a blended learning approach that combines classroom and online components, the program aims to provide essential expertise for industrial HPC users in a way that is convenient for them. Though still under development, the effort, funded by the European Social Fund and the Ministry of Science, Business, and Culture in Baden-Württemberg, aims to provide a rounded program of training offerings focusing on topics such as parallel programming, performance optimization, visualization, and other key topics that are important for industrial users.

In this context, the first Industrial HPC User Roundtable was part of an ongoing outreach effort at HLRS to learn from its current and potential users and to target its offerings to their needs. The dialogue will be ongoing, as attendees expressed interest in forming topic-focused working groups and returning for a second annual Industrial HPC User Roundtable, which is scheduled at HLRS for December 11, 2018.

Christopher Williams