Considering the Limits of Simulation

07 August 2017

In a public lecture at the Carl-Zeiss-Planetarium Stuttgart, HLRS Director Michael Resch explored some key theoretical issues underlying the nature and reliability of computer simulation.

Simulation is an important application of high-performance computing (HPC), but as Professor Michael Resch explained in the Stuttgart Planetarium's Keplersaal on July 26, it is in some ways merely an extension of something humans do everyday. Nevertheless, those qualities that distinguish supercomputing from human cognition offer unique opportunities and present particular challenges.

The lecture was the last in a series of scientific talks held for the general public in conjunction with the exhibit Im digitalen Labor (Inside the Digital Laboratory), organized by the University of Stuttgart. The Cluster of Excellence SimTech, the Internationales Zentrum für Kultur- und Technikforschung (IZKT), and the High-Performance Computing Center Stuttgart (HLRS) were additional cosponsors.

"Simulation," Resch pointed out, "plays an essential role in our lives." The human brain, he argued, can be seen as a tool for simulation in the sense that it integrates data from our environment, makes inferences to predict what will happen in the future, and uses these inferences to guide our actions. When facing complex problems, however—such as understanding and responding to climate change—the brain quickly encounters its limits. In such cases, HPC can extend the brain's basic capabilities, ingesting large amounts of data to simulate complex phenomena and produce predictive models that could not be developed in any other way.

Professor Michael Resch discusses the nature of simulation
at the Stuttgart Planetarium. (Photo: HLRS)

In a scientific context, Resch described simulation as an iterative process in which investigators observe reality, develop a model that tries to reproduce those observations, and then test the model to determine how well it corresponds to reality. Although computer simulation is powerful, he cautioned that its users must always be careful to distinguish between models and reality.

"Whenever we use simulation," Resch said, "we have to use our reason and our experience to observe the result critically and ask the question, 'Can that be true?'" During his lecture Resch identified several elements of computer simulation that can raise questions about the reliability of the models it produces.

Although computers can faithfully execute an algorithm with great consistency, for example, hardware and software are the product of human minds and hands. Processors fail, chip operation can't be observed directly, and mistakes in programming code can be repeated every time the computer runs an operation.

Considering the complexity of HPC simulations, Resch pointed out that most of these errors are typically irrelevant and do not have a large effect on a simulation's outcome. Nevertheless, because the enormous numbers of calculations that go into a computer simulation are impossible to observe and reconstruct, scientists are inevitably in the position of needing to determine whether or not the results are trustworthy. Moreover, mathematics itself can run into its logical limits in fields like fluid dynamics, where numerical approximation can characterize reality in an abstract way, but not necessarily with complete precision. For these reasons, experimental validation and confirmation using other methods are essential in assessing a model's usefulness.

In addition, Resch emphasized, a model is only as valuable as the assumptions that go into designing it. And, indeed, simulation can fail. The collapse of financial markets in 2008 was unforeseen, for example, because predominant models were unreasonably optimistic. He also discussed the disaster at the 2010 Love Parade in Duisburg, in which a panic broke out in the crowd that resulted in 21 deaths and more than 600 injuries. Although simulations by a crowd control specialist had indicated that the security plan for the event would be safe, attendees behaved in unanticipated ways. Indeed, Resch pointed out, human actions do not follow physical laws, making them particularly challenging to calculate.

Such inherent limitations and challenges do not mean that we should not trust simulation as a method of investigation in general. And considering the accelerated speed of global business and technology development, simulation is necessary in industry to remain competitive. But in the end, Resch suggested, scientists and those who apply the models they develop must be aware that simulation will always be a representation of reality that is subject to interpretation, testing, and potentially re-evaluation as more evidence becomes available.

Following the lecture, the near-capacity audience stayed for a lively question-and-answer period, gaining a better understanding of how scientists think about simulation and are applying it in their research.

Christopher Williams