High-Performance Computing Center Stuttgart

BOOTCAMP: Multi-GPU Programming

This Bootcamp will be held online with Zoom. Cluster Dry Run already on Mon, June 16.

This Bootcamp will cover scaling applications to multiple GPUs across multiple nodes. Doing so requires one to be adept at programming models and optimization techniques, and proficient at performing root-cause analysis using in-depth profiling to identify and minimize bottlenecks.

The N-Ways to Multi-GPU Programming Bootcamp covers step-by-step ways to improve application performance using cues from profilers and provides an understanding of the underlying technologies and communication topology needed to leverage powerful NVIDIA® libraries to extract more performance from the system.

The Bootcamp is co-organised by HLRS, JSC, LRZ, VSC Vienna, Cyfronet AGH, IT4Innovations, RISE, LiU, OpenACC.org and NVIDIA for EuroCC Austria, EuroCC Czechia, EuroCC Poland, EuroCC Montenegro, SLING, EuroCC@GCS and EuroCC Sweden, all National Competence Centres for High-Performance Computing.

Location

Online course
Organizer: OpenHackathons, OpenACC.org, NVIDIA

Start date

Jun 17, 2025
09:00

End date

Jun 18, 2025
14:30

Language

English

Entry level

Advanced

Course subject areas

Bootcamp/Hackathon

Hardware Accelerators

Parallel Programming

Topics

Code Optimization

GPU Programming

OpenACC

OpenMP

Back to list

Prerequisites and content levels

Prerequisites

Experience with C/C++, NVIDIA CUDA C++, and MPI  knowledge is required.

Content levels
  • Intermediate level: 3:30 hours
  • Advanced level: 6:30 hours

Learn more about course curricula and content levels.

Instructors

Main lecturers: Paul Graham (NVIDIA)

Agenda

- preliminary -

All times in CEST:

Day 0 (Mon, June 16) : Cluster Dry Run

  • 11:00 - 12:00 Cluster Dry Run Session

Day 1 (Tue, June 17): Parallelizing a Single GPU Code

  • 09:00 - 09:05 Welcome
  • 09:05 - 09:30 Introduction to Nsight Systems and GPU Computing (Lecture and Read-only Lab)
  • 09:30 - 10:30 Overview of a Single GPU Code
  • 10:30 - 12:30 Parallelizing a Single GPU Code using CUDA memcpy, Streams, and an Introduction to Single Node Multi-GPU ( P2P ) (Lecture and Lab)
  • 12:30 - 13:00 Lunch
  • 13:00 - 15:00 Introduction to Multi-Node Multi-GPU (GPU Direct ) programming (Lecture and Lab)

Day 2 (Wed, June 18): Introduction to NCCL and NVSHMEM

  • 09:00 - 11:00 Introduction to NVIDIA Collectives Communications Library (NCCL) (Lecture and Lab)
  • 11:00 - 13:00 Introduction to NVSHMEM (Lecture and Lab)
  • 13:00 - 13:30 Demo of the Nsight Systems Profiler Report for a Multi-Node Application

Hands-on sessions

Attendees will be given access to a GPU cluster for the duration of the Bootcamp.

The code is publicly available on github.

Registration information

Registration is now open.

We will inform you until 2 June 2025 about your acceptance for the bootcamp.

This course is offered in cooperation by HLRS, JSC, LRZ, VSC Vienna, Cyfronet AGH, IT4Innovations, RISE, LiU, OpenACC.org and NVIDIA.

Registration closes on May 19, 2025 at 23:55.

Fees

  • Students without Master's degree or equivalent. Participants from EU or EuroCC countries only: 0 EUR
  • PhD students or employees at a German university or public research institute: 0 EUR
  • PhD students or employees at a university or public research institute in an EU or EuroCC country other than Germany: 0 EUR.
  • Other participants, e.g., from industry, other public service providers, or government. Participants from EU or EuroCC countries only: 0 EUR

Our course fee includes coffee breaks (in classroom courses only).

For lists of EU and EuroCC countries have a look at the Horizon Europe and EuroCC website.

Only participants from institutions belonging to these countries can take part in this course.

Contact

Maksym Deliyergiyev phone 0711 685 87261, training(at)hlrs.de

HLRS training collaborations in HPC and AI

HLRS is part of the Gauss Centre for Supercomputing (GCS), together with JSC in Jülich and LRZ in Garching near Munich. EuroCC@GCS is the German National Competence Centre (NCC) for High-Performance Computing. HLRS is also a member of the Baden-Württemberg initiative bwHPC. Since 2025, HLRS coordinates HammerHAI. 

This course is provided within the framework of EuroCC2.

Acknowledgements

Within the scope of the EuroCC2 project this training is offered free of charge. By participating in a training free of charge, companies receive state-aid corresponding to the regular market-prices listed in the service portfolio. Please note state-aid regulations in Germany.

EuroCC 2 and EuroCC4SEE have received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 101101903 and No 101191697. The JU receives support from the European Union’s Digital Europe Programme and Germany, Bulgaria, Austria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Greece, Hungary, Ireland, Italy, Lithuania, Latvia, Poland, Portugal, Romania, Slovenia, Spain, Sweden, France, Netherlands, Belgium, Luxembourg, Slovakia, Norway, Türkiye, Republic of North Macedonia, Iceland, Montenegro, Serbia, Bosnia and Herzegovina.

Further courses and training team

See the training overview and the Supercomputing Academy pages.
See also information about the HLRS training department and staff.

Related training
All training

Nov 11 - 20, 2025

Online

English

GPU Programming using CUDA

Nov 20 - Dec 05, 2025

Hybrid, Stuttgart

English

Hackathon: Porting and Optimization for Hunter

Dec 01 - 04, 2025

Online

English

Advanced Parallel Programming with MPI and OpenMP

Jan 12 - 16, 2026

Hybrid, Garching

English

GENE/GENE-X user training and tutorial

Jan 20 - 29, 2026

Online

English

GPU Programming using CUDA

Jan 26 - Feb 06, 2026

Online (flexible)

English

Supercomputing Academy - Introduction to OpenMP

Feb 10 - 12, 2026

Hybrid, Stuttgart

English

Hybrid Programming in HPC – MPI+X

Mar 23 - 27, 2026

Hybrid, Stuttgart

English

Iterative Linear Solvers and Parallelization

Back to list

High-Performance Computing Center Stuttgart Nobelstraße 19, 70569 Stuttgart, Germany

Contact & Location

+49 711 685-87269 info(at)hlrs.de

A member of the Gauss Centre for Supercomputing, HLRS is one of three German national centers for high-performance computing.

www.gauss-centre.eu

HLRS is a central unit of the University of Stuttgart.

www.uni-stuttgart.de/en/

© 2025 HLRS. All rights reserved.