HLRS High Performance Computing Center Stuttgart: BOOTCAMP: Multi-GPU Programming

Prerequisites and content levels

Prerequisites

Experience with C/C++, NVIDIA CUDA C++, and MPI knowledge is required.

Content levels

Intermediate level: 3:30 hours
Advanced level: 6:30 hours

Learn more about course curricula and content levels.

Instructors

Main lecturer: Paul Graham (NVIDIA)

Event Moderator: Claudia Blaas-Schenner (ASC)

Agenda

- preliminary -

All times in CEST:

Day 0 (Mon, September 28):

12:00–13:00 Cluster Dry Run Session (optional; please attend in case you need help with login)

Day 1 (Tue, September 29):

08:45–09:00 Join in
09:00–09:15 Welcome to the Multi-GPU Programming Bootcamp (Moderator)
09:15–09:40 Introduction to Nsight Systems and GPU Computing (Lecture and Read-Only Lab)
09:40–10:30 Overview of a Single GPU Code
10:30–10:45 Break
10:45–12:30 Parallelizing a Single GPU Code using CUDA memcpy, Streams, and an Introduction
to Single Node Multi-GPU (P2P) (Lecture and Lab)
12:30–13:00 Lunch
13:00–15:00 Introduction to Multi-Node Multi-GPU (GPU Direct) programming (Lecture and Lab)

Day 2 (Wed, September 30):

08:45–09:00 Join in
09:00–10:55 Introduction to NVIDIA Collectives Communications Library (NCCL) (Lecture and Lab)
10:55–11:10 Break
11:10–13:00 Introduction to NVSHMEM (Lecture and Lab)
13:00–13:30 Demo of the Nsight Systems Profiler Report for a Multi-Node Application

Hands-on sessions

Attendees will be given access to a GPU cluster (Athena) by Cyfronet for the duration of the Bootcamp.

The code is publicly available on github.

Registration information

Apply for this course via the button at the top of this page.

Registration is now open and handled by Austrian Scientific Computing (ASC).

Please register with your official email address to prove your affiliation.
You will get an automatic confirmation via email (subject starting with "[Indico] Registration"), please check your Spam/Junk folders.
Following your successful registration, you will receive further information approximately 1 week before the bootcamp.

Registration closes on September 21, 2026 at 23:55.

Fees

This course is free of charge and includes coffee breaks (in classroom courses only).

Only participants from institutions belonging to EU and EuroHPC JU member states can apply to this course.

Contact

Please contact our partner directly via training(at)asc.ac.at

HLRS training collaborations in HPC and AI

HLRS is part of the Gauss Centre for Supercomputing (GCS), together with JSC in Jülich and LRZ in Garching near Munich. EuroCC@GCS is the German National Competence Centre (NCC) for High-Performance Computing. HLRS is also a member of the Baden-Württemberg initiative bwHPC. Since 2025, HLRS coordinates HammerHAI.

This course is provided within the framework of EuroCC2.

Acknowledgements

EuroCC 3 has received funding from the European High-Performance Computing Joint Undertaking (JU) under Grant Agreement No. 101306701. The JU receives support from the European Union‘s Digital Europe Programme and Germany, Albania, Austria, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Cyprus, Czechia, Denmark, Estonia, Finland, France, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Montenegro, the Netherlands, North Macedonia, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Türkiye, and Kosovo.

Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or EuroHPC Joint Undertaking. Neither the European Union nor the EuroHPC Joint Undertaking can be held responsible for them.

HPC in Europe is the umbrella brand uniting Europe's high-performance computing initiatives across 36+ countries.

Official course URL(s)

https://events.asc.ac.at/event/358/

Further courses and training team

See the training overview and the Supercomputing Academy pages.
See also information about the HLRS training department and staff.

BOOTCAMP: Multi-GPU Programming

Location

Prerequisites and content levels

Prerequisites

Content levels

Instructors

Agenda

Hands-on sessions

Registration information

Fees

Contact

HLRS training collaborations in HPC and AI

Acknowledgements

Official course URL(s)

Further courses and training team

Related training

All training

BOOTCAMP: N-Ways to GPU Programming

Finding and fixing bugs in parallel and GPU applications on Hunter

Parallel Programming Workshop (Train the Trainer)

Parallel Programming Workshop (MPI, OpenMP and Advanced Topics)

Introduction to OpenMP Offloading with AMD GPUs

GPU Programming using CUDA

Hackathon: Porting and Optimization for Hunter