BOOTCAMP: Multi-GPU Programming

This Bootcamp will be held online with Zoom. Cluster Dry Run already on Fri, May 3.

This Bootcamp will cover scaling applications to multiple GPUs across multiple nodes. Doing so requires one to be adept at programming models and optimization techniques, and proficient at performing root-cause analysis using in-depth profiling to identify and minimize bottlenecks. The N-Ways to Multi-GPU Programming Bootcamp covers step-by-step ways to improve application performance using cues from profilers and provides an understanding of the underlying technologies and communication topology needed to leverage powerful NVIDIA® libraries to extract more performance from the system.

The Bootcamp is co-organised by HLRS, JSC, LRZ, VSC Vienna, RISE, LiU, OpenACC.org and NVIDIA for EuroCC Austria, EuroCC@GCS and EuroCC Sweden, all National Competence Centres for High-Performance Computing.

Veranstaltungsort

Online course
Organizer: HLRS, University of Stuttgart, Germany

Veranstaltungsbeginn

06. Mai 2024
09:00

Verstaltungsende

07. Mai 2024
14:30

Sprache

Englisch

Einstiegslevel

Fortgeschritten

Themenbereiche

Paralleles Programmieren

Themen

Beschleuniger

Code-Optimierung

GPU-Programmierung

OpenACC

OpenMP

Zurück zur Liste

Prerequisites and content levels

Prerequisites

Experience with C/C++, NVIDIA CUDA C++, and MPI  knowledge is required.

Content levels
  • Intermediate level: 3:30 hours
  • Advanced level: 6:30 hours

Learn more about course curricula and content levels.

Instructors

Main lecturers: Paul Graham (NVIDIA)

Teaching Assistants:

  • Claudia Blaas-Schenner, Siegfried Höfinger, Moritz Siegel, Soner Steiner, Ivan Vialov (VSC Vienna)
  • Tobias Haas, Maksym Deliyergiyev (HLRS)
  • Jayesh Badwaik, Kaveh Haghighi-Mood (JSC)
  • Juan Durillo (LRZ)
  • Thor Wikfeldt (RISE), Wei Li (LiU)

Agenda

- preliminary -

All times in CEST:

Day 0 (Fri, May 3) : Cluster Dry Run

  • 11:00 - 12:00 Cluster Dry Run Session

Day 1 (Mon, May 6): Parallelizing a Single GPU Code

  • 09:00 - 09:05 Welcome
  • 09:05 - 09:30 Introduction to Nsight Systems and GPU Computing (Lecture and Read-only Lab)
  • 09:30 - 10:30 Overview of a Single GPU Code
  • 10:30 - 12:30 Parallelizing a Single GPU Code using CUDA memcpy, Streams, and an Introduction to Single Node Multi-GPU ( P2P ) (Lecture and Lab)
  • 12:30 - 13:00 Lunch
  • 13:00 - 15:00 Introduction to Multi-Node Multi-GPU (GPU Direct ) programming (Lecture and Lab)

Day 2 (Tue, May 7): Introduction to NCCL and NVSHMEM

  • 09:00 - 11:00 Introduction to NVIDIA Collectives Communications Library (NCCL) (Lecture and Lab)
  • 11:00 - 13:00 Introduction to NVSHMEM (Lecture and Lab)
  • 13:00 - 13:30 Demo of the Nsight Systems Profiler Report for a Multi-Node Application

Hands-on sessions

Attendees will be given access to a GPU cluster for the duration of the Bootcamp.

The code is publicly available on github.

Registration information

Please register at OpenACC-Standard.org via the button at the top of this page with your institutional e-mail address to prove your affiliation.

The final participants will be selected and informed after the registration deadline has passed.

This course is offered in cooperation by HLRS, JSC, LRZ, VSC Vienna, RISE, LiU, OpenACC.org and NVIDIA. Registration is done via www.gpuhackathons.org hosted by OpenACC-Standard.org. Your registration data will be transferred to these partners. For legal notes see the Privacy Policy.

Registration closes on March 26, 2024 at 23:55.

Fees

  • Students without Master's degree or equivalent. Participants from EU or EuroCC countries only: 0 EUR
  • PhD students or employees at a German university or public research institute: 0 EUR
  • PhD students or employees at a university or public research institute in an EU or EuroCC country other than Germany: 0 EUR.
  • Other participants, e.g., from industry, other public service providers, or government. Participants from EU or EuroCC countries only: 0 EUR

Our course fee includes coffee breaks (in classroom courses only).

For lists of EU and EuroCC countries have a look at the Horizon Europe and EuroCC website.

Only participants from institutions belonging to these countries can take part in this course.

Contact

Maksym Deliyergiyev phone 0711 685 87261, maksym.deliyergiyev(at)hlrs.de

HLRS Training Collaborations in HPC

HLRS is part of the Gauss Centre for Supercomputing (GCS), together with JSC in Jülich and LRZ in Garching near Munich. EuroCC@GCS is the German National Competence Centre (NCC) for High-Performance Computing. HLRS is also a member of the Baden-Württemberg initiative bwHPC.

This course is provided within the framework of EuroCC2.

Official course URL(s)

http://www.hlrs.de/training/2024/BC-MULTIGPU-NV and course website at Open Hackathons.

Further courses

See the training overview and the Supercomputing Academy pages.

Ähnliche Trainingskurse

Alle Trainingskurse

April 22 - 25, 2024

Online


Mai 13 - 17, 2024

Hybrid Event - Stuttgart, Germany


Juni 03 - 07, 2024

Hybrid Event - Stuttgart, Germany


Juli 02 - 05, 2024

Stuttgart, Germany


Oktober 14 - 18, 2024

Stuttgart, Germany