Deep Learning and GPU programming using OpenACC

Photo of scientists participating in a training course in HLRS's Ruehle Saal

NVIDIA Deep Learning Institute (DLI) offers hands-on training for developers, data scientists, and researchers looking to solve challenging problems with deep learning.

Learn how to accelerate your applications with OpenACC, how to train and deploy a neural network to solve real-world problems, and how to effectively parallelize training of deep neural networks on Multi-GPUs.

The workshop combines lectures about Fundamentals of Deep Learning and Fundamentals of Deep Learning for Multi-GPUs with a lecture about Accelerated Computing with OpenACC.

The lectures are interleaved with many hands-on sessions using Jupyter Notebooks. The exercises will be done on a fully configured GPU-accelerated cluster in the cloud.

The first three days are organized in cooperation with LRZ (Germany) and Nvidia. All instructors are NVIDIA certified University Ambassadors. On the last day, you will learn more about DL on the systems at HLRS.

Veranstaltungsort

HLRS, University of Stuttgart
Nobelstraße 19
70569 Stuttgart, Germany
Room 0.439 / Rühle Saal
Location and nearby accommodations

Veranstaltungsbeginn

12. Jul 2022
08:30

Verstaltungsende

14. Jul 2022
17:00

Sprache

Englisch

Einstiegslevel

Basic

Intermediate

Themenbereiche

Data in HPC / Deep Learning / Machine Learning

Themen

Artificial Intelligence

Deep Learning

GPU Programming

Machine Learning

Scientific Machine Learning

Zurück zur Liste

Prerequisites and content levels

Prerequisites

For day one, you need basic experience with C/C++ or Fortran. Suggested resources to satisfy prerequisites: the learn-c.org interactive tutorial, https://www.learn-c.org/.

On day two, you need an understanding of fundamental programming concepts in Python 3, such as functions, loops, dictionaries, and arrays; familiarity with Pandas data structures; and an understanding of how to compute a regression line.
Suggested resources to satisfy prerequisites: Python Beginner’s Guide.

Experience with Deep Learning using Python 3 and, in particular, gradient descent model training will be needed on day three.

Familiarity with TensorFlow and Keras will be a plus as it will be used in the hands-on sessions. For those who did not use these before, you can find tutorials here: github.com/tensorflow/docs/tree/master/site/en/r1/tutorials/keras.

Content levels
  • Basic: 4 hours
  • Intermediate: 11 hours
  • Advanced: 4.5 hours

Learn more about course curricula and content levels.

Instructors

Lectures: Dr. Momme Allalen, PD Dr. Juan Durillo Barrionuevo, Dr. Volker Weinberg (LRZ and NVIDIA University Ambassadors).
Teaching Assistants: Dr. Tobias Haas, Dr. Khatuna Kakhiani, Dr.-Ing. Lorenzo Zanon (HLRS).

Learning outcomes

1st day: Fundamentals of Accelerated Computing with OpenACC

  • How to profile and optimize your CPU-only applications to identify hot spots for acceleration
  • How to use OpenACC directives to GPU accelerate your codebase
  • How to optimize data movement between the CPU and GPU accelerator

Upon completion, you'll be ready to use OpenACC to GPU accelerate CPU-only applications.

2nd day: Fundamentals of Deep Learning

  • Implement common deep learning workflows, such as image classification and object detection
  • Experiment with data, training parameters, network structure, and other strategies to increase performance and capability
  • Deploy your neural networks to start solving real-world problems

Upon completion, you’ll be able to start solving problems on your own with deep learning.

3rd day: Fundamentals of Deep Learning for Multi-GPUs

  • Approaches to multi-GPUs training
  • Algorithmic and engineering challenges to large-scale training
  • Key techniques used to overcome the challenges mentioned above

Upon completion, you'll be able to effectively parallelize training of deep neural networks using TensorFlow.

Agenda

- preliminary -

 1st day: Fundamentals of Accelerated Computing with OpenACC (9:00 - 17:00)

On the first day you learn the basics of OpenACC, a high-level programming language for programming on GPUs. Discover how to accelerate the performance of your applications beyond the limits of CPU-only programming with simple pragmas.

2nd day: Fundamentals of Deep Learning  (9:00 - 17:00)

Explore the fundamentals of deep learning by training neural networks and using results to improve performance and capabilities.

During this day, you’ll learn the basics of deep learning by training and deploying neural networks.

3rd day: Fundamentals of Deep Learning for Multi-GPUs (9:00 - 17:00)

The computational requirements of deep neural networks used to enable AI applications like self-driving cars are enormous. A single training cycle can take weeks on a single GPU or even years for larger datasets like those used in self-driving car research. Using multiple GPUs for deep learning can significantly shorten the time required to train lots of data, making solving complex problems with deep learning feasible.

On the third day we will teach you how to use multiple GPUs to train neural networks.

In addition, we will give an online introduction on how ML Examples and Methods can be used on HLRS's systems (tbc, online using zoom, Friday, July 15, 14:00 - 16:30)

Exercises

On the first three days, the exercises will be carried out on a cloud system using Jupyter Notebooks.

On-site course & COVID rules

Besides the content of the training itself, another important aspect of this event is the scientific exchange among the participants. We try to facilitate such communication by

  • offering common coffee and lunch breaks and
  • working together in groups of two during the exercises (if desired by the individual participants and permitted by the COVID-19 rules).

For your safety, we will only allow fully vaccinated or fully recovered or COVID-19 negative tested participants on all days. You must wear a medical face mask or FFP2 mask everywhere on site. If a distance of 1.5 m cannot be guaranteed inside, e.g., if you are working in pairs in exercises, it must be an FFP2 mask. Details can be found on the registration page.

We strongly recommend to choose travel options and hotels with the possibility to cancel (even close to the event) because we might be forced to deliver the course as an online course.

Registration information

Register via the button at the top of this page.

This course is only open to academic participants, i. e. students or members of universities. Please register with your institutional email address. If you are not interested in all days, please select only those days in which you are interested while registering.

Registration closes on June 27, 2022 (extended registration phase).

Late registrations after that date are still possible according to the course capacity, possibly with reduced quality of service.

Important Information: After you are accepted, please create an account under courses.nvidia.com/join.

NVIDIA Deep Learning Institute:

The NVIDIA Deep Learning Institute delivers hands-on training for developers, data scientists, and engineers. The program is designed to help you get started with training, optimizing, and deploying neural networks to solve real-world problems across diverse industries such as self-driving cars, healthcare, online services, and robotics.

Fees

This course is open to academic participants only.

  • Students without Diploma/Master: 0 EUR
  • Students with Diploma/Master (PhD students) at German universities: 0 EUR
  • Members of German universities and public research institutes: 0 EUR
  • Members of universities and public research institutes within EU or PRACE member countries: 0 EUR.
  • Members of other universities and public research institutes: 0 EUR

Our course fee includes coffee breaks (in classroom courses only).

Lunch Breaks

From Monday to Thursday a nearby canteen is opened and we plan to have lunch there. Further two other near possibilities for lunch are on all days:

We will update this information if opening times etc. change and inform you more precisely before the course starts.

Contact

Tobias Haas phone 0711 685 87223, tobias.haas(at)hlrs.de
Khatuna Kakhiani phone 0711 685 65796, kakhiani(at)hlrs.de

PRACE PATC and bwHPC

HLRS is part of the Gauss Centre for Supercomputing (GCS), which is one of the six PRACE Advanced Training Centres (PATCs) that started in Feb. 2012.

HLRS is also member of the Baden-Württemberg initiative bwHPC.

This course is also provided within the framework of the bwHPC training program. This course is not part of the PATC curriculum and is not sponsored by the PATC program.

Further courses

See the training overview and the Supercomputing Academy pages.

Ähnliche Trainingskurse

Alle Trainingskurse

September 12 - Oktober 24, 2022

Online


September 29 - 30, 2022

Online


Oktober 24 - 25, 2022

Online