Self-Study Materials

MPI course material (password-free, actual version, without recordings)

Using the material on this web page requires to accept our terms and conditions of use.

---

Slides: (please download, internal hyperlinks work best with Acrobat)

• MPI course with C, Fortran and Python (mpi4py) language bindings: mpi_3.1_rab.pdf
  • This file is always updated to the latest version.
  • The pdf file contains internal cross-references, (see the blue boxes on the 2^nd page).
  • It is highly recommended to use Acrobat, as some other viewers seem to go to the wrong pages, e.g. one page later.
• Same content, but the animated slides are expanded by one slide per animation step: mpi_3.1_rab-animated.pdf
  • This PDF file was created using the PPspliT add-in for PowerPoint, developed by Massimo Rimondini.
  • Many thanks to him for his version 2.5 where he added all the extra features needed to split all the many slides of this course with all their animations and links to other slides within this slide-set.
  • If you find any bugs with this slide-set, please report them directly to Rolf Rabenseifner.

---

Content:

1. MPI Overview
   
   
   • one program on several processors
   • work and data distribution
2. Process model and language bindings
    
   • starting several MPI processes
3. Messages and point-to-point communication
4. Nonblocking communication
    
   • to avoid idle time, deadlocks and serializations
5. The New Fortran Module mpi_f08
6. Collective communication
    
   • (1) e.g., broadcast slides
   • (2) e.g., nonblocking collectives, neighborhood communication
7. Error handling
8. Groups & Communicators, Environment Management
    
   • (1) MPI_Comm_split, intra- & inter-communicators
   • (2) Re-numbering on a cluster, collective communication on inter-communicators, info object, naming & attribute caching, implementation information
9. Virtual topologies
    
   • (1) A multi-dimensional process naming scheme
   • (2) Neighborhood communication + MPI_BOTTOM
   • (3) Optimization through reordering
10. One-sided Communication
11. Shared Memory One-sided Communication
     
    • (1) MPI_Comm_split_type & MPI_Win_allocate_shared
            Hybrid MPI and MPI-3 shared memory programming
    • (2) MPI memory models and synchronization rules
12. Derived datatypes
     
    • (1) transfer any combination of typed data
    • (2) advanced features, alignment, resizing
13. Parallel File I/O
     
    • (1) Writing and reading a file in parallel
    • (2) Fileviews
    • (3) Shared Filepointers, Collective I/O
14. MPI and Threads – e.g., hybrid MPI and OpenMP
15. Probe, Persistent Requests, Cancel
16. Process Creation and Management
     
    • Spawning additional processes
    • Singleton MPI_INIT
    • Connecting two independent sets of MPI processes
17. Other MPI features
18. Best practice
     
    • Parallelization strategies (e.g. Foster’s Design Methodology)
    • Performance considerations
    • Pitfalls and progress / weak local
19. Heat example
20. Summary
21. Appendix 

---

Exercises - please download (zip or tar.gz) and expand both MPI:

• preferred: MPI31single.tar.gz (alternative: MPI31single.zip)
• The exercises are provided in C, Fortran (with the mpi_f08 module) and Python (through mpi4py)

Your environment for the exercises:

To be able to do the hands-on exercises of this course, you need a computer with a C/C++ or Fortran compiler and a corresponding, up-to-date MPI library (in case of Fortran, the mpi_f08 module is required).

Please download TEST archive file TEST.tar.gz or TEST.zip
After uncompressing archive file via
        tar -xvzf TEST.tar.gz
or     unzip TEST.zip
please verify your MPI and OpenMP installation with the tests described in TEST/README.txt within the archive (or here).

---

Standards and API definitions:

• Please download the current MPI standard from the official web-page: MPI documents
  •  The MPI standard includes the MPI language bindings for C and Fortran.
• For Python language bindings with mpi4py, you may look at
  • MPI for Python (https://mpi4py.github.io/),
  • MPI for Python 3.1.1 documentation (https://mpi4py.readthedocs.io/), and
  • The API reference (https://mpi4py.github.io/apiref/index.html)

---

Recordings:

• For recordings, please visit: Online course recordings and look for the latest MPI course