Short Course Description
General info:
This course is primarily designed for graduate students to learn the basics and the advances of shared memory programming with OpenMP.
This course will introduce you to the multiple forms of parallelism in modern multi and many-core architecture processors and teach you the programming frameworks for handling this parallelism in applications. You will get access to a cluster of modern multi (CPUs) and many (GPUs) core processors for experiments with graded programming exercises.
You will learn how to handle data parallelism with vector instructions, task parallelism in shared memory with threads, vector programming, SIMD extensions, and offloading to accelerators. This knowledge will help you to accelerate computational applications by orders of magnitude, all the while keeping your code portable and future-proof.
This course can apply to various HPC and data center workloads and frameworks, including artificial intelligence (AI). We will use oneAPI - an open standard for a unified application programming interface intended to be used across different compute processors and accelerator (coprocessor) architectures, and Intel DevCloud, which offers complimentary access to a wide range of Intel architectures. We will also review other heterogeneous APIs' new avenues.
----------------------------------------------------
Grading Policy: Project 100%.
----------------------------------------------------
Prerequisites:
Programming in C/C++ or Fortran - MUST (we will also use Python!)
Programming in the Linux environment and Linux shell proficiency (navigation, file copying, editing files in text-based editors, compilation).
OS profound understanding.
Computer Architecture course (or similar).
----------------------------------------------------
Tentative Course Outline:
Introduction to OpenMP
Creating Threads
Synchronization
Parallel Loops
Data Environment
Memory Model
Irregular Parallelism and Tasks
Memory Access / NUMA
Thread Affinity
SIMD -- Vectorization
Heterogeneous Architectures
OneAPI base and HPC toolkits
----------------------------------------------------
Main References:
Thomas, Sterling, Brodowicz Maciej , Matthew Anderson, High Performance Computing: Modern Systems and Practices, Morgan Kaufmann, 2017
Van der Pas, Ruud, Eric Stotzer, and Christian Terboven. Using OpenMP -- The Next Step: Affinity, Accelerators, Tasking, and SIMD. MIT press, 2017.
Mattson, Timothy G., Yun Helen He, and Alice E. Koniges. The OpenMP Common Core: Making OpenMP Simple Again. MIT Press, 2019.
Deakin, Tom, Mattson, Timothy G., Programming Your GPU with OpenMP: Performance Portability for GPUs, MIP Press, 2023
Full Syllabus
