Course Syllabus

Overview

This course aims to provide knowledge and hands-on experience in developing software for processors with massively parallel computing resources. A massively parallel processor has the hardware ability to allow many threads to execute simultaneously. Many commercial offerings from NVIDIA, Intel, AMD, Qualcomm, Google, and ARM offer such levels of concurrency. Effectively programming these processors requires in-depth knowledge of parallel programming principles and the parallelism models, communication models, and resource limitations of these processors. In addition, this course also prepares the students to use the parallelism models of modern programming languages. The target audiences of the course are students who want to develop applications for these processors for fun, work, and research, as well as those who want to develop programming tools and future processors.

The course involves a series of programming assignments of steadily growing complexity. All assignments will involve programming a massively parallel GPU system using CUDA, a popular commercial language extension of C/C++ for GPU programming.  

Textbook

Wen-mei W. Hwu, David B. Kirk and Izzat El Hajj, “Programming Massively Parallel Processors: A Hands-on Approach,” Morgan Kaufman Publisher, 4th edition, 2022, ISBN-13 978-0323912310

NVIDIA documentation

NVIDIA, CUDA C Programming Guide (CUDA PG)
https://docs.nvidia.com/cuda/cuda-c-programming-guide
https://docs.nvidia.com/cuda/pdf/CUDA_C_Programming_Guide.pdf

NVIDIA, CUDA C++ Best Practices Guide (CUDA BPG)
https://docs.nvidia.com/cuda/cuda-c-best-practices-guide
https://docs.nvidia.com/cuda/pdf/CUDA_C_Best_Practices_Guide.pdf

NVIDIA Developer Blog (NVIDIA DB)
https://developer.nvidia.com/blog

Discussion Board

We will use Piazza for announcements, questions, and general class discussions. Please post all non-personal questions there.

Lecture Materials

Lecture materials will be posted before class. If you do not see something there, please ask on piazza. 

Class Schedule

The class schedule is subject to change.

Late Days

If you run into an issue and need a little more time (up to 24 hours) for a PA, you can use a late day. This is meant for exceptional circumstances. We will discuss what is happening if you're using them too much. You do not need to inform us that you are doing this. We will see it on Gradescope.