Georg Stadler

Courant Institute of Mathematical Sciences

New York University

Menu


Spring 17: MATH-GA 2012.001 and CSCI-GA 2945.001:
Advanced Topics in Numerical Analysis: High Performance Computing

Here’s a flyer for the class.

To get a better idea about the contents of the class, please take a look at my Spring ‘15 version of this class here.

Instructor:

Georg Stadler, Warner Weaver Hall Office #1111

Lectures: Thursday 5:10-7:00pm, class starts on January Feb 2!

Location: Warren Weaver Hall #512

Office Hours: Wed 10-12 or by appointment-please email.

Teaching Assistant: Bill Bao, yxb201@nyu.edu

If you email me about the class, please add [hpc17] in your subject line, or use this link.

I will be using Piazza to organize the class, as well as for all class-related communication. Please email me if you want to be added and have not received an invitation yet.

Literature:

T. Rauber and G. Ruenger: Parallel Programming: for Multicore and Cluster Sytems, Springer 2010. Pdf is available for download on the NYU campus.

A. Munshi, B. Gaster, T. Mattson, J. Fung and D. Ginsburg; OpenCL Programming Guide, Addison-Wesley, 2011.

Online Resources:

To remind yourself of the basic command line tools, a good source are the first chapters of From Bash to Z Shell by Kiddle, Stephenson and Peek, Apress, 2005.

If you want to freshen up your C knowledge, thy this tutorial: Learn C the hard way

Great overview of algorithms for MPI collective communication functions: Collective Communication: Theory, Practice, and Experience by E. Chan, M. Heimlich, A. Purkayastha and R. van de Geijn.

Course list and homework assignments

Class preparation and setup: HW Assignment #1, Running code at CIMS. For ways to run code on a Mac or on Windows computers, please see the discussion on Piazza.

  1. 2/2: Class organization; a tour through HPC; example problems (serial, shared memory parallel, distributed memory parallel). Slides, and example code on github/lecture1.

  2. 2/16: Memory hierarchies, computational intensity, matrix-matrix multiplication using tiling; the module command; cachgrind; Slides, Notes on matrix tiling, and example code on github/lecture2.

  3. 2/23: Scalability, Amdahl’s law, parallel programming models, git, Slides. Here’s a git cheat sheet.

  4. 3/2: Shared memory parallel computing with Openmp; parallel loops, false and correct sharing, Slides. You can download the example problems form github/lecture4.

  5. 3/7: Shared memory parallel computing II: Jacobi and Gauss-Seidel in parallel. Makefiles. See slides from previous class, and you can download the example problems form github.

  6. 3/9: Some more OpenMP examples; Valgrind debugging; Start with distributed memory parallel programming model; Slides from class; Example problems on github/lecture6. Here is the 2nd homework assignment.

  7. 3/23: Sources of parallelism; blocking/nonblocking MPI send and recv; Git revisited; Slides from class; Example MPI problems on github/lecture7. Here is the 3nd (mini)homework assignment.

  8. 3/30: Collective MPI communication; running through a schedular; Gitk; Parallelization of 1D Jacobi Slides from class; Example MPI problem and job script for Stampede on github/lecture8. Here is the 4nd homework assignment.

  9. 4/6: MPI blocking and non-blocking parallelization of 1D Jacobi, Hybrid MPI-OpenMP computing; Short slides from class; Notes on MPI and hybrid parallelization of Jacobi. MPI Jacobi examples and job script for Stampede in last week’s repo: github/lecture8. Broadcast comparision, hybrid Jacobi and hybrid job script for Stampede: github/lecture9.

  10. 4/13: MPI bugs; Intro to accelerators and OpenCL. github/lecture10.

  11. 4/20: Open OpenCL memory hierarchies. Convolution example problem. github/lecture11.

  12. 4/27: Multigrid; Serial example for 1D multigrid github/multigrid; partitioning and balancing: slides. Here is the 5nd homework assignment.

FAQs:

1) I am looking for an easy course that does not require too much time commitment. Should I take your course? Better don’t. I believe that you can only benefit from a course like that (and make it worth your and my time) if you expose yourself to the material, e.g., write/modify small programs, read up on things you are not yet familiar with, sometimes read online discussions or blog entries, compile and run on different computers, and so on. That requires some effort and time.

2) I am a student in Courant’s Scientific Computing MS program. Can I use your course as capstone project? Yes.

3) How often is this course offered? Every two years.

4) I am not a Mathematics student. Is that a problem? Likely not. While I will mostly focus on Mathematical algorithms, some basic linear algebra knowledge will be sufficient. It is nice if you also know some numerical Mathematics, but not crucial.

5) Is this a programming course? Not primarily. You are expected to have basic programming knowledge already. The main purpose of the course is to develop some parallel thinking and understanding of algorithms. You will learn some programming and programming tools on the side through.


© G.St.