Skip to main content

Unit 4.1.3 What you will learn

In this week, we discover how to parallelize matrix-matrix multiplication among multiple cores of a processor.

Upon completion of this week, we will be able to

  • Exploit multiple cores by multithreading your implementation.

  • Direct the compiler to parallelize code sections with OpenMP.

  • Parallelize the different loops and interpret the resulting performance.

  • Experience when loops can be more easily parallelized and when more care must be taken.

  • Apply the concepts of speedup and efficiency to implementations of matrix-matrix multiplication.

  • Analyze limitations on parallel efficiency due to Ahmdahl's law.

The enrichments introduce us to

  • The casting of other linear algebra operations in terms of matrix-matrix multiplication.

  • The benefits of having a family of algorithms for a specific linear algebra operation and where to learn how to systematically derive such a family.

  • Operations that resemble matrix-matrix multiplication that are encountered in Machine Learning, allowing the techniques to be extended.

  • Parallelizing matrix-matrix multiplication for distributed memory architectures.

  • Applying the learned techniques to the implementation of matrix-matrix multiplication on GPUs.