Using PLAPACK:
Parallel Linear Algebra Package

Robert A. van de Geijn
with contributions by
Philip Alpatov
Greg Baker
Carter Edwards
John Gunnels
Greg Morrow
James Overfelt

• Contents
• 1 Introduction
  • 1.1 Why a New Infrastructure?
  • 1.2 Natural Description of Linear Algebra Algorithms
  • 1.3 Physically Based Matrix Distribution
  • 1.4 Redistributing and Duplicating Matrices and Vectors
    • 1.4.1 Redistributing vectors, matrix rows, and matrix columns
    • 1.4.2 Spreading vectors, matrix rows, and matrix columns
    • 1.4.3 Reducing vectors, matrix rows, and matrix columns
    • 1.4.4 Terminology
  • 1.5 Implementation of Basic Matrix-Vector Operations (Preview)
    • 1.5.1 Matrix-vector multiplication
    • 1.5.2 Rank-1 update
  • 1.6 Implementation of Basic Matrix-Matrix Operations (Preview)
    • 1.6.1 Matrix-matrix multiplication
    • 1.6.2 Attaining better performance
  • 1.7 Basic Linear Algebra Subprograms
  • 1.8 Message-Passing Interface
  • 1.9 Parallel Sparse Linear Algebra
  • 1.10 FORTRAN Interface
  • 1.11 Availability
• 2 Templates and Linear Algebra Objects
  • 2.1 Initializing PLAPACK
  • 2.2 Distribution Templates
    • 2.2.1 Template creation
    • 2.2.2 Template destruction
    • 2.2.3 Template inquiry routines
  • 2.3 Linear Algebra Objects
    • 2.3.1 Linear algebra object creation
    • 2.3.2 Linear algebra object destruction
    • 2.3.3 Linear algebra object inquiry routines
    • 2.3.4 Extracting and setting local data
    • 2.3.5 Initializing object data
  • 2.4 Example
  • 2.5 Return Values
  • 2.6 More Operations and Information
• 3 Advanced Linear Algebra Object Manipulation
  • 3.1 Creating Views into Objects
  • 3.2 Splitting of Linear Algebra Objects
    • 3.2.1 Splitting into four quadrants
    • 3.2.2 Splitting into two parts
  • 3.3 Shifting of Linear Algebra Objects
  • 3.4 Determining Where to Split
  • 3.5 Creating Objects ``Conformal to'' Other Objects
  • 3.6 Annotating Object Orientation
  • 3.7 Casting Object Types
  • 3.8 More Operations and Information
• 4 Application Program Interface
  • 4.1 Introduction
  • 4.2 API-Activation
  • 4.3 Opening and Closing an Object
  • 4.4 Accessing a Vector
  • 4.5 Accessing a Matrix
  • 4.6 Completion and Synchronization
  • 4.7 Examples
  • 4.8 More Operations and Information
• Data Duplication and Consolidation
  • Copy
    • Copy involving vectors
    • Copy involving multivectors
    • Copy involving matrices
    • Copy involving multiscalars
    • Specialized copy
  • Reduce
  • Pipelining Computation and Communication
  • A Building Block Approach to Implementing Copy and Reduce
    • Collective communication operations
    • Efficient implementation of collective communication
    • Implementation of the copy
    • Implementation of the reduce
  • More Operations and Information
• Vector-Vector Operations
  • Copy
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Swap
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Scaling a Vector (Object)
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Scaled Vector (Object) Addition
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Inner Product of Vectors
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Norms of Vectors
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Maximum Absolute Value in Vector
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Example: Parallelizing Inner Product
  • Example: Parallelizing ``axpy'' for Vector Objects
  • More Operations and Information
• Matrix-Vector Operations
  • General Matrix-Vector Multiplication
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Symmetric Matrix-Vector Multiplication
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Triangular Matrix-Vector Multiplication
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Triangular Solve
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Rank-1 Update
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Symmetric Rank-1 Update
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Symmetric Rank-2 Update
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Example: Parallelizing Matrix-Vector Multiplication
    • Simple implementation
    • General implementation
  • Example: Parallelizing Rank-1 Update
    • Simple implementation
    • General implementation
  • More Operations and Information
• Matrix-Matrix Operations
  • General Matrix-Matrix Multiplication
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Symmetric Matrix-Matrix Multiplication
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Symmetric Rank-k Update
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Symmetric Rank-2k Update
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Triangular Matrix-Matrix Multiplication
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Triangular Solve with Multiple Right-Hand-Sides
    • Standard FORTRAN call
    • PLAPACK FORTRAN-C interface
    • PLAPACK calls
  • Example: Parallelizing Matrix-Matrix Multiplication
    • Forming
    • Forming
    • Forming
    • Forming
    • A more general approach
    • Performance results
  • Querying Algorithmic Blocking Size
  • More Operations and Information
• Application of the Infrastructure
  • Cholesky Factorization
  • Right-Looking Variant
    • Level-2 BLAS implementation
    • Level-3 BLAS implementation
  • Left-Looking Variant
    • Level-2 BLAS implementation
    • Level-3 BLAS implementation
    • Towards further performance improvements
  • More Operations and Information
• Summary of PLAPACK Routines and Their Arguments
• Summary of BLAS Related Routines
  • Level-1 BLAS (most commonly used subset)
  • Level-2 BLAS (most commonly used subset)
  • Level-3 BLAS (most commonly used subset)
• References
• About this document ...