Notice that after a vector, matrix row, or matrix column has been spread, multiple copies of the data exist. Often, as part of an operation, multiple instances of data exist, but each instance holds a contribution to a global result. These contributions must then be reduced (typically summed) into the global result. Often, the reduction needs only be performed (simultaneously) within one or all row(s) or column(s) of the node mesh. Looking ahead slightly, consider the situation in Figure 1.5. Each column of nodes holds a column vector, which must be summed element-wise into a single vector. The collective communication requires is a reduction that leaves a sub-vector of the result on each node. We call this a distributed reduction. We will discuss reduction (consolidation) of data again in Chapter .