CS 377P: Programming for Performance

Assignment 1: Performance counters

Due date: February 10, 2021, 10:00PM

Late submission policy: Submissions can be at most 1 day late. There will be a 10% penalty for late submissions.

Description

1) Write C code for the 6 variants of matrix-matrix multiply (MMM)  you can generate by permuting loops in the standard three-nested loop version of MMM. The data type in the matrix should be doubles.

• Instrument your implementations to use PAPI to measure:
  • Total cycles
  • Total instructions
  • Total Load Store Instructions
  • Total Floating Point Instructions
  • L1 data cache accesses and misses
  • L2 data cache accesses and misses

  Remember that you may not be able to measure all these values at the same time.
  
• Compile your code in ICC with flags '-lpapi -O3 -fp-model precise'.
• Collect these measurements for the following 8 matrix sizes: 50x50, 100x100, 200x200, 400x400, 800x800, 1200x1200, 1600x1600 and 2000x2000. To ensure that there is no interference, make sure that you are the only one running experiments on the machine, and do one measurement at a time.
• You can collect measurements on the following 8 CS machines (with your CS login): orcrist-20.cs.utexas.edu, orcrist-21, orcrist-22, orcrist-24, orcrist-25, orcrist-26, orcrist-27 and orcrist-28.
  
• Create a table in which the rows correspond to the loop-order variant (i-j-k, j-i-k, j-k-i, k-j-i, i-k-j, k-i-j) and the columns correspond to the matrix sizes, and fill in each position in the table with four values: L1 and L2 miss rate, total load and store instructions, and the number of committed floating point instructions. You can create four separate tables if you prefer.
  
• Answer the following questions.
  
• For the smallest matrix size, do the L1 and L2 miss rates vary for the different loop-order variants? Do they vary for the larger matrix sizes? Is there any difference in behavior between the different problem sizes? Can you explain intuitively the reasons for this behavior?
• Re-instrument your code by removing PAPI calls, and using clock_gettime with CLOCK_THREAD_CPUTIME_ID to measure the execution times for the six versions of MMM and the eight matrix sizes specified above. How do your timing measurements compare to the execution times you obtained from using PAPI? Repeat this study using CLOCK_REALTIME. Explain your results briefly.
  

    Hint: To check cache sizes on the machine, run:  lscpu

2) Answer the following questions, using a few sentences for each one.

• What are data and control dependences? Give simple examples to illustrate these concepts.
• Explain out-of-order execution and in-order retirement/commit. Why do high-performance processors execute instructions out of order but retire them in order? What hardware structure(s) are used to implement in-order retirement?

Deliverables

Submit (in canvas) the following two files:

• A .tar.gz file with your code, a README.txt and a Makefile. 
• The README.txt describes how to run your program and what the output will be. A reasonable output will be pairs of "name of measured event, value".
• A Makefile that produces an executable named mmm by running only "make" on any of the 8 CS machines.
• The executable mmm, when no flag is given, should open a file named matrix.txt and output the matrix multiplication result to res.txt. The idea of fixed filename is C++ string parsing is not fun for some people and not the focus of the class.
  • You can have other flags to specify which loop-order variant to run, size of matrix or input/output filename for easier experiment. However, that's completely open to you, as long as it doesn't contradict with the previous requirement.
  • Here's sample matrix.txt and res.txt. The first line of matrix.txt contains two number, number of rows and number of columns of the first matrix, followed by the first matrix with each row separated by a newline and each element separated by a space. Then another line follows containing dimensions of the second matrix, followed by its contents.
• Matrix files should NOT be included in the submission, as that can be very big. You can just generate matrix in memory, without reading it from files, when running experiments.
• The tar file structure should look like this, ./{Makefile,README.txt,[source code files]} , not ./ProjectDirectoryName/{Makefile,README.txt,[source code files]} .
• A report (in .pdf) containing the tables, and the answers to the questions in both parts.
  

Grading

Code: 40 points

Measurements (plots): 30 points

Explanation: 10 points

 Answers to short questions in (2): 20 points

PAPI:

To see which papi counters are available on a host, run:

papi_avail

To see which papi counters can be collected at the same time, run:

papi_event_chooser

Read the PAPI manual http://icl.cs.utk.edu/projects/papi/wiki/PAPIC:EventSets and http://icl.cs.utk.edu/papi/docs/index.html for more information, including example code.

"Warning! num_cntrs is more than num_mpx_cntrs" can be ignored.

ICC:

To run ICC on the indicated CS machines, run:

export PATH=$PATH:/opt/intel/bin
icc [compiler commands]

To check the availability of icc, run:

icc -v