Parallel and Distributed Computing

Butterfly Beating its Wings on Water
Image by Gerd Altmann from Pixabay

We have all of our building blocks, and so now it is time to start combining them! In this module, we're going to investigate the world of parallel and distributed computing, which combines, in some sense, everything we have seen, but particularly computers, operating systems, processes, networks, and atomicity.

Learning Objectives

We're going to learn what parallel computing and distributed computing are, and the challenges that come with each.

Upon completion of this module, you will be able to:

Learning

Note that these videos have embedded formative assessments---you'll want to watch the videos and complete those before class, so that you can earn participation credit.

Recommended Readings

OSTEP: Multiprocessor Scheduling
Distributed Systems for Fun and Profit, by Mikito Takada, Chapter 2 and Chapter 3

Fun Insights

From Class

Any in-class resources will be posted here after class.

Evaluating Your Understanding

We are providing these rubrics for each module so that you can better understand what is expected and evaluate your understanding against those expectations. The hope is that, if you find yourself earning a 3 or below, you will continue to enhance your understanding until you find yourself earning a 4 or a 5.

Standards for Parallel and Distributed Computing Sub-Competency 5 4 3 2
Parallel and Distributed Computing Student accurately names and describes the two types of parallel programming and accurately identifies situations where each is appropriate, accurately explains the characteristics of parallel and distributed systems and gives examples of each, accurately explains the challenges of distributed computing and algorithms we can apply to mitigate those challenges, accurately classifies examples systems as parallel or distributed systems (or a combination), and accurately applies logical clocks, the happened-before relation, and two-phase commit to given scenarios to determine ordering and the ability to commit transactions. Student accurately names and describes the two types of parallel programming and accurately identifies situations where each is appropriate, accurately explains the characteristics of parallel and distributed systems and gives examples of each, accurately explains the challenges of distributed computing and algorithms we can apply to mitigate those challenges, satisfactorily classifies examples systems as parallel or distributed systems (or a combination), and satisfactorily applies logical clocks, the happened-before relation, and two-phase commit to given scenarios to determine ordering and the ability to commit transactions. Student satisfactorily names and describes the two types of parallel programming and satisfactorily identifies situations where each is appropriate, satisfactorily explains the characteristics of parallel and distributed systems and gives examples of each, satisfactorily explains the challenges of distributed computing and algorithms we can apply to mitigate those challenges, unsatisfactorily classifies examples systems as parallel or distributed systems (or a combination), and unsatisfactorily applies logical clocks, the happened-before relation, and two-phase commit to given scenarios to determine ordering and the ability to commit transactions. Student does not submit work OR student unsatisfactorily names and describes the two types of parallel programming and unsatisfactorily identifies situations where each is appropriate, unsatisfactorily explains the characteristics of parallel and distributed systems and gives examples of each, unsatisfactorily explains the challenges of distributed computing and algorithms we can apply to mitigate those challenges, unsatisfactorily classifies examples systems as parallel or distributed systems (or a combination), and unsatisfactorily applies logical clocks, the happened-before relation, and two-phase commit to given scenarios to determine ordering and the ability to commit transactions.
Short Answer Questions: Parallel-and-Distributed-Computing-Specific Student submitted answers to given questions that accurately and succinctly answer the questions. Student submitted answers to given questions that satisfactorily and succinctly answer the questions. Student submitted answers to given questions that unsatisfactorily answer the questions. Student does not submit the answers OR submitted answers to given questions that incorrectly answer the questions.
Short Answer Questions: Parallel and Distributed Systems as part of larger systems Student submitted answers to given questions that accurately and succinctly answer the questions. Student submitted answers to given questions that satisfactorily and succinctly answer the questions. Student submitted answers to given questions that unsatisfactorily answer the questions. Student does not submit the answers OR submitted answers to given questions that incorrectly answer the questions.