NETS 212: Scalable and Cloud Computing (Fall 2020)

What is the "cloud"? How do we build software systems and components that scale to millions of users and petabytes of data, and are "always available"?

In the modern Internet, virtually all large Web services run atop multiple geographically distributed data centers: Google, Yahoo, Facebook, iTunes, Amazon, eBay, Bing, etc. Services must scale across thousands of machines, tolerate faults, and support thousands of concurrent requests. Increasingly, the major providers (including Amazon, Google, Microsoft, HP, and IBM) are looking at "hosting" third-party applications in their data centers - forming so-called "cloud computing" services. A significant number of these services also process "streaming" data: geocoding information from cell phones, tweets, streaming video, etc.

This course, aimed at a sophomore with exposure to basic programming within the context of a single machine, focuses on the issues and programming models related to such cloud and distributed data processing technologies: data partitioning, storage schemes, stream processing, and "mostly shared-nothing" parallel algorithms.

Instructors

Zachary G. Ives
Office hour: Mondays 2:00-3:00pm (via gather.town)

Teaching assistants

We will be using ohq.io for the TA office hours.

Format

The Fall 2020 version of this class will be entirely online, due to COVID-19. We will make prerecorded lectures available for download, and we will use the class slots for discussion, review, and Q&A. The review sessions will be recorded as well. There will be regular homework assignments, two midterms (online, via GradeScope), and a final team project. We will use Piazza for course-related discussions, and there will be occasional lab sessions.

Time and location

Prerequisites

CIS 120, Introduction to Programming
CIS 160, Discrete Mathematics
Co-requisite: CIS 121, Data Structures

Textbooks

Spark: The Definitive Guide, by Bill Chambers and Matei Zaharia (O'Reilly)
ISBN 9781491912218; read online for free, or buy for approx. $54.

Data-Intensive Text Processing with MapReduce, by Jimmy Lin and Chris Dyer (Morgan & Claypool)
ISBN 978-1608453429; read online for free, or buy for approx. $40.

Additional materials will be provided as handouts or in the form of light technical papers.

Grading

Policies

You are encouraged to discuss your homework assignments with your classmates; however, any code you submit must be your own work. You may not share code with others or copy code from outside sources, except where the assignment specifically allows it. Plagiarism can have serious consequences.

Recordings and other materials

We will make the recordings from the lectures, Q&A sessions, and labs available on this web page for the duration of this course. These recordings, as well as the other course materials (slides, handouts, framework code) are solely for your personal, educational use and may not be shared, copied, or redistributed without permission of the instructors. You are not allowed to record class sessions yourself. Unauthorized sharing or recording is a violation of the Code of Academic Integrity.

Project and awards

The final team project is to build a small Facebook-like application using Node.js and Amazon's DynamoDB. Based on network analysis, the application should make friend recommendations; it should also visualize the social network. In previous years, Facebook sponsored an award for the best term project. You can learn more about the winners from previous years in the Hall of Fame.

Assignments

Homework assignments will be available for download; you can submit your solution here. If necessary, you can request an extension.

Tentative schedule