Engineering.wustl.edu

E81 CSE 422S: Operating Systems Organization

Fall 2017

Acknowledgement: portions of this course and web site have been derived (with permission) directly from materials
that were developed and taught at Washington University by David Ferry in the spring semester of 2016.


Instructor Chris Gill (office hours by appointment)
Course Web Site http://www.cse.wustl.edu/~cdgill/courses/cse422/
Backup Web Site http://classes.cec.wustl.edu/~cse422
Course Meetings Mondays and Wednesdays 8:30 - 10:00 AM in Urbauer 218.
Most days will start with lectures and discussions followed by assisted group work on studios and labs.
Exam reviews will be held in Urbauer 218 and the two in-semester exams will be held in locations TBA. Each of these meetings will occupy the entire meeting time on their designated dates.
Midterm Exam
8:30am Wed Oct 11th, location TBA
Semester Exam
8:30am Wed, Nov 29th, location TBA
Prerequisites CSE 361S, or graduate student standing and C/C++ programming experience (CSE 332S/504N is strongly encouraged)

Contents
  1. Course Description
  2. Prerequisites
  3. Discussions/Studios
  4. Lab Projects
  5. Class FAQ
  6. Textbooks and Other Resources
  7. Grading
  8. Accessibility
  9. Academic Integrity

Course Description

This course intends to give hands-on experience working with the Linux operating system kernel, and covering a breadth of basic operating systems issues. The course will emphasize the developing and applying skills and techniques for profiling, quantifying, and evaluating operating system behaviors based on an understanding of the policies and mechanisms that govern those behaviors.

The objectives of this course are for each student to:

This is a hands-on 400-level course dealing with complex system behaviors, using off-the-shelf platforms and software. Hiccups are to be expected, and finding previously unknown flaws in (or improvements to) the course exercises is encouraged, as the instructor seeks to refine the content and scope of the course.


Prerequisites


Discussions and Studios

This course is conducted in a studio format, both to emphasize the practical and hands-on nature of "kernel hacking" and to engage students in applied active learning exercises. The focus of lecture components is to facilitate in-class discussion of kernel design and features, while the focus of studio exercises is (1) to offer direct experience working with (and within) the kernel and (2) to familiarize students with key tools and techniques for using, profiling, analyzing, and extending kernel features. Studios and the labs may be completed in groups of up to 3 students, and are submitted for credit. All studios assigned before the midterm are due the Friday prior to the midterm, and all studios assigned after the midterm are due on the last day of Engineering School classes.

Most class periods are accompanied by additional suggested readings. The Linux kernel is notable in that many of the discussions (and disagreements) among the original developers have been saved verbatim in repositories such as the the Linux Kernel Mailing List (LKML) or documented by firsthand witnesses at sites such as lwn.net. The course textbook can be used as a technical reference for kernel mechanisms, and the suggested supplemental readings can be used to understand particular design choices affecting the Linux kernel.

The course schedule is as follows. If any changes are made to this schedule, students will be notified both in class and through e-mail, and will be given enough advance notice so that rescheduling readings and other preparation can be accommodated readily.

# Date Topic Readings Studio/Lab
1 Mon, Aug 28th Course Introduction

Academic Integrity
LKD chapter 1 Setting up the Raspberry Pi 3
2 Wed, Aug 30th Building Linux LKD chapter 2
Code Pointers
Building the Linux kernel
Mon, Sep 4th: Labor Day - no classes
3 Wed, Sep 6th How and when does the kernel run? LKD chapter 5
Userspace / Kernelspace API Split
Code Pointers
Creating a syscall
4 Mon, Sep 11th Time sources and timing LKD chapter 11
High Resolution Timers Subsystem
Description of the Timer Wheel System
Introduction of hrtimer Patch (Formerly ktimers)
Code Pointers
Userspace benchmarking
5 Wed, Sep 13th Kernel tracing Ftrace design document
Kernelshark
Code Pointers
Tracing with ftrace and Kernelshark
6 Mon, Sep 18th Structure and infrastructure of the Linux kernel LKD Ch. 6
LKD pp. 338-348
Writing a kernel module
7 Wed, Sep 20th Processes LKD Ch. 3
Code Pointers
Process Family Tree
Lab 1 Assigned (due Fri, Oct 6th at 11:59PM)
8 Mon, Sep 25th Traditional process scheduling LKD Ch. 4 Completely Fair Scheduling
9 Wed, Sep 27th Real-time process scheduling Deadline Scheduling vs POSIX Real-time Scheduling
Software Interrupts and Real-time
Real-time Scheduling
10 Mon, Oct 2nd Interrupts and interrupt handlers LKD Ch. 7 In-class lab (or studio catch-up) time
11 Wed, Oct 4th Top half / bottom half processing LKD Ch. 8 In-class lab (or studio catch-up) time
Lab 1 due Fri, Oct 6th at 11:59PM
12 Mon, Oct 9th Midterm review   All studios assigned so far are due Tue, Oct 10th at 11:59PM
13 Wed, Oct 11th Midterm exam 8:30-10:00am in (location TBA)
Mon, Oct 16th: Fall break - no classes
14 Wed, Oct 18th Midterms returned, discussed
Overview of the lab 2 assignment
  In-class lab time
Lab 2 assigned (due Fri, Nov 10th at 11:59PM)
15 Mon, Oct 23rd Kernel synchronization I LKD Ch. 9 In-class lab time
16 Wed, Oct 25th Kernel synchronization II LKD Ch. 10 Synchronizing Threads in the Kernel
17 Mon, Oct 30th Read-copy-update (RCU) synchronization What is RCU, fundamentally?
Kernel Reference on RCU
Read these man pages (run these commands) on a Linux machine:
man 7 futex
man 2 futex
Build Your Own Locks
18 Wed, Nov 1st Kernel memory LKD Ch. 12 Kernel memory management
19 Mon, Nov 6th Process address space LKD Ch. 15 Program memory management
20 Wed, Nov 8th Program execution, linking, layout   In-class lab (or studio catch-up) time
Lab 2 due Fri, Nov 10th at 11:59PM
21 Mon, Nov 13th Virtual filesystem LKD Ch. 13 VFS layer
Lab 3 assigned (due Fri, Dec 8th at 11:59PM)
22 Wed, Nov 15th Block I/O layer LKD Ch. 14 Block devices
23 Mon, Nov 20th Page cache and page writeback LKD Ch. 16 In-class lab (or studio catch-up) time
Wed, Nov 22nd: Thanksgiving break - no classes
24 Mon, Nov 27th Semester review in Urbauer 218 All studios due Tue, Nov 28th at 11:59PM
25 Wed, Nov 29th Semester Exam 8:30-10am (location TBA)    
26 Mon, Dec 4th No lecture, readings, or studio assigned   Open lab time
Guest lab facilitators: Son Dinh and James Orr
27 Wed, Dec 6th No lecture, readings, or studio assigned   Open lab time
Guest lab facilitators: Son Dinh and James Orr
Lab 3 due Fri, Dec 8th at 11:59PM

Labs

There will be three lab assignments for this course. The purpose of these labs is to apply course concepts and to evaluate kernel mechanisms and behaviors. As such, each lab will require a written report detailing your findings in addition to the code you wrote.

Each lab will be completed in a team of two or three students, and teams may be different for each lab. Students from different teams may discuss the lab assignments only during course meeting times. Students on the same team are of course encouraged to discuss and work on lab assignments at any time.

Labs submitted on time (as determined by our email server's receipt time stamp) will be given full credit. Labs submitted up to 24 hours late will be given a ten percent penalty. Labs submitted between 24 and 48 hours late will be given a twenty percent penalty. Labs submitted after 48 hours late will not be given credit, except in the case of extenuating circumstances approved by the instructor.


Class FAQ

Current and past instructors, TAs, and students of the CSE 522 course have contributed various tips, tricks, and solutions to problems that they have encountered. Please let your instructor know if you have something you'd like to add here!

Class FAQ

   What if your Pi is freezing up?

   Leaving your Pi at home


Textbook and Class Resources

Students should consider the in-class studios and assigned readings to be a guided tour of the Linux kernel. Like any good tour group, we want to see the sights and learn some neat highlights about who built it and what they were trying to accomplish. But, the studios and assigned readings will not yet make you an expert on the Linux kernel. When it comes to learning the details of a code base as deep and complex as this, there is no substitute for reading source code. Lectures and studios will come with pointers to relevant source code files, and it is expected that students will spend time absorbing the content there as well.

To be clear, you are not expected to understand or memorize every line of the Linux source. Exams will not have questions derived from the Linux source (though basic kernel concepts and pseudo-code are fair game). However, the kernel is a huge set of deeply interdependent source code files. A useful way to understand it more and more thoroughly is by looking through it many times.

There is one required course textbook: Linux Kernel Development by Robert Love, 2010 (noted as LKD in the assigned readings). This is an excellent, compact, and inexpensive text that gives the reader a basic understanding of kernel design, written with the depth of a kernel veteran.

There are a number of useful references. None of these will give you the level of competency that comes from looking at code itself, but they are very useful for starting points and clarifying problems. However, the Linux kernel is rapidly updated, so while the general information in these books is correct, details such as source code, names of files, and where files exist are somewhat likely to change.

Students will need a Raspberry Pi 3 in order to complete daily studios and lab assignments. The course has been designed with the intention that students bring these devices to class, plug them in, and work on them there. Monitors, keyboards, mice, and cables will all be provided in the lab for this purpose - students will need to provide the Raspberry Pi 3 and a power cord.

If you so desire, you may set up your Raspberry Pi 3 in your home or office and configure it for remote access. However, the instructor cannot support you in this, and at certain points we will be modifying the kernel so you may be rather lost if you reboot and can't ssh back into your machine. Also, be forewarned that some exercises will require an X11 window, so simple ssh access will not be enough to do everything remotely.


Grading

There are three activities for which you will receive credit in this course: studios, labs, and exams. Studios are daily guided assignments primarily designed to familiarize students with course concepts, development tools, and the kernel source code (i.e. knowledge and comprehension tasks). The lab assignments will ask you to apply general course concepts and analyze and implement design alternatives. The midterm and semester exams will evaluate your technical understanding of course concepts.

Your grade will be determined as follows:

Activity Grade Percentage
Midterm Exam 15%
Semester Exam 20%
Studios 20%
Lab 1 10%
Lab 2 15%
Lab 3 20%

Accessibility

Students with disabilities or suspected disabilities are strongly encouraged both to bring any additional considerations to the attention of the instructor and to make full use of the University's Disability Resource Center (http://disability.wustl.edu), potentially including accommodations for studios, labs, and/or exams.

Academic Integrity

Each studio and lab assigned in this course is expected to be completed collaboratively by two or three people (and not more than that). Student teams may change from assignment to assignment, but the sharing of code between teams is strictly prohibited and you must acknowledge and document in detail all contributions that anyone has made to the work.

Exams must be completed individually without assistance from any other person and without reference to materials or devices, except as specifically allowed by the instructor (documentation of what is allowed will be described in class, provided in the corresponding review slides, and written on the font page of the exam).

Cheating costs everyone something. Someone who cheats misses out on the intended opportunity to improve through the assigned work, and like anyone helping them cheat is at risk of diminished reputation as well as specific sanctions (see below). Cheating also degrades the value of the degree earned by those who complete their work with integrity.

Academic integrity is a serious matter in this course, and anyone found to be cheating or helping someone else cheat will receive a negative score equal in magnitude to the value of the assignment in question (e.g., cheating on an assignment worth 10% of the course grade would result in -10% being assigned for a total loss of 20% from the course grade, which is twice as large a loss as simply not turning in the assignment). Extreme cases also may be referred to disciplinary processes managed by the university. The instructor will make final determinations on what constitutes cheating in this course. If in any doubt, please ask first.

Academic integrity is itself worth studying and thinking about as a key component of your education. Please read, familiarize yourself with, and reflect on the Engineering School's and Washington University's undergraduate and graduate policies on academic integrity.