
| Instructors | Chris Gill (Virtual office hours Mondays and Wednesdays
8:30-9:45am and by appointment) Marion Sudvarg (Virtual office hours Mondays and Wednesdays 8:30-9:45am and by appointment) |
| Teaching Assistants | Uki Malla (Virtual office hours Mondays, Wednesdays, and Fridays 6-7pm) |
| Course Web Site | http://www.cse.wustl.edu/~cdgill/courses/cse422_fl20/ |
| Piazza Page | https://piazza.com/wustl/fall2020/cse422s/home |
| Course Meetings | Due to the
Covid-19 pandemic, CSE 422S Operating Systems Organization will be
conducted entirely on-line this semester, using significant additional
infrastructure based on tools like Canvas, Zoom, and Kaltura to
support effective on-line interactions and active learning even from
remote locations. The regularly scheduled class meetings will be held
on Mondays and Wednesdays 8:30am - 9:45am, by university policy ending
5 minutes earlier this semester than is listed in the bulletin to
facilitate students' transitions between classes.
Class lectures will be recorded and made available on specific pages of the course modules in Canvas. The regularly scheduled class meetings (Mondays and Wednesdays 8:30am - 9:45am) will be used as virtual office hours, beginning with a brief summary of tips, traps, and pitfalls for the assigned studios and labs, and nuances of the assigned readings and other materials, with an opportunity for students enrolled in the course to ask questions via zoom. Following that, the instructors will be available to answer questions about the studios and labs, to help debug issues students have run into while doing those, and otherwise to facilitate group work on the studios and/or labs. Students will be expected to keep up on the readings and recorded lectures, to use Piazza and other course resources, and to attend sufficient virtual office hours on Zoom each week, in order to be able to understand and apply the technical material in the course, to ask and get answers to questions about the material, to prepare for the exams, and to get help debugging studio and lab assignments. Each student is responsible for determining how much engagement in each of those modes is needed for them to learn and perform effectively in the course, though attending at least a couple of virtual office hour sessions per week (ideally the scheduled class times) is strongly encouraged as a baseline. Students who will be on campus this semester, or who
require accommodations due to the Covid-19 pandemic, should please
read and follow Washington University's COVID-19
Health and Safety Protocols and other related policies. |
| Take Home Exam 1 |
Released in Canvas 8:30am Thu Oct 8th, Due in Canvas 8:30am Fri Oct 9th (24 hours later) |
| Take Home Exam 2 |
Released in Canvas 8:30am Thu Nov 5th, Due in Canvas 8:30am Fri Nov 6th (24 hours later) |
| Take Home Exam 3 |
Released in Canvas 8:30am Thu Dec 3rd, Due in Canvas 8:30am Fri Dec 4th (24 hours later) |
| Prerequisites | CSE 361S (or graduate student standing) and C/C++ programming experience are strictly required CSE 332S/504N is encouraged but not required |
The objectives of this course are for each student to:
This is a hands-on 400-level course dealing with complex system behaviors, using off-the-shelf platforms and software. Hiccups are to be expected, and finding previously unknown flaws in (or improvements to) the course exercises is encouraged, as the instructors pursue ongoing refinement of the content and scope of this course.
The following are recommended but not required:
Most class periods are accompanied by additional assigned readings (and possibly additional suggested readings that may be useful). The Linux kernel is notable in that many of the discussions (and disagreements) among the original developers have been saved verbatim in repositories such as the the Linux Kernel Mailing List (LKML) or documented by firsthand witnesses at sites such as lwn.net. The course textbooks can be used as technical references for kernel mechanisms, and the suggested supplemental readings can be used to understand particular design choices affecting the Linux kernel.
The course schedule is as follows. If any changes are made to this schedule, students will be notified both in class and through e-mail, and will be given enough advance notice so that rescheduling readings and other preparation can be accommodated readily.
| # | Date | Topic | Readings/Resources | Studio/Lab |
|---|---|---|---|---|
| 1 | Mon, Sep 14th |
Academic Integrity Introduction to Linux |
LKD chapters 1,2
A Brief Overview of the Linux Source Tree |
Building the Linux kernel |
| 2 | Wed, Sep 16th | Setting up the Raspberry Pi 3 | LPI chapters 1-3 | Setting up the Raspberry Pi 3 |
| 3 | Mon, Sep 21st | Program execution, linking, layout | LPI pp. 515-522 and 563-571 LKD pp. 29-33 and 38-40 The inside story on shared libraries and dynamic loading Read these man pages (opened when you run these commands) on a Linux machine: man 3 forkman 3 execman 1 lddman 1 nmman 1 readelfman 1 pmap
|
Program Execution and Debugging |
| 4 | Wed, Sep 23rd | Inter-process communication: signals | LPI chapters 20, 21.1, 21.4-21.6, 22.1-22.8 Read these man pages (opened when you run these commands) on a Linux machine: man 7 signalman 2 sigactionman 2 writeman 2 getrlimit
|
Linux Signals |
| 5 | Mon, Sep 28th | Inter-process communication: pipes, fifos, and sockets | LPI chapters 44.1-44.3, 44.6-44.11, 56.1-56.6, 57.1, 57.2, 57.4,
58.5, 58.6.1, 59.4-59.6, 59.11
The LPG pages on Pipes and FIFOs Read these man pages (opened when you run these commands) on a Linux machine: man 2 pipeman 2 readman 2 writeman 3 mkfifoman 2 bindman 7 unixman 7 ip
|
Linux pipes, FIFOs, and sockets |
| 6 | Wed, Sep 30th | I/O multiplexing
mechanisms and performance considerations Overview of the Lab 1 Assignment |
LPI chapter 63 LKD Chapter 14 |
I/O event handling Lab 1 Assigned (due Fri, Oct 23rd at 11:59PM) |
| 7 | Mon, Oct 5th | Time sources and timing | LKD chapter 11
LPI chapter 23.1-23.6.6 Description of the Timer Wheel System High Resolution Timers Subsystem hrtimer Patch (Formerly ktimers) Kernel Timing and Timers Source Code Pointers |
Userspace benchmarking |
| 8 | Wed, Oct 7th | Exam 1 Review
All studios assigned so far are due by Wed, Oct 7th at 11:59PM Take Home Exam 1 released in Canvas 8:30am Thu, Oct 8th, due in Canvas by 8:30am Fri, Oct 9th (24 hours later) |
||
| 9 | Mon, Oct 12th | How and when does the kernel run? | LKD chapter 5
Userspace / Kernelspace API Split System Call Source Code Pointers |
Creating a syscall |
| 10 | Wed, Oct 14th | Kernel tracing and debugging | LKD chapter 18: pp. 363-371
Ftrace overview document Kernelshark Ftrace Source Code Pointers |
Tracing with ftrace and Kernelshark |
| 11 | Mon, Oct 19th | Structure and infrastructure of the Linux kernel | LKD chapter 6 and pp. 337-348 | Writing a kernel module
|
| 12 | Wed, Oct 21st | Processes | LKD chapter 3
LPI chapters 24.1-24.2, 24.4-24.6, 25, 26.1.1-26.1.4, and 26.2-26.4 Linux Process Source Code Pointers |
Process Family Tree Lab 1 due Fri, Oct 23rd at 11:59PM |
| 13 | Mon, Oct 26th | Traditional Process Scheduling Overview of the Lab 2 Assignment |
LKD chapter 4
LPI chapter 35: sections 35.1 and 35.4 |
Completely Fair Scheduling Lab 2 Assigned (due Fri, Nov 20th at 11:59PM) |
| 14 | Wed, Oct 28th | Real-time process scheduling | LKD chapter 4: pp. 64-65
LPI chapter 35: sections 35.2, 35.3, and 35.5 Deadline Scheduling vs POSIX Real-time Scheduling The on-line man 7 sched page. | Real-time Scheduling |
| 15 | Mon, Nov 2nd | Interrupts and interrupt handlers | LKD chapter 7 Linux source: current softirqs Linux source: current irq flags Why IRQF_DISABLED is no longer used Threaded interrupt handlers Thread prioritization in kernel modules |
In-class lab (or studio catch up) time |
| 16 | Wed, Nov 4th | Exam 2 Review
All studios assigned since Exam 1 are due by Wed, Nov 4th at 11:59PM Take Home Exam 2 released in Canvas 8:30am Thu, Nov 5th, due in Canvas by 8:30am Fri, Nov 6th (24 hours later) |
||
| 17 | Mon, Nov 9th | Bottom half processing | LKD chapter 8
Software Interrupts and Real-time |
In-class lab time |
| 18 | Wed, Nov 11th | Kernel synchronization I | LKD chapter 9 | In-class lab time |
| 19 | Mon, Nov 16th | Kernel synchronization II | LKD chapter 10 What is RCU, fundamentally? Kernel Reference on RCU |
Kernel Races, Atomicity, and Synchronization |
| 20 | Wed, Nov 18th | Userspace synchronization | LPI chapters 30 and 31 Read these man pages (opened when you run these commands) on a Linux machine: man 7 futex
man 2 futex |
Build Your Own Locks
Lab 2 due Fri, Nov 20th at 11:59PM |
| 21 | Mon Nov 23rd | Virtual memory and paging Kernel memory Overview of the Lab 3 Assigment |
LKD chapter 12 | Kernel memory management Lab 3 Assigned (due Fri, Dec 18th at 11:59PM) |
| 22 | Wed Nov 25th | Address spaces / shared memory | LKD chapter 15
Read these man pages (opened when you run these commands) on a Linux machine: man 2 mmap
man 3 shm_open |
Shared memory management |
| 23 | Mon, Nov 30th | Virtual filesystem | LKD chapter 13 | VFS layer |
| 24 | Wed, Dec 2nd | Exam 3 Review
All studios assigned since Exam 2 are due by Wed, Dec 2nd at 11:59PM Take Home Exam 3 released in Canvas 8:30am Thu, Dec 3rd, due in Canvas by 8:30am Fri, Dec 4th (24 hours later) |
||
| 25 | Mon, Dec 7th | No lecture or assigned readings: in-class lab time | ||
| 26 | Wed Dec 9th | No lecture or assigned readings: in-class lab time | ||
| 27 | Mon, Dec 14th | No lecture or assigned readings: in-class lab time | ||
| 28 | Wed, Dec 16th | No lecture or assigned readings:
in-class lab time Lab 3 due Fri, Dec 18th at 11:59PM |
||
There will be three lab assignments for this course. The purpose of these labs is to apply course concepts and to evaluate kernel mechanisms and behaviors. As such, each lab will require a written report detailing your findings in addition to the code you wrote.
Each lab will be completed in a team of up to three students, and teams may be different for each lab or may remain the same. Students from different teams may discuss the lab assignments only during scheduled zoom sessions that the instructors are attending, when anyone is able to join in those discussions. Students on the same team are of course encouraged to discuss and work on lab assignments at any time.
Labs submitted on time (as determined by timestamps on Box) will be given full credit. Labs submitted up to 24 hours late will be given a ten percent penalty. Labs submitted between 24 and 48 hours late will be given a twenty percent penalty. Labs submitted after 48 hours late will not be given credit, except in the case of extenuating circumstances approved by the instructor. If you anticipate that your team may need an extension on a lab assignment, please make sure to ask prior to the deadline.
Current and past instructors, TAs, and students of the CSE 422/522 courses have contributed various tips, tricks, and solutions to problems that they have encountered. Please let your instructor know if you have something you'd like to add here!
What if your Pi is freezing up? Accessing your Raspberry Pi Remotely
To be able to complete the studios and lab assignments this semester, you will need to purchase a CanaKit Raspberry Pi 3 B+ Starter Kit (32 GB EVO+ Edition), which is the platform on which we have tested out our solutions to those assignments.
The in-class studios and assigned readings offer a reasonable coverage of Linux. However, the studios and assigned readings will not yet make you an expert on the Linux kernel. When it comes to learning the details of a code base as deep and complex as Linux, there is no substitute for reading source code. Some of the lectures and studios will give pointers to relevant source code files, and for overall understanding it is expected that students will spend some time absorbing the content there as well.
To be clear, you are not expected to understand or memorize every line of the Linux source. The exams will not have questions derived directly from the Linux source (though basic kernel concepts and pseudo-code are fair game). However, the kernel is a huge set of deeply interdependent source code files. A useful way to understand it more and more thoroughly is by looking through it many times.
There are two required textbooks for this course:
Linux Kernel Development by Robert Love, 2010 (noted as LKD in the assigned readings). This is an excellent, compact, and inexpensive text that gives the reader a basic but also reasonably detailed understanding of kernel design, written in depth by a Linux kernel expert.
The Linux Programming Interface by Michael Kerrisk, 2010 (noted as LPI in the assigned readings). This an excellent, somewhat more extensive reference manual, also written in depth by a Linux kernel expert.
Please note that the Linux kernel source code is frequently (and sometimes rapidly) updated, so that while the general information in these textbooks is correct, many details of the source code itself, the names of source code files, and where those files exist, are somewhat likely to change (and have done so since the required text books above and the additional references below were written).
There are a number of other references that you may find useful for this course and more broadly for your study and use of the Linux kernel.
shell.cec.wustl.edu or on one of the Linux lab machines, you can issue the command man man to see more about those manual pages, or issue the appropriate man command that's listed in the assigned readings in the course syllabus, to see a specific manual page.
/usr/include/ directory
on it are worth looking through in an editor of your choice, to see
the declarations of various functions and variables that also are
shown in the relevant
man pages we've assigned in the readings. Emacs (you may
need to run sudo apt-get install emacs to obtain and
install it on your Raspberry Pi) is one good general purpose editor
that also can be used to run gdb and other useful tools
from within it.
Students will need to acquire a Raspberry Pi 3 B+ in order to complete daily studios and lab assignments. To increase flexibility during the Covid-19 pandemic, so that students can work from remote locations as well as on campus, the studio and lab instructions in this course have been redesigned to support a variety of different settings in which the Raspberry Pi 3 B+. If you run into any problems using your Raspberry Pi 3 B+ in this course this semester please let us know right away so we can help.
There are three activities for which you will receive credit in this course: studios, exams, and labs. Studios are daily guided assignments primarily designed to familiarize you with course concepts, development tools, and the kernel source code (i.e. knowledge and comprehension tasks). Exams will evaluate your technical understanding of course concepts, including nuances of the material covered in the lectures and in the assigned readings. The lab assignments will ask you to apply course concepts and analyze and implement design alternatives.
We will make every effort to grade your work, and to post a grade in Canvas and provide comments to you regarding it, within one week (7 days) of when it is submitted.
Especially for the early studios, even though they are not due until just before the first exam, please do work to submit them as early as you can so that you'll have grades and feedback on at least a few of them prior to the end of the semester's drop/add period.
Your semester grade will be determined as follows, with an overall weighted average of 90% corresponding to at least an A-, of 80% corresponding to at least a B-, and of 70% to at least a C- and to a passing grade if the course is being taken P/F rather than for a letter grade:
| Activity | Grade Percentage |
|---|---|
| Studios | 25% |
| Exam 1 | 10% |
| Exam 2 | 10% |
| Exam 3 | 10% |
| Lab 1 | 10% |
| Lab 2 | 15% |
| Lab 3 | 20% |
Especially with the challenges many students may face in an on-line learning environment, we encourage everyone to take a look at, and potentially make use of, the many academic support resources available at Washington University in St. Louis, including: The Learning Center, and The Writing Center, and Student Technology Services.
Washington University recognizes that students serving in the U.S. Armed Forces and their family members may encounter situations where military service forces them to withdraw from a course of study, sometimes with little notice. Students may contact the Office of Military and Veteran Services at (314) 935-2609 or veterans@wustl.edu and their academic dean for guidance and assistance.
The Habif Health and Wellness Center, WashU Cares, Mental Health Services, and RSVP are important university resources for issues pertaining to physical and mental health.
Students with disabilities or suspected disabilities are strongly encouraged both to bring any additional considerations to the attention of the instructor and to make full use of Washington University's Disability Resources, potentially including accommodations for studios, labs, and/or exams.
Washington University is firmly committed to addressing and preventing sexual misconduct on our campuses: please see the Washington University statement on new Title IX rules. If a student discusses or discloses an instance of sexual assault, sex discrimination, sexual harassment, dating violence, domestic violence or stalking, or if a faculty member otherwise observes or becomes aware of such an allegation, the faculty member will keep the information as private as possible, but as a faculty member of Washington University, they are required to report it immediately to their Department Chair or Dean or directly to Ms. Jessica Kennedy, the University's Title IX Director, at (314) 935-3118 or jwkennedy@wustl.edu. Additionally, incidents or complaints can be reported to the Office of Student Conduct and Community Standards or by contacting WUPD at (314) 935-5555 or contacting your local law enforcement agency. To explore options for medical care, protections, or reporting, free and confidential support resources and professional counseling services are available through the Relationship and Sexual Violence Prevention (RSVP) Center in Seigle Hall, Suite 435, which can be reached at rsvpcenter@wustl.edu or at (314) 935-3445. For after-hours emergency response services, call 314-935-6666 or 314-935-5555 and ask to speak with an RSVP Counselor on call.
Washington University's Center for Diversity and Inclusion supports and advocates for undergraduate, graduate, and professional school students from underrepresented and/or marginalized populations, collaborates with campus and community partners, and promotes dialogue and social change to cultivate and foster a supportive campus climate for students of all backgrounds, cultures, and identities. The University has a process through which students, faculty, staff, and community members who have experienced or witnessed incidents of bias, prejudice, or discrimination against a student can report their experiences to the University's Bias Report and Support System team. For procedures and information on reporting an instance of bias, please visit the Bias Report and Support System page on the Center for Diversity and Inclusion web site. In order to affirm each person's gender identity and lived experiences, it is important that we each ask and check in with others about pronouns. This simple effort can make a profound difference in a person's experience of safety, respect, and support. Please see the University's Gender Pronouns Information page and the Office of the Registrar's Preferred Name page for additional information and resources.
Each studio and lab assigned in this course is expected to be completed collaboratively in teams of up to three people (and not more than that), though you are permitted to work individually on any assignment instead, if you prefer. During the scheduled zoom sessions that the instructors are attending, students may be allowed to share their screens to show code for purposes of getting help with debugging, illustrating solution approaches that are being discussed, etc., and communication about a particular studio or lab assignment within your team of (3 or fewer) people who are working together on it is allowed at any time. Communication between different teams is not allowed outside of the scheduled zoom sessions that the instructors are attending. Student teams may change from assignment to assignment, but the sharing of code between teams without prior permission of the instructor is strictly prohibited, and you must acknowledge and document in detail all contributions that anyone has made to the work.
To encourage you to read the Linuxman pages and LPI and LKD text
books carefully, you are allowed to use snippets of code from those
sources in your studios, labs, and exams without asking permission, as
long as (in comments before and after it) you clearly identify the
start and end of each such code snippet and attribute the source from
which it came (including page and/or line numbers where those are
available). For code from any other sources you must first ask for
and receive permission from your professor (and should identify and
attribute it similarly if so).
Exams must be completed individually without assistance from any other person, and without reference to external materials except as is specifically allowed by the instructor (documentation of what is allowed will be described in the review lecture, provided in the corresponding review slides, and written on the font page of the exam).
Cheating costs everyone something. Someone who cheats misses out on the intended opportunity to improve through the assigned work, and like anyone helping them cheat is at risk of diminished reputation as well as specific sanctions (see below). Cheating also degrades the value of the degree earned by those who complete their work with integrity.
Academic integrity is a serious matter in this course, and the instructor will decide whether any situation of concern in this course warrants referral to the appropriate formal academic integrity adjudication process at the school or university level. If in any doubt, please ask your instructor first.
Anyone found to have cheated or to have helped someone else cheat will face penalties that may include receiving a negative score equal in magnitude to the value of the assignment in question (e.g., cheating on an assignment worth 10% of the course grade would result in -10% being assigned for a total loss of 20% from the course grade, which is twice as large a loss as simply not turning in the assignment).
Academic integrity is itself worth studying and thinking about as a key component of your education. Please read, familiarize yourself with, and reflect on the Engineering School's and Washington University's undergraduate and graduate policies on academic integrity.