CSE 559A: Computer Vision (Fall 2017)

[credit: http://danjodon.deviantart.com/]

This course introduces the fundamentals of designing computer vision systems: that can "look at" images and videos and reason about the physical objects and scenes they represent. We will learn about methods for image restoration and enhancement; for estimating color, shape, geometry, and motion from images; and for image segmentation, recognition, and classification. The focus of the course will be on the mathematical tools and intuition underlying these methods: models for the physics and geometry of image formation, and statistical and machine learning-based techniques for inference.



No prior knowledge of computer vision, image processing, or graphics is required. The following skills are necessary for the class:

  1. Math: Linear algebra, vector calculus, and probability.
  2. Programming: The problem sets will have programming questions in python with numpy/scipy. You should have experience writing complex programs, using data structures, etc. You don't need prior experience with python itself, but should be willing to put in the time to pick it up . We will help with pointers, starter code for the problem sets, and support.

Additionally, prior experience with signal processing and machine learning will be helpful (but isn't necessary).


Late Policy: All problem sets are due by 11:59 pm of the due date (although we don't suggest waiting till then to submit). You get three "late days" total for the course. Beyond that, you will lose 25% for each day a submission is late (this is quantized to days: anything that is submitted between 1 minute and 24 hours after the due date will be penalized 25%). Late penalties will not be reflected in the initial grade for each set, but will be applied at the end of the term after factoring in the late days (in a way that maximizes your points).

Collaboration Policy: All problem sets, and the final project, are expected to be completed individually. All code and written work you submit must be your own. Discussion about course topics with your classmates is encouraged (in person, and on the course discussion forum), but remember:

  1. Do not share your code for any of the problem sets with anyone (to help them or to seek help).
  2. Do not use external libraries or code shared online in any of the problem sets. It is fine to search online for general numpy/python help, but not for specific solutions to the questions. Better yet, ask the course staff instead !
  3. Do not carry away any written notes, pictures of a whiteboard, or examples of code away from a discussion session. And do something else for an hour before going back to the problem set. Make sure you aren't solving the problem set from memory.
  4. In every problem set you turn in, acknowledge all sources of assistance, as well as names of classmates you had a discussion with.
  5. You may use external resources (including code) for the final project when appropriate, provided these sources are properly acknowledged.

Follow both the letter and spirit of this policy. The problem sets account for most of your grade, and it is important we be able to evaluate how well you, personally, have understood the course material. Discussions with classmates should be to gain a better understanding of the course material in general, not the specific problems in the problem sets. If you are stuck on a problem, ask for help from the course staff instead of from your classmates (or online). If a classmate asks for help and you can't be sure you can help them without "revealing" a crucial part of the answer, ask them to contact us.


There are no required textbooks for the course. The following are useful for reference:

  1. Computer Vision: Algorithms and Applications by Richard Szeliski. (Follow the link for a free online copy)
  2. Computer Vision: A Modern Approach by David Forsyth and Jean Ponce.

Syllabus (tentative)

Problem sets, proposals, and reports are due at 11:59 PM (Central) on the dates indicated.
Week Tuesday Thursday
1 Aug 29

Introduction. Cameras & Image Formation I.

[Slides] [PDF] [PDF/4]

Aug 31

Image Formation II. Per-pixel Image Ops. Convolutions I.

[Slides] [PDF] [PDF/4]

2 Sep 5

Convolutions II. Edge & line detection. Other spatial image ops.

[Slides] [PDF] [PDF/4]

Sep 7

Fourier Transforms & Convolution Theorem.

[Slides] [PDF] [PDF/4]

3 Sep 12 PS1 Out

Scale & Multi-scale Representations. Efficient Convolutions. Image Restoration.

[Slides] [PDF] [PDF/4]

Sep 14

Image Restoration II. Optimization.

[Slides] [PDF] [PDF/4]

4 Sep 19

Color, spectrum, and illumination.

[Slides] [PDF] [PDF/4]

Sep 21

Material & Shading: Lambertian, specular, BRDFs.

[Slides] [PDF] [PDF/4]

5 Sep 26 PS1 Due! PS2 Out.

Photometric Stereo. Camera Projection & Geometry.

Sep 28

Feature matching & view alignment.

6 Oct 3

Binocular Stereo. Smoothness & Graphical Models. Global Optimization I.

Oct 5 Project Proposals Due.

Global Optimization II.

7 Oct 10 PS2 Due! PS3 Out.

Optical Flow I.

Oct 12

Optical Flow II.

8 Oct 17

Fall Break: No class.

Oct 19

Segmentation & Grouping I.

9 Oct 24

Segmentation & Grouping II.

Oct 26 PS3 Due! PS4 Out.

Classification and Recognition. ML Recap: Linear classifiers & Loss functions.

10 Oct 31

Intro to Deep Neural networks. Learning with Stochastic Gradient Descent.

Nov 2

Backpropagation: automatic differentiation in a computation graph.

11 Nov 7

Network Architectures I.

Nov 9 PS4 Due! PS5 Out.

Over-fitting & Regularization. Batch Normalization.
Multi-task Training.

12 Nov 14

Network Architectures II: Recurrent & Very Deep Networks.

Nov 16

Beyond SGD: adaptive and distributed optimizers.

13 Nov 21

Generative Adversarial Networks.

Nov 23

Thanksgiving Break: No class.

14 Nov 28 PS5 Due!

Recent advances.

Nov 30

Recent advances.

15 Dec 5

Project Presentations.

Dec 7 Project Final Reports Due.

Project Presentations.



The slides, syllabus, and problem sets are based on excellent computer vision courses taught elsewhere by Todd Zickler, Bill Freeman, Svetlana Lazebnik, James Hays, Alyosha Efros, Subhransu Maji, and many many others.