CSE 559A
Computer Vision
Computer Vision

FL2018: Tue/Th 11:30‑1 @ Lopata 101


This course introduces the fundamentals of designing computer vision systems—that can "look at" images and videos and reason about the physical objects and scenes they represent. We will learn about methods for image restoration and enhancement; for estimating color, shape, geometry, and motion from images; and for image segmentation, recognition, and classification. The focus of the course will be on the mathematical tools and intuition underlying these methods: models for the physics and geometry of image formation, and statistical and machine learning-based techniques for inference.

Syllabus*

*Caveat: The schedule is tentative, and may change over the course of the semester.

Problem sets, proposals, and reports are due at 11:59 PM (Central) on the dates indicated.

Aug 28 PS0 Out
Introduction. Cameras & Image Formation I.
Aug 30
Image Formation II. Per-pixel Image Operations. Convolutions I.
Sep 4
Convolutions II. Edge & line detection.
Sep 6 PS0 Due
Other spatial image operations. Fourier Transforms & Convolution Theorem.
Sep 11 PS1 Out
Scale & Multi-scale Representations. Efficient Convolutions.
Sep 13
Image Restoration & Optimization.
Sep 18
Restoration & Optimization II. Color, spectrum, and illumination.
Sep 20
Material & Shading: Lambertian, specular, BRDFs.
Sep 25 PS1 Due PS2 Out
Light Sources. Photometric Stereo.
Sep 27
Camera Projection & Geometry I.
Oct 2
Camera Projection & Geometry II.
Oct 4
Multi-view Geometry. Epipolar lines and Rectification.
Oct 9 PS2 Due PS3 Out
Stereo Matching & Globalization.
Oct 11
Stereo Globalization II.
Oct 16
Fall Break
Oct 18 Project Proposals Due
Optical Flow.
Oct 23
Segmentation & Grouping I.
Oct 25 PS3 Due PS4 Out
Segmentation & Grouping II.
Oct 30
Intro to ML I.
Nov 1
Intro to ML II.
Nov 6
Loss Functions & Learning with SGD.
Nov 8
Neural Networks & Back-prop I
Nov 13 PS4 Due PS5 Out
Neural Networks & Back-prop II
Nov 15
Convolutional Layers & Architectures.
Nov 20
Training in Practice: initialization, regularization, batch-normalization.
Nov 22
Thanksgiving
Nov 27
Image Classification, Object Detection, Semantic Segmentation. Fully Convolutional Networks.
Nov 29 PS5 Due
Neural Networks for Physical Tasks: Depth Estimation, Stereo, Image Restoration, Illuminant Estimation, ...
Dec 4
Networks on Point Clouds. Generative Adversarial Networks.
Dec 6
Semi-supervised and Un-supervised Learning. Domain Adaptation.

Final Project Reports Due Dec 9th

Policies


Grade: The grade will be based on five problem sets (15% each for a total of 75%), and a final project (25%). There will be no final exam. Students may receive up to 5% of extra credit for in-class and piazza participation.

Late Policy: All problem sets are due by 11:59 pm of the due date (although we don't suggest waiting till then to submit). You get three "late days" total for the course. Beyond that, you will lose 25% for each day a submission is late (this is quantized to days: anything that is submitted between 1 minute and 24 hours after the due date will be penalized 25%). Late penalties will not be reflected in the initial grade for each set, but will be applied at the end of the term after factoring in the late days (in a way that maximizes your points). Late days CAN NOT be applied to the final project proposal or report submission.

Collaboration and Academic Honesty: All problem sets, and the final project, are expected to be completed individually. All code and written work you submit must be your own. Discussion about course topics with your classmates is encouraged (in person, and on piazza), but remember:

  1. Do not look for answers to the problem set questions online !
  2. Do not share your code, or snippets of code, for any of the problem sets with anyone (to help them or to seek help).
  3. Do not use external libraries or code shared online in any of the problem sets. It is fine to look at general python/numpy/scipy manuals, but not for specific solutions to the questions. When in doubt, ask the course staff !
  4. Do not carry away any written notes, pictures of a whiteboard, or examples of code away from a discussion session. And do something else for an hour before going back to the problem set. Make sure you aren't solving the problem set from memory.
  5. In every problem set you turn in, acknowledge all sources of assistance, as well as names of classmates you had a discussion with.
  6. You may use external resources (including code) for the final project when appropriate, provided these sources are also properly acknowledged.
Follow both the letter and spirit of this policy. The problem sets account for most of your grade, and it is important we be able to evaluate how well you, personally, have understood the course material. Discussions with classmates should be to gain a better understanding of the course material in general, not the specific problems in the problem sets. If you are stuck on a problem, ask for help from the course staff instead of from your classmates (or online). If a classmate asks for help and you can't be sure you can help them without "revealing" a crucial part of the answer, ask them to contact us.

We will be using both automated and manual checks for plagiarism and cheating. If you use a resource or collaborate in violation of this policy, then:

  • If you have acknowledged and reported this in the submitted problem set, you will get a zero for the affected problem(s), but this will not be considered a case of academic dishonesty.
  • If you have not acknowledged this and are discovered, this will be considered cheating, reported to the school, and will likely invite far stricter sanctions. In comparison, turning in a partially complete problem set is a much better option.

Start early on problem sets, and if you need help, ask us !

Problem Sets


Problem sets will be distributed and collected through GIT repositories. Please install git on your machine, if it's not already there. You will access the homework repositories using SSH using public key authentication. If you are not familiar with this, look at this tutorial. Once you have a public-private keypair, use the form below to submit your public key (DO NOT send us your private key). You will need to copy the line containing your public key into a text box in the form (this will typically be the line in ~/.ssh/id_rsa.pub that begins with ssh-rsa).

Submit Public Key
 

Once we have received your public key, you will be able to access each homework, once it's been posted, by "cloning" a repository as:
 
$ git clone cse559@euclid.seas.wustl.edu:wustl.key/psetN
 
Here, wustl.key is the username part of your WUSTL key that you submitted in the public-key submission form (i.e., if your e-mail address is jane.doe@wustl.edu, use jane.doe in place of wustl.key above and in the form). psetN will be name of the problem set (e.g., pset0, pset1, etc.). Once you do a clone, you will see a new sub-directory psetN where you ran the clone command with the cloned repository.

The repository will contain the problem set in a file called pset.pdf, starter code and data in the code/ directory, as well as a TeX template for the solutions (solution.tex) and the corresponding PDF (solution.pdf). Once you have completed your homework, you should add, commit, and push your solutions to the repository server. Make sure you add the write-up solution.pdf as well as the updated .py files in the code/ directory. You should not add the generated output image files to the repository (these should be included in the write-up file).

Note that just committing the files is not enough, because that just updates the local repository on your machine. You must push the files for your submission to be complete. After the repository receives your push, it will create an empty commit on top with comments detailing the timestamp and the list of files receive. So, you need to do a pull immediately after, and use git log to verify your submission. The whole process will look something like this (run this within the psetN sub-directory created by clone command):

$ git add solution.pdf code/*.py
$ git commit -m "submission"
$ git push
$ git pull
$ git log

Once we have graded your problem set, the grades will be posted on Canvas, and a file with your grade + comments will be added to the repository. You can do a git pull to retrieve these.

The coding portion of the assignments will require the use of the Anaconda distribution of Python + NumPy, SciPy. Please download the Python 3.6 version of the latest Anaconda distribution here. The write-ups for all homeworks must be done in LaTeX. Please see the LaTex portion of the resources section if you haven't used LaTeX before. For questions that require math derivations, you will need to TeX up your answers. Scanned versions of hand-written equations ARE NOT acceptable.

Note that pset0 is a simple un-graded homework that will help test your git, anaconda, and LaTex setup. Please complete it as soon as possible, and contact course staff if you have any issues.

Resources


Staff & Office Hours

  • Instructor: Ayan Chakrabarti. Office Hours. Wed: 9:30-10:30 AM @ Jolley 205.
  • Graduate Assistant: Zhihao Xia. Office Hours. Fri: 10:30 AM-Noon @ Jolley 309.
  • Teaching Assistants: Charlie Wu & Han Liu. Office Hours. Mon: 5:30-6:30 PM @ TBD.
The slides, syllabus, and problem sets are based on excellent computer vision courses taught elsewhere by Todd Zickler, Bill Freeman, Svetlana Lazebnik, James Hays, Alyosha Efros, Subhransu Maji, and many many others. The bionic hand/sphere image at the top of the page is Katsuhiro Otomo's "After Escher".