CSE 659A Advances in Computer Vision

SP2020: Tue/Th 1:00‑2:20pm @ Whitaker 216

Computer vision is a fast moving field, with the past few years seeing tremendous advances in the development of computational algorithms for solving visual tasks. This course is designed to introduce you to advanced and recently published techniques for problems in low-level vision, recognition and classification, and computational photography.

Pre-requisites: CSE 559A

Instructor: Ayan Chakrabarti


Jan 14Course Intro & norm penalty gradient regularizers.HTML PDF 1UP PDF 4UP
Jan 16Optimization: IRLS & HQS. Intro to Learned Priors.HTML PDF 1UP PDF 4UP
Jan 21Gaussian Mixture Model-based Priors: Learning and Inference.HTML PDF 1UP PDF 4UP
Jan 23Sparse Dictionaries. CNN-denoiser priors. Intro to MRFs. Fields of Experts.HTML PDF 1UP PDF 4UP


*List of topics are tentative and may change. In the last month, one class per week will be for student project presentations.

  1. Image Priors & Spatial Models
    • Gradient regularizers, patch models based on GMMs, sparse dictionaries, field-of-experts, etc. Optimization algorithms for inference. Restoration with plug-and-play denoisers.
    • Graphical Models and Markov Random Fields (MRFs). Inference algorithms including loopy belief-propagation, graph cuts with expansion and swap moves, etc. Mean field with efficient data-structures for fully-connected MRFs.
    • Combining neural networks outputs with MRF models. Incorporating MRF inference within a network.
  2. Depth and Motion Estimation
    • Un-rectified and multi-view stereo with plane sweep.
    • Inference with planarity and higher-order priors on depth.
    • Large displacement optical flow, and layered models for flow.
    • CNN-based methods for stereo and flow.
    • Monocular Depth and Normal Estimation.
  3. Classification & Recognition
    • Interest point detectors. Traditional region (SIFT, HoG) and scene (GIST) descriptors.
    • Content-based Image Retrieval. Object Detection with the Deformable Parts Model.
    • CNN-based object detection. Image and Instance Segmentation.
    • Image Captioning. Visual Question Answering.
  4. Computational Photography I
    • Texture Synthesis. Seam Carving.
    • Image Harmonization. CG2Real.
    • Motion Magnification.
    • Photo UnCrop. Image Inpainting. Image Editing with Smart Contours.
  5. Advanced Photometric Reasoning
    • Uniqueness results: when does shading determine shape ?
    • Modern algorithms for shape from shading, intrinsic image decomposition, and photometric stereo.
    • Neural network based Color Constancy. Lighting separation with flash/no-flash.
  6. Computational Photography II
    • Dark flash photography.
    • Coded aperture and coded exposure photography. Light-field cameras.
    • Structured light, Time-of-Flight cameras.
    • Other non-traditional cameras.


Grade: Evaluation in this course will be based on the following

  • 25%: Five homework "paper reviews". You will be asked to read a paper and write a short (1-2 page) review.
    The following is a tentative schedule of when the papers will be assigned and reviews due:
    • Review 1. Assigned: Jan 23, Due: Feb 6.
    • Review 2. Assigned: Feb 6, Due: Feb 20.
    • Review 3. Assigned: Feb 20, Due: Mar 19.
    • Review 4. Assigned: Mar 19, Due: Apr 2.
    • Review 5. Assigned: Apr 2, Due: Apr 21.
  • 60%: Two Projects (30% each).
    • Project I: Topics 1-3. Report Due: Mar 3.
    • Project II: Topics 3-5. Report Due: Apr 24.
  • 15%: Presentation (based on project I). Presentation dates will be assigned randomly and posted during the semester. Irrespective of the presentation date, slides for the presentation for all students will be due before the first presentation date (~ in late March).
All submissions will be through Canvas. The final grade boundaries will be decided at the end of the course based on the distribution of scores.

Late Policy: All homework reviews, project reports, and presentation slides must be submitted by 11:59 PM on their due dates. There will be no extensions given. We recommend you submit early leaving a buffer of a few day to account for unexpected delays.

Collaboration and Academic Honesty: Discussion about course topics with your classmates is encouraged (in person, and on piazza) but all homework reviews are projects are expected to be completed individually. While completing the projects, it is OK to rely on code posted online, but this should be acknowledged in the project report. Any instances of plagiarism will be reported to the school, and will attract strict penalties.

Office Hours: Are by appointment. Please make a private post on Piazza, and let me know what times in the next 2-3 days work for you. I'll get back to you with a time slot.