ECE 5554: Computer Vision, Fall 2013

Instructor: 
Devi Parikh
Email: parikh@vt.edu
Office: 440 Whittemore Hall
Office hours: Fridays from 2:30 PM to 3:30 PM

TA: 
Neelima Chavali
Email: gneelima@vt.edu
Office Hours: Mondays and Wednesdays from 11:00 AM to 12:00 @ WHIT 215

Announcements      Course overview      Requirements      Schedule      References

Announcements:

*** Best Project Prizes ***

Computer Vision Applications for UAV Control, Mapping and Road Recognition (Matt Frauenthal and Scott Radford)
Optical Character Recognition for Math Equations (Allan Kirchhoff)

Welcome to ECE 5554!

Course overview:

"To see is to know what is where by looking"- David Marr

Over the past few decades, machines have come a long way in their ability to "see". Some examples are autonomous navigators like Mars Rover and Google car, medical imaging technologies, pedestrian detectors in vehicles, image search engines, face detectors in cameras and Facebook, aids for the visually impaired, control-free video games, and industrial automation systems.

In this introductory Computer Vision course, we will learn how to "teach machines to see". We will explore several fundamental concepts including image formation, feature detection, segmentation, multiple view geometry, recognition, and video processing. We will use these concepts to build applications that aid machines to see the world around them.

Requirements:

Problem Sets (55% of final grade): You will be given 6 problem sets, one approximately every two weeks. These will involve a combination of conceptual questions and programming problems. The programming problems will provide hands-on experience working with techniques covered in or related to the lectures. All code and written responses must be completed individually and submitted to Scholar. Most problem sets will take significant time to complete. Please start early. Problem Set 0 (PS0) will be worth 5% the final grade, and the remaining 5 problem sets will be worth 10% each.

Project (25% of final grade): Your project can be about applying any of the techniques we studied in class to real world problems. You can also extend a technique, or empirically analyze it. Comparisons between two approaches are also welcome. It is wonderful if you design and evaluate a novel approach to an important existing or new vision problem. Be creative! You are welcome to work with a partner. If you need help with ideas for your project please come talk to the instructor and/or TA. The following are deliverables for your project:

• Proposal (10% of project grade): A 1-page description that describes the following:
• Problem statement: Clearly state the goal of your project.
• Approach: Describe the technical approach you plan to employ. 
• Experiments and results: Describe the experimental setup you will follow, which datasets you will use, which existing code you will exploit, what you will implement yourself, and what you would define as a success for the project. If you plan on collecting your own data, describe what data collection protocol you will follow. Provide a list of experiments you will perform. Describe what you expect the experiments to reveal, or what is uncertain about the potential outcomes.
• Project Report (50% of project grade): We expect you to have completed your project by this time. The only thing we expect between this and the final poster presentation is incorporating feedback from the instructor. The report should be a 4-page self-contained document describing your project. Any student in the class should be able to clearly understand your report such that he/she can implement it. If your project has a new idea and promising results, this report could serve as a first draft for a future conference submission! Please use the CVPR latex template (http://www.pamitc.org/cvpr13/files/cvpr2013AuthorKit.zip) to write your report. Below is an outline of what we expect your report to cover:
• Abstract: One or two sentences on the motivation behind the problem you are solving. One or two sentences describing the approach you took. One or two sentences on the main result you obtained.
• Introduction: Motivation behind the problem you are solving, what applications it has, any brief background on the particular domain you are working in (if not regular RBG photographs), etc. If you are using a new way to solve an existing problem, briefly mention and describe the existing approaches and tell us how your approach is new.
• Approach: Describe very clearly and systematically your approach to solve the problem. Tell us exactly what existing implementations you used to build your system. Tell us what obstacles you faced and how you addressed them. Justify any design choices or judgment calls you made in your approach.
• Experiments and results: Provide details about the experimental set up (number of images/videos, number of datasets you experimented with, train/test split if you used machine learning algorithms, etc.). Describe the evaluation metrics you used to evaluate how well your approach is working. Include clear figures and tables, as well as illustrative qualitative examples if appropriate. Be sure to include obvious baselines to see if your approach is doing better than a naive approach (e.g. for classification accuracy, how well would a classifier do that made random decisions?). Also discuss any parameters of your algorithms, and tell us how you set the values of those parameters. You can also show us how the performance varies as you change those parameter values. Be sure to discuss any trends you see in your results, and explain why these trends make sense. Are the results as expected? Why?
• Conclusion and future work: Conclusion would likely make the same points as the abstract. Discuss any future ideas you have to make your approach better.
• References: List out all the references you have used for your work.
• Poster Presentation (40% of project grade): A poster presentation that describes the same points as the proposal and the project report. Sample poster template can be downloaded from here. Describe the results you have obtained. This may involve some additional experiments, etc based on the instructor's feedback on your report. Live demos when applicable are highly encouraged. Prepare a 1 minute schpeel that presents the main summary of your work. The instructor and TA will ask further questions as necessary.

Due Dates: All problem sets/reports are to be submitted by the due date noted on the assignment. Deadlines are firm. Anything from 1 minute to 24 hours is one day late

Late Day policy: Throughout the term you have an allowance of four free late days for your submissions, meaning you can accrue up to four days in late submissions with no penalty. For example, you could turn in one assignment four days late, or two assignments/project proposal/project report each one day late. Once you have used all your free late days, a late submission will not be accepted. Please plan ahead so you can spend your late days wisely. In particular, note that we expect you will find the earlier assignments easier than those later in the course. We will count a full additional day as having passed for submissions 1 minute to 24 hours late. No submissions will be accepted after all the late days are used up.

Exams (15% of final grade): There will be an in-class final exam.

Participation and attendance (5% of final grade): Participation in class and regular attendance is expected. If for whatever reason you are absent, it is your responsibility to find out what you missed that day. Note that attendance and participation do factor into the final grade.

Textbook: Computer Vision: Algorithms and Applications, by Rick Szeliski. An electronic copy is available free online here. Some background reading on object recognition is from Kristen Grauman and Bastian Leibe's short book on Visual Object Recognition.

Scholar: The scholar webpage for this course is here.

• All problem sets, proposals and project reports must be submitted in the Assignments section. No hard copies are accepted.
• Any questions relating to the course must be first raised in the Forums section, before reaching out to the Instructor/TA.
• Lectures and problem sets will be available both on scholar and on this webpage

Schedule (tentative):

References:

This course closely follows the following course:

• CS 376: Computer Vision at UT Austin taught by Kristen Grauman

(Just a few) Other similar courses (by no means an exhaustive list):

• CSE P 576: Computer Vision at University of Washington taught by Larry Zitnick
• CS 543 / ECE 549 Computer Vision at UIUC taught by Derek Hoiem (Video lectures)
• CS 143 Introduction to Computer Vision at Brown University taught by James Hays

Other code and data:

• Getting started with Matlab
• Visual Object Recognition synthesis lecture by Grauman and Leibe (short book on object recognition methods)
• Compiled list of recognition datasets
  
• OpenCV (open source computer vision library)
• Weka (Java data mining software)
• Netlab (Matlab toolbox for data analysis techniques, written by Ian Nabney and Christopher Bishop)
• CV Online
• Annotated Computer Vision Bibliography
• Oxford group interest point software
• Andrea Vedaldi's VLFeat code, including SIFT, MSER, hierarchical k-means.
• INRIA LEAR team's software, including interest points, shape features
• FLANN - Fast Library for Approximate Nearest Neighbors.  Marius Muja et al. 
• Google Goggles
• Kooaba
• LSH homepage
• Code for downloading Flickr images, by James Hays
• UW Community Photo Collections homepage
• INRIA Holiday images dataset
• NUS-WIDE tagged image dataset of 269K images
• MIRFlickr dataset
• LIBPMK feature extraction code, includes dense sampling
• LIBSVM library for support vector machines
• PASCAL VOC Visual Object Classes Challenge
  
• Fast SLIC superpixels
  
• Greg Mori's superpixel code
• Berkeley Segmentation Dataset and code
• Pedro Felzenszwalb's graph-based segmentation code
• Mean-shift: a Robust Approach Towards Feature Space Analysis [pdf]  [ code, Matlab interface by Shai Bagon]
• David Blei's Topic modeling code
• Berkeley 3D object dataset (kinect)
• Labelme Database 
• Stanford Event Dataset
  
• SUN Scene and object dataset
• ImageNet dataset of 15K objects and ImageNet challenge
• Animals with Attributes dataset
• aYahoo and aPascal attributes datasets
• Attribute discovery dataset of shopping categories
• Public Figures Face database with attributes
• Relative attributes data
• WhittleSearch relative attributes data
• SUN Scenes attribute dataset
• Cross-category object recognition (CORE) dataset
• Leeds Butterfly Dataset
• FaceTracer database from Columbia
• Caltech-UCSD Birds dataset
• Database of human attributes
  
• Face detection code in OpenCV
• Gallagher's Person Dataset
• Face data from Buffy episode, from Oxford Visual Geometry Group
• CALVIN upper-body detector code
• UMass Labeled Faces in the Wild
• Ivan Laptev's Space-Time Interest Points code
• Hollywood activity dataset
• Stanford 40 Actions still image dataset
• Stanford People Playing Musical Instrument dataset
  
• UCF activity datasets
• PASCAL VOC action recognition taster challenge
• TRECVID video retrieval challenge
• UMich Collective Activity dataset
• Egovision workshop at CVPR 2012
• Amazon Mechanical Turk
• Using Mechanical Turk with LabelMe
• Point Cloud Library
• Robot Operating System
• KITTI Benchmark
  

Tutorials, workshops, summer schools:

• Frontiers in Computer Vision Workshop
• Scene Understanding Symposium SUnS
• Como Workshop on Category-level Object Recognition(2008)
• IMA Visual Learning and Recognition Workshop(2006)
• MSRI Visual Recognition Workshop(2006)
• ENS/INRIA Visual Recognition and Machine Learning Summer School
• International Computer Vision Summer School
• Vision and Sports Summer School
• Computer vision conferences
• ICCV 2005 / CVPR 2007 / ICCV 2009 Short Course on Recognition
• AAAI 2008 Tutorial on Recognition
• ICML 2008 Tutorial on Recognition
• Greg Mori and Ivan Laptev's tutorial on action recognition at ECCV 2010
• CVPR 2009 Workshop on Visual Place Categorization