ECE 5984: Advanced Topics in Computer Vision, Spring 2014

Electrical and Computer Engineering Department, Virginia Tech

Meets: TR 3:30 pm to 4:45 pm in Hutcheson Hall (HUTCH) 207 McBryde Hall (MCB) 233.

Instructor: Devi Parikh
Email: parikh@vt.edu
Office: 440 Whittemore Hall
Office hours: By appointment.

Course overview Pre-requisite Requirements Important dates Schedule Resources

Prize Winners!

Best Paper Presentation: Jason Ziglar and John Peterson

Best Discussion Participant: Ramakrishna Vedantam

Best Project: Michael Cogswell

Congratulations!

Class projects:

Dual Channel Analytics and Tracking of Cells Experiencing the Dielectrophoretic Force (Lisa Anders)

Semantic Segmentation with Deep Learning (Michael Cosgswell)

Building High-Level Object Vocabularies (Jacob Dennis)

Application of Selective Search to Pose Estimation (Ujwal Krothapalli)

Making Intelligent and Interpretable Classification Systems (Shrenik Lad)

Towards Cascade Object Detection (John Peterson)

Person of Interest in Images (Clint Solomon)

Understanding Predictions of Structured Probabilistic Vision Systems (Qing Sun)

Improving Image Segmentation using Object Proposals (Sean Thweatt)

Understanding and Predicting Importance for Abstract Images (Ramakrishna Vedantam)

Improving SIFT Matching by Interest Points Filtering (Rabih Younes)

Salient Superpixels (Jason Ziglar)

Improving DPM accuracy with Component Selection Strategy (Peng Zhang)

Course overview:

This is a graduate course in computer vision. The focus of this course is to survey and critique current and state-of-the-art approaches in computer vision. We will read and analyze the strengths and weaknesses of research papers on a variety of important topics pertaining to visual recognition and identify open research questions. See the schedule for a list of topics we will cover.

Pre-requisite:

An introduction to computer vision or equivalent course.  A machine learning or pattern recognition course may be beneficial.

Requirements:

Following are the requirements to successfully complete this course:

Discussion Paper reviews Presentations Project

• Discussion (15% of your grade): Students will be required to read the assigned papers before each class and actively participate in discussions in class.

• Paper reviews (25% of your grade): Students will be required to write a detailed review of one assigned paper and a high-level review of another assigned paper before each class. The combined reviews should be not more than one page (11 point, times new roman, 1 inch margins). Please submit the review as firstname_lastname_MM_DD.pdf (all in small letters, where MM is the month and DD is the day).
• Detailed review: Each review should summarize the paper in 2-3 sentences, describe the approach taken, and clearly identify the main contribution of the paper. The review should describe the strengths and weaknesses of the paper. Comments on how convincing the experiments were, if the material was well presented, and how the paper can be improved and extended should be included. A good review also comments on how the paper relates to other papers we have read or you know of. Finally, identify any interesting open research questions or applications that arise from reading the paper. 
• High-level review: Describe the problem being addressed and provide a high-level description of the general approach or intuition behind the approach. Details of the approach, discussion of strengths and weaknesses, etc. are not required. 
  
• Due dates: The reviews should be emailed to the instructor by 12:00 pm (noon) the day of the class (i.e. on Tuesdays and Thursdays). You can use upto 3 late days over the course of the semester. Beyond that, your submission will not be accepted. You need not submit paper reviews during the classes where you are presenting the papers (see below). 

• Presentations (25% of your grade): Each student will be asked to present the topic associated with a class 2 or 3 times over the course of the semester (number will depend on class enrollment). Each presentation should be 45 minutes long. Students should practice their talks ahead of time to make sure they are 45 minutes long -- not shorter by more than a few minutes, and certainly not longer. The talks should be well organized and polished. Following will be required to prepare the presentation.

• Papers (65% of the presentation grade): The student should read the assigned papers and other background papers to gain a good perspective on the topic as a whole. In addition to presenting this perspective and background information, students should present at least two papers in detail. For each paper, students should clearly state the problem statement, and motivate why the problem is interesting and important. The key technical ideas should be presented. The student should describe the experimental set-up and present the results obtained. The strengths and weaknesses of the paper should be discussed. The student should discuss how the different papers relate to each other (similarities and differences). Finally, interesting open research questions should be identified. The slides should be made as visual (with videos, images, animations) and clear as possible. The student should look at the links provided next to each paper, as well as the authors webpages for extra material such as slides, videos, extra results, etc. Students are encouraged to search for relevant material online, the links provided here are not comprehensive. Please clearly cite the source of each slide that is not your own. Even if you use slides made by the author, it is your responsibility to make sure your presentation as a whole flows well.

• Experiments (35% of the presentation grade): The student should also conduct some (small-scale) experiments on at least one of the papers (any of those listed or other relevant papers) to analyze an interesting and meaningful aspect of the approach that the paper has not analyzed (e.g. different datasets, sensitivity to any parameters in the approach, etc.) to gain a more complete understanding of the paper, and to see if the approach "really works". A distilled demo-version of the main idea of the approach presented in the paper should be implemented. The goal is not to regenerate the results already present in the paper. You may implement it yourself, or download code if available. Again, the experimental setup, any non-trivial implementation choices you made, results obtained and conclusions one can draw from them should be described in the presentation (within the 45 minutes). Please cite any existing code or data you use for your experiments.

• Announcement: please email the instructor 6 topics preferences (along with the associated date) for your presentations by the end of the day Wednesday, January 22nd. See the schedule for the semester.

• Project (35% of your grade): Your project can be about extending a technique we studied in class, or empirically analyzing it. Comparisons between two approaches are also welcome. It is wonderful if you design and evaluate a novel approach to an important vision problem. Look at the schedule to get ideas on what topics might be of interest to you for your project. Perhaps the experiments part of your presentation above might give you project ideas. Be creative! If you need help with ideas for your project please come talk to me. You can work with a partner if you like. The following are deliverables for your project.

• Proposal (10% of your project grade, due March 6th): A 1-page description that describes the following:
• Problem statement: Clearly state the goal of your project.
• Related work: Briefly describe existing related work (with citations) and what your project brings to the table that these other works do not. The most relevant papers may not necessarily be papers listed on the schedule, so be sure to also look beyond the list.
• Approach: Describe the technical approach you plan to employ
• Experiments and results: Describe the experimental setup you will follow, which datasets you will use, which existing code you will exploit, what you will implement yourself, and what you would define as a success for the project. If you plan on collecting your own data, describe what data collection protocol you will follow. Specify if you plan on experimentally analyzing different characteristics of your approach, or if you will compare to existing techniques. Provide a list of experiments you will perform. Describe what you expect the experiments to reveal, or what is uncertain about the potential outcomes. If you have any preliminary results, please summarize those as well.

• Mid-semester presentation (20% of your project grade, in class on March 27th and April 1st): A 10-minute presentation that describes the same points as the proposal in as much detail as 10 minutes allow. Describe any challenges you are facing. Clearly state what has already been accomplished, and what remains for the end of the semester. 

• Final presentation (35% of your project grade, in class between April 24th and May 6th): A 10-minute presentation describing the same points as above. Any assumptions of your approach should be clearly stated. Any insights on future extensions of this project should be discussed.

• Final report (35% of your project grade, due May 12th): A 4-page self-contained document describing your project. It should follow the same format and quality of presentation as conference papers you have been reading in this class. Thoroughly describe related work and explain what your work brings to the table that existing work did not. Any student in the class should be able to clearly understand your report such that he/she can implement it. The reader should not have to read another paper to understand your approach. Justify any design choices or judgment calls you made in your approach. When describing the experiments, describe why you conducted each experiment, and what you can conclude from the results. Include clear figures and tables, as well as illustrative qualitative examples if appropriate. If your project has a new idea and promising results, this report could serve as a first draft for a future conference submission! Please use the CVPR latex template (http://www.pamitc.org/cvpr13/files/cvpr2013AuthorKit.zip) to write your report.

• The class will vote on best paper presentation, best project and best discussion-participant! 

• Feedback is very welcome. If you have any questions or concerns about the class or the requirements, please be sure to discuss them with the instructor early on. Please email the instructor to set up an appointment.

• No laptops, cell phone or other distractions in class please.

• January 22nd: Email the instructor 6 topic preferences (along with the associated dates) you would like to present in class. Please see the schedule for the semester.
• March 6th: Project proposals due
• March 27th and April 1st: Mid-semester project presentations in class
• April 24th-May 6th: Final project presentations in class May 10th: Final project presentations from 10:00 am to 1:30 pm
• May 12th: Final project report due

Schedule (topic and papers)

Resources

Other code and data:

• Visual Object Recognition synthesis lecture by Grauman and Leibe (short book on object recognition methods)
• Compiled list of recognition datasets
  
• OpenCV (open source computer vision library)
• Weka (Java data mining software)
• Netlab (Matlab toolbox for data analysis techniques, written by Ian Nabney and Christopher Bishop)
• CV Online
• Annotated Computer Vision Bibliography
• Oxford group interest point software
• Andrea Vedaldi's VLFeat code, including SIFT, MSER, hierarchical k-means.
• INRIA LEAR team's software, including interest points, shape features
• FLANN - Fast Library for Approximate Nearest Neighbors.  Marius Muja et al. 
• Google Goggles
• Kooaba
• LSH homepage
• Code for downloading Flickr images, by James Hays
• UW Community Photo Collections homepage
• INRIA Holiday images dataset
• NUS-WIDE tagged image dataset of 269K images
• MIRFlickr dataset
• LIBPMK feature extraction code, includes dense sampling
• LIBSVM library for support vector machines
• PASCAL VOC Visual Object Classes Challenge
  
• Fast SLIC superpixels
  
• Greg Mori's superpixel code
• Berkeley Segmentation Dataset and code
• Pedro Felzenszwalb's graph-based segmentation code
• Mean-shift: a Robust Approach Towards Feature Space Analysis [pdf]  [code, Matlab interface by Shai Bagon]
• David Blei's Topic modeling code
• Berkeley 3D object dataset (kinect)
• Labelme Database 
• Stanford Event Dataset
  
• SUN Scene and object dataset
• ImageNet dataset of 15K objects and ImageNet challenge
• Animals with Attributes dataset
• aYahoo and aPascal attributes datasets
• Attribute discovery dataset of shopping categories
• Public Figures Face database with attributes
• Relative attributes data
• WhittleSearch relative attributes data
• SUN Scenes attribute dataset
• Cross-category object recognition (CORE) dataset
• Leeds Butterfly Dataset
• FaceTracer database from Columbia
• Caltech-UCSD Birds dataset
• Database of human attributes
  
• Face detection code in OpenCV
• Gallagher's Person Dataset
• Face data from Buffy episode, from Oxford Visual Geometry Group
• CALVIN upper-body detector code
• UMass Labeled Faces in the Wild
• Ivan Laptev's Space-Time Interest Points code
• Hollywood activity dataset
• Stanford 40 Actions still image dataset
• Stanford People Playing Musical Instrument dataset
  
• UCF activity datasets
• PASCAL VOC action recognition taster challenge
• TRECVID video retrieval challenge
• UMich Collective Activity dataset
• Egovision workshop at CVPR 2012
• Amazon Mechanical Turk
• Using Mechanical Turk with LabelMe
• Point Cloud Library
• Robot Operating System
• KITTI Benchmark
  

• Frontiers in Computer Vision Workshop
• Scene Understanding Symposium SUnS
• Como Workshop on Category-level Object Recognition (2008)
• IMA Visual Learning and Recognition Workshop (2006)
• MSRI Visual Recognition Workshop (2006)
• ENS/INRIA Visual Recognition and Machine Learning Summer School
• International Computer Vision Summer School
• Vision and Sports Summer School
• Computer vision conferences
• ICCV 2005 / CVPR 2007 / ICCV 2009 Short Course on Recognition
• AAAI 2008 Tutorial on Recognition
• ICML 2008 Tutorial on Recognition
• Greg Mori and Ivan Laptev's tutorial on action recognition at ECCV 2010
• CVPR 2009 Workshop on Visual Place Categorization

Similar courses:

This course has been inspired by the following two courses:

• Visual Recognition (Kristen Grauman, Texas-Austin, Fall 2012)
• Learning-Based Methods in Vision (Alyosha Efros, CMU, Spring 2012) 

• Grounding Object Recognition and Scene Understanding (Antonio Torralba, MIT, Fall 2011)
• Visual Scene Understanding (Derek Hoiem, UIUC, Spring 2009)
• Statistical Models for Visual Recognition (Deva Ramanan, UCI, Winter 2009)
• Object Recognition and Scene Understanding (Antonio Torralba, MIT, Fall 2008)
• Scene Understanding Seminar (Aude Oliva, MIT, Fall 2006)
• Selected Topics in Vision & Learning (Serge Belongie, UCSD, Spring 2011)
• Learning and Inference in Vision (Bill Freeman, MIT)
• Cutting Edge of Computer Vision (Fei-Fei Li, Stanford)
• Recognizing People, Objects, and Scenes (Jitendra Malik, Berkeley)
• Recognition Problems in Computer Vision (Greg Mori, SFU)
• Vision and Learning (Jianbo Shi, UPenn)