ECE 6504: Advanced Topics in Computer Vision, Spring 2013

Electrical and Computer Engineering Department
, Virginia Tech

Meets: TR 5:00 pm to 6:15 pm in McBryde Hall (MCB) 216

Instructor: Devi Parikh
Office: 440 Whittemore Hall
Office hours: by appointment (via email)

Announcements Course overview Pre-requisite Requirements Important dates Schedule Resources


*** Best Presentation Prize: Xiao Lin

*** Best Project Prize: Neelima Chavali

*** Best Discussion Participant Prize: Stanislaw Antol

Two paper reviews are due by 12:00 pm (noon) the day of the class.

The schedule of presenters is available. Please remember to meet the instructor 3 days before your presentation for a dry-run with a complete set of slides. Email the instructor to set up an appointment a couple of days ahead of time.

By Thursday January 24th: Email 6 preferences (dates and topics) for when you would like to present in class. See the schedule for the semester.

Welcome to 6504!

Course overview:

This is a graduate course in computer vision. The focus of this course is to survey and critique current and state-of-the-art approaches in computer vision. We will read and analyze the strengths and weaknesses of research papers on a variety of important topics pertaining to visual recognition and identify open research questions. See the schedule for a list of topics we will cover.


An introduction to computer vision or equivalent course.  A machine learning or pattern recognition course may be beneficial.


Following are the requirements to successfully complete this course:

Discussion Paper reviews Presentations Project

Important dates:

Schedule (topic and papers)

Date Topic and papers Presenter
01/22 Introduction Devi
01/24 Research overview Devi
01/29 Local features-based image descriptions

A Performance Evaluation of Local Descriptors. K. Mikolajczyk and C. Schmid.  CVPR 2003.

Object Categorization by Learned Universal Visual Dictionary. J. Winn, A. Criminisi and T. Minka. ICCV 2005. [project page]

Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. S. Lazebnik, C. Schmid and J. Ponce. CVPR 2006. [15 scenes dataset]  [pyramid match tooklit] [Matlab code]


Seminal paper: Object Recognition from Local Scale-Invariant Features. D. Lowe. ICCV 1999.  [code] [other implementations of SIFT] [IJCV paper]

01/31 Object discovery

Using Multiple Segmentations to Discover Objects and their Extent in Image Collections. B. C. Russell, A. A. Efros, J. Sivic, W. T. Freeman and A. Zisserman.  CVPR 2006. [code]

Foreground Focus: Finding Meaningful Features in Unlabeled Images. Y. J. Lee and K. Grauman. BMV 2008. [project page]

02/05 Object detection

Rapid Object Detection Using a Boosted Cascade of Simple Features. P. Viola and M. Jones. CVPR 2001. [code]

Histograms of Oriented Gradients for Human Detection. N. Dalal and B. Triggs. CVPR 2005. [video] [code] [PASCAL datasets]


A very popular object detector: A Discriminatively Trained, Multiscale, Deformable Part Model. P. Felzenszwalb,  D.  McAllester and D. Ramanan. CVPR 2008. [code]

Diagnosing Error in Object Detectors. D. Hoiem, Y. Chodpathumwan and Q. Dai. ECCV 2012. [code and data]
02/07 Object proposals

What is an Object?  B. Alexe, T. Deselaers and V. Ferrari.  CVPR 2010. [code]

Category Independent Object Proposals. I. Endres and D. Hoiem. ECCV 2010. [project]

Neelima (papers)
Yao (experiments)
02/12 Segmentation

Learning a Classification Model for Segmentation. X. Ren and J. Malik. ICCV 2003.

Constrained Parametric Min-Cuts for Automatic Object Segmentation. J. Carreira and C. Sminchisescu. CVPR 2010. [code]

Combining Top-down and Bottom-up Segmentation. E. Borenstein, E. Sharon and S. Ullman.  CVPR  workshop 2004. [data]
02/14 Project proposals due

Images of people

Names and Faces in the News. T. Berg, A. Berg, J. Edwards, M. Maire, R. White, Y. Teh, E. Learned-Miller and D. Forsyth. CVPR 2004. [project page]

Estimating Age, Gender and Identity using First Name Priors. A. Gallagher and T. Chen. CVPR 2008. [project page]


Exploring Photobios.  I. Kemelmacher-Shlizerman, E. Shechtman, R. Garg and S. Seitz. SIGGRAPH 2011. [project page]

Autotagging Facebook: Social Network Context Improves Photo Annotation. Z. Stone, T. Zickler and T. Darrell.  CVPR Internet Vision Workshop 2008.

02/19 Pose

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations. L. Bourdev and J. Malik.  ICCV 2009. [code]

Real-Time Human Pose Recognition in Parts from a Single Depth Image.  J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman and A. Blake. CVPR 2011. [video] [project page]
02/21 Context
An Empirical Study of Context in Object Detection. S. Divvala, D. Hoiem, J. Hays, A. Efros and M. Hebert. CVPR 2009. [project page]

Object-Graphs for Context-Aware Category Discovery.  Y. J. Lee and K. Grauman.  CVPR 2010. [code]
02/26 Holistic scene understanding

TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-Class Object Recognition and Segmentation.  J. Shotton, J. Winn, C. Rother and A. Criminisi.  ECCV 2006. [project page] [data] [code]

Describing the Scene as a Whole: Joint Object Detection, Scene Classification and Semantic Segmentation. J. Yao, S. Fidler and R. Urtasun. CVPR 2012. 
02/28 Groups of objects

Recognition Using Visual Phrases.  M. Sadeghi and A. Farhadi.  CVPR 2011.

Automatic Discovery of Groups of Objects for Scene Understanding. C. Li, D. Parikh and T. Chen. CVPR 2012. [project page]
03/05 Saliency

A Model of Saliency-based Visual Attention for Rapid Scene Analysis.  L. Itti, C. Koch, and E. Niebur. PAMI 1998.

Learning to Detect a Salient Object.  T. Liu, J. Sun, N. Zheng, X. Tang, H. Shum. CVPR 2007. [results] [data] [code]

Learning to Predict Where Humans Look. T. Judd, K. Ehinger, F. Durand, and A. Torralba. ICCV 2009. [project page]
03/07 Importance

Reading Between the Lines: Object Localization Using Implicit Cues from Image Tags. S. J. Hwang and K. Grauman. CVPR 2010.

Understanding and Predicting Importance in Images.  A. Berg, T. Berg, H. Daume, J. Dodge, A. Goyal, X. Han, A, Mensch, M. Mitchell, A. Sood, K. Stratos and K. Yamaguchi. CVPR 2012. [UIUC sentence dataset] [ImageClef dataset]

Some Objects are More Equal Than Others: Measuring and Predicting Importance. M. Spain and P. Perona. ECCV 2008.
What Makes an Image Memorable?  P. Isola, J. Xiao, A. Torralba, A. Oliva. CVPR 2011. [project page] [code and data]
[presentation material]
03/12 Spring break: no class
03/14 Spring break: no class
03/19 Mid-semester project presentations Xiao
Ahmed and Sherin Abhijit
03/21 Mid-semester project presentations Yao
03/26 Action Recognition

Action Recognition from a Distributed Representation of Pose and Appearance. S. Maji, L. Bourdev and J.  Malik. CVPR 2011. [code]

Learning a Hierarchy of Discriminative Space-Time Neighborhood Features for Human Action Recognition. A. Kovashka and K. Grauman. CVPR 2010.
03/28 Global and high-level image descriptions

Modeling the Shape of the Scene: a Holistic Representation of the Spatial Envelope. A. Oliva and A. Torralba. IJCV 2001. [Gist code
Efficient Object Category Recognition Using Classemes. L. Torresani, M. Szummer and A. Fitzgibbon. ECCV 2010. [code and data]
Objects as Attributes for Scene Classification. L.-J. Li, H. Su, Y. Lim and L. Fei-Fei, 1st International Workshop on Parts and Attributes, ECCV 2010.
Follow up: Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification.  L-J. Li, H. Su, E. Xing, L. Fei-Fei.  NIPS 2010. [code]
04/02 Attributes

Learning To Detect Unseen Object Classes by Between-Class Attribute Transfer. C. Lampert, H. Nickisch and S. Harmeling. CVPR 2009. [project page with data]

Describing Objects by Their Attributes. A. Farhadi, I. Endres, D. Hoiem and D. Forsyth, CVPR 2009. [data]

Relative Attributes.  D. Parikh and K. Grauman. ICCV 2011. [code and data]
04/04 Classes Canceled (Snow)

04/09 Human-in-the-loop

Visual Recognition with Humans in the Loop.  S. Branson, C. Wah, B. Babenko, F. Schroff, P. Welinder, P. Perona and S. Belongie. ECCV 2010.

Large-Scale Live Active Learning: Training Object Detectors with Crawled Data and Crowds. S. Vijayanarasimhan and K. Grauman.  CVPR 2011.

iCoseg: Interactive Co-segmentation with Intelligent Scribble Guidance. D. Batra, A. Kowdle, D. Parikh, J. Luo and T. Chen. CVPR 2010. [project page]
04/11 Crowdsourcing

Adaptively Learning the Crowd Kernel.  O. Tamuz, C. Liu, S. Belongie, O. Shamir and A. Kalai. ICML 2011.

Crowdclustering. R. Gomes, P. Welinder, A. Krause and P. Perona. NIPS 2011.

Labeling Images with a Computer Game. L. von Ahn and L. Dabbish. CHI 2004. [game]
04/16 Big data

Unbiased Look at Dataset Bias. A. Torralba and A. Efros. CVPR 2011. [project page]

IM2GPS: Estimating Geographic Information From a Single Image. J. Hays and A. Efros. CVPR 2008. [project page with data and Flickr download scripts]

Scene Completion using Millions of Photographs. J. Hays and A. Efros. SIGGRAPH 2007. [project page]
04/18 Applications

LeafSnap: A Computer Vision System for Automatic Plant Species Identification.  N. Kumar, P. Belhumeur, A. Biswas, D. Jacobs, W. Kress, I. Lopez, J. Soares. ECCV 2012.

Photo Tourism: Exploring Photo Collections in 3D. N. Snavely, S. Seitz and R. Szeliski. SIGGRAPH 2006. [project page]

FaceTracer: A Search Engine for Large Collections of Images with Faces.  N. Kumar, P. Belhumeur and S. Nayar.  ECCV 2008. [code, data, demo]
Sherin (papers)
Xiao (experiments)
04/23 Human abilities

80 Million Tiny Images: A Large Dataset for Non-Parametric Object and Scene Recognition. A. Torralba, R. Fergus and W. Freeman.  PAMI 2008. [project page]
What Do We Perceive in a Glance of a Real-World Scene? L. Fei-Fei, A. Iyer, C. Koch and P. Perona. Journal of Vision, 2007.
04/25 Language and images

Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers. A. Gupta and Larry S. Davis. ECCV 2008.

Every Picture Tells a Story: Generating Sentences for Images. A. Farhadi, M. Hejrati, A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier and D. Forsyth. ECCV 2010. [UIUC sentence dataset]

Baby Talk: Understanding and Generating Simple Image Descriptions. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. C. Berg and T. L. Berg. CVPR 2012.
Stan (papers)
Abhijit (experiments)
04/30 Final project presentations Ahmed and Sherin
05/02 Final project presentations Yao
05/07 Final project presentations Mahmoud
05/15 Final project reports due


Other code and data:

Tutorials, workshops, summer schools:

Similar courses:

This course has been inspired by the following two courses:

Other similar courses: