110 likes | 191 Views
Challenges to image parsing researchers. Lana Lazebnik UNC Chapel Hill. sky. mountain. building. person. car. car. sidewalk. road. The past: “closed universe ” datasets Tens of classes, hundreds of images, offline learning. Figure from Shotton et al. (2009).
E N D
Challenges to image parsing researchers Lana Lazebnik UNC Chapel Hill sky mountain building person car car sidewalk road
The past: “closed universe” datasetsTens of classes, hundreds of images, offline learning Figure from Shotton et al. (2009) He et al. (2004), Hoiem et al. (2005), Shotton et al. (2006, 2008, 2009), Verbeek and Triggs (2007), Rabinovich et al. (2007), Galleguillos et al. (2008), Gould et al. (2009), etc.
The future: “open universe” datasets Evolving images, annotations http://labelme.csail.mit.edu/
The future: “open universe” datasets Non-uniform class frequencies
Which “closed universe” techniques can survive in the “open universe” setting? • Combination of local cues? • Multiple segmentations/grouping hypotheses? • Context? • Graphical models (MRFs, CRFs, etc.)? • Offline learning and inference?
Learning from all of LabelMe50K images, 232 labels sky window building tree car door road road sun sky sea sky sky building ceiling car car building wall road sidewalk mountain floor Tighe & Lazebnik, work in progress
Learning from all of LabelMe50K images, 232 labels Per-class classification rates Tighe & Lazebnik, work in progress
Challenge: Dynamic image interpretation • Image parsing algorithms should become autonomous decision-making agents • Visual “detective task”: Where was this photo taken?
Challenge: Dynamic image interpretation • Image parsing algorithms should become autonomous decision-making agents Input
Summary • Challenges to image parsing researchers: • Learn to parse images from “open universe” evolving datasets • Try parsing gigapixel images! • Develop active, sequential image interpretation strategies