Semantic Robot Vision Challenge: Current State and Future Directions

1. Semantic Robot Vision Challenge: Current State and Future Directions Scott Helmer, David Meger, Pooja Viswanathan, Sancho McCann, Matthew Dockrey, Pooyan Fazli, Tristram Southey, Marius Muja, Michael Joya, Jim Little, David Lowe, Alan Mackworth

2. What is the point of robotics research? • To do what humans cannot do:

3. What is the point of robotics research? • To do tasks that humans prefer not to do: From WALL-E

4. Current State in Home Robotics • Often split in a myriad of subtasks: navigation, recognition, scene understanding, manipulation, reasoning, etc. • Boundaries and interfaces often ignored and are problematic • Systems engineering is challenging ut generally not publishable • Integrated systems are rare: eg. Stanford’s STAIR, etc.

5. Embodied Vision • Actively “seeing” for some task • How images are acquired are not considered traditionally in computer vision, encouraging unrealistic assumptions • Eg. Benchmark datasets in object recognition • not representative of actual situations • learning algorithms rely on simplifications • hard to evaluate whether systems work outside lab

6. What is SRVC? • Photo scavenger hunt, where training data is acquired from internet

7. UBC’s Experience • Curious George (2007, 2008, …)

8. UBC and Collaboration • Integrated our lab • New research directions • Platform on which to test ideas • Provides quick way to introduce new students

9. Designing a winner … • Good design choices: • Eye level camera on PTU • Peripheral / foveal system with high res. camera • Good Algorithms: • SLAM navigation • Saliency and visual coverage • SIFT based recognition • Category recognition • After initial phase, can now focus more on research

10. What does the SRVC do well? • Compelling task • Visibility • AAAI 2007, Vancouver, Canada • CVPR 2008, Anchorage, USA • ISVC 2009, Las Vegas, USA • Responsive to entrants • Encourages open source • Evolves • Interesting for audience

11. Future Directions for SRVC • Attract more competitors • more synthesis • greater exposure • more exciting • Improve research outcomes • Research competitions should advance research rather than simply display current technology • Should reflect successful research, not engineering that doesn’t transfer

12. Attracting Competitors • Currently: • 2 leagues, Software league and Robot league • Software league is too similar to competitions like PASCAL VOC • Robot league poses challenges due to shipping, unknown environment, etc.

13. Software League • Offer more sensory modalities • Stereo vision, high res images, video • Offer mapping info, camera pose • Larger test sets for more statistical validity • Improve research outcomes (later)

14. Robot League • Provide more detailed specifications for contest environment • Provide standardized robot platform and architecture (like ROS) + Avoids per team risk of shipping/unknowns + Provides more opportunities for code sharing - Also involves numerous challenges • Focus on more interesting challenges like viewpoint planning

15. Improving Research Outcomes

16. Improving Research Outcomes • Improve realism – more clutter, occlusions, no white tablecloths etc. • Make context relevant • Allow access to pre-built datasets and priors • Web data is generally not suited for 3D recognition • Forefront of vision research requires richer datasets • Greater variety of objects and situations • Points for

17. Conclusion • Competitions can provide an evolving setting in which to evaluate current technologies • SRVC frames a challenging problem for embodied vision, which is difficult to evaluate using benchmarks • Numerous changes can be made to attract more competitors and improve research outcomes