1 / 27

Variable Viewpoint Reality

Experience sporting events from various viewpoints while accessing high-level commentary and statistics. Build a system that allows users to observe any viewpoint and analyze plays.

tdoe
Download Presentation

Variable Viewpoint Reality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Variable Viewpoint Reality Professor Paul Viola & Professor Eric Grimson Collaborators: Jeremy De Bonet, John Winn, Owen Ozier, Chris Stauffer, John Fisher, Kinh Tieu, Dan Snow, Tom Rikert, Lily Lee, Raquel Romano, Huizhen Yu, Mike Ross, Nick Matsakis, Jeff Norris, Todd Atkins Mark Pipes

  2. The BIG picture: User selected viewing of sporting events. • show me that play from the viewpoint of the goalie • … from the viewpoint of the ball • … from a viewpoint along the sideline • what offensive plays does Brazil run from this formation • how often has Italy had possession in the offensive zone

  3. The BIG picture: User selected viewing of sporting events. • Let me see my son’s motion from the following viewpoint • Let me see what has changed in his motion in the past year • Show me his swing now and a week ago • How often does he swing at pitches low and away • What is his normal sequence of pitches with men on base and less than 2 outs

  4. A wish list of capabilities • Construct a system that will allow each/every user to observe any viewpoint of a sporting event. • Provide high level commentary/statistics • analyze plays

  5. A wish list of capabilities • Recover human dynamics • Search databases for similar events

  6. VVR Spectator Environment • Build an exciting, fun, high-profile system • Sports: Soccer, Hockey, Tennis, Basketball, Baseball • Drama, Dance, Ballet • Leverage MIT technology in: • Vision/Video Analysis • Tracking, Calibration, Action Recognition • Image/Video Databases • Graphics • Build a system that provides data available nowhere else… • Record/Study Human movements and actions • Motion Capture / Motion Generation

  7. Window of Opportunity • 20-50 cameras in a stadium • Soon there will be many more • US HDTV is digital • Flexible, very high bandwidth digital transmissions • Future Televisions will be Computers • Plenty of extra computation available • 3D Graphics hardware will be integrated • Economics of sports • Dollar investments by broadcasters is huge (Billions) • Computation is getting cheaper

  8. For example … Computed using a single view… some steps by hand

  9. ViewCube: Reconstructing action & movement • Twelve cameras, computers, digitizers • Parallel software for real-time processing

  10. The View from ViewCube Multi-camera Movie

  11. Robust adaptive tracker Video Frames Adaptive Background Model Pixel Consistent with Background? X,Y,Size,Dx,Dy X,Y,Size,Dx,Dy X,Y,Size,Dx,Dy Local Tracking Histories

  12. Examples of tracking moving objects • Example of tracking results

  13. Dynamic calibration

  14. Multi-camera coordination

  15. Mapping patterns to groundplane

  16. Projecting Silhouettes to form 3D Models Real-time 3D Reconstruction is computed by intersecting silhouettes 3D Reconstruction Movie

  17. First 3D reconstructions ... 3D Movement Reconstruction Movie

  18. A more detailed reconstruction… Model

  19. Finding an articulate human body Segment Human Virtual Human 3D Model

  20. Automatically generated result: Body Tracking Movie

  21. Analyzing Human Motion • Key Difficulty: Complex Time Trajectories Complex Inter-dependencies • Our Approach: Multi-scale statistical models

  22. Detect Regularities & Anomalies in Events?

  23. Example track patterns • Running continuously for almost 3 years • during snow, wind, rain, dark of night, … • have processed 1 Billion images • one can observe patterns over space and over time • have a machine learning method that detects patterns automatically

  24. Automatic activity classification

  25. Example categories of patterns • Video of sorted activities

  26. Histogram of activity over a single day 12am 6am 12pm 6pm 12pm people (1993 total with .1% FP) Resulting classifier 12am 6am 12pm 6pm 12pm groups of people (712 total with 2.2% FP) 12am 6am 12pm 6pm 12pm clutter/lighting effects (647 total with 10.5% FP) 12am 6am 12pm 6pm 12pm cars (1564 total with 3.4% FP) Analyzing event sequences

  27. …and this works for other problems • Sporting events • Eldercare monitoring • Disease progression tracking • Parkinson’s • … anything else that involves capturing, archiving, recognizing and reconstructing events!

More Related