1 / 26

Immersive actor feedback system

Immersive actor feedback system. What sees the actor on set ?. ( Answer! ). Immersive actor feedback system. Viewpoint dependent projection Virtual reality: CAVE system Integration into Chroma-keying environment Retro-reflective cloth (BBC patent). Immersive actor feedback system.

katelynn-victory
Download Presentation

Immersive actor feedback system

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Immersive actor feedback system • What sees the actor on set ? ( Answer! )

  2. Immersive actor feedback system • Viewpoint dependent projection • Virtual reality: CAVE system • Integration into Chroma-keying environment • Retro-reflective cloth (BBC patent)

  3. Immersive actor feedback system • Retro-reflective cloth • “truematte” developed by BBC LED off Camera with LED ring LED on

  4. Immersive actor feedback system Mask generation Projector Head position 3D model of virtual scene Viewpoint Renderer 3D model of Actor black

  5. Immersive actor feedback system • View dependent projection The Ambassadors (1533), by Hans Holbein the Younger. National Gallery London

  6. Immersive actor feedback • Head tracking Fast template matching filter on voxel data

  7. real-time, low-quality 3D model of virtual scene 3D model of real scene Production Visualisation Tools Director’s View Planning, On-set Monitor Actor Feedback On-set

  8. Immersive actor feedback system

  9. Bamzooki • BBC Children Production

  10. Free-viewpoint video for sports • ‘Virtual Replay’ of football scenes • Allows viewer interactive review • Manually modelled offline • Not realistic

  11. Free-viewpoint video for Sport visualisation • Piero • Overcome limitations: use multi-camera system, iview project

  12. Applying multi-camera techniques to sport scenarios • Test shoot England-Wales in Old Trafford 9 Oct 2004

  13. Where to put the cameras? • Broadcast coverage cameras • Pro: • Works without extra setup • Cons: • Unreliable framing

  14. Calibration with wand Problems: Cameras knocked overnight Cameras not rigidly mounted Applying multi-camera techniques to sport scenarios

  15. Line-based calibration • Colour-based edge detection • Line-fitting • Calibration of camera parameters (T,R,f) against dimensions of the pitch • Runs at video frame rate

  16. Processing - Segmentation • Chroma-key against green

  17. Processing – 3D Reconstruction • 3D representations • Billboard (flat polygon, sprite) • Visual hull (Shape from silhouette)

  18. Results From 16 cameras, visual hull + graph-cut shape optimisation (results from iview project, jointly with University of Surrey)

  19. Results

  20. Summary • Computer vision techniques are important building blocks for the automated generation of visual effects • For the integration of video and graphics several optical phenomena have to be harmonised. In descending order of importance: • Camera perspective, occlusions, shadows, .. • Visual feedback systems integrated into the production pipeline are important to allow interaction with virtual objects • Free-viewpoint video allows new insights • Goal: Stream 3D to the viewer at home

  21. 3D Modelling • Graph-cut • Details: A Bayesian Framework for Simultaneous Matting and 3D Reconstruction, J.-Y. Guillemaut, A. Hilton, J. Starck, J. Kilner, and O. Grau, 3DIM’07

  22. Results • Video from ORIGAMI demo production

  23. Applications: Requirements Shape classes: IV 2-D 2 1/2-D III e c d I II II b 3-D coarse rich detailed a I III IV 1 2 3 4 Optical phenomenon: 1) camera perspective2) occlusions3) shadows4) reflections Applications: a) virtual studiob) On-set Visualisationc) Pre-Visualisationd) Interactive e) Digital effects

  24. expensive Introduction • Conventional work flow in (feature) TV or film productions Animations / Virtual Set Final Programme Story board Planning Phase On-set Phase Post Production

  25. Introduction • ORIGAMI approach • Mixed-reality techniques for visualisation through the production Animations / Virtual Set Story board Final Programme Planning Phase On-set Phase Post Production

More Related