1 / 41

Pixmotor: A Pixel Motion Integrator

Pixmotor: A Pixel Motion Integrator. Ivan Neulander Rhythm & Hues. What is Pixmotor ?. A 2D motion blur solution at Rhythm & Hues Makes motion-blurred images based on static images generated by 3D renderer. Garfield: A Tail of Two Kitties. Night At The Museum. What is Pixmotor?.

kasen
Download Presentation

Pixmotor: A Pixel Motion Integrator

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pixmotor:A Pixel Motion Integrator Ivan Neulander Rhythm & Hues

  2. What is Pixmotor? A 2D motion blur solution at Rhythm & Hues • Makes motion-blurred images based on static images generated by 3D renderer Garfield: A Tail of Two Kitties Night At The Museum

  3. What is Pixmotor? A 2D motion blur solution at Rhythm & Hues • Makes motion-blurred images based on static images generated by 3D renderer • Not a physically correct but plausible • Incorporates heuristics to minimize artifacts Garfield: A Tail of Two Kitties Night At The Museum

  4. What is Pixmotor? A 2D motion blur solution at Rhythm & Hues • Makes motion-blurred images based on static images generated by 3D renderer • Not a physically correct but plausible • Incorporates heuristics to minimize artifacts • Fits well into our production pipeline • Render objects separately, combine in comp • Zero-centered shutter minimizesinterpolation distance from static frame Garfield: A Tail of Two Kitties Night At The Museum

  5. What is Pixmotor? A 2D motion blur solution at Rhythm & Hues • Makes motion-blurred images based on static images generated by 3D renderer • Not a physically correct but plausible • Incorporates heuristics to minimize artifacts • Fits well into our production pipeline • Render objects separately, combine in comp • Zero-centered shutter minimizesinterpolation distance from static frame Garfield: A Tail of Two Kitties Night At The Museum

  6. Motivation • Processing Speed • Invariant of geometric complexity: Typical 2k element completes in under 1 minute • Many speed/quality controls for interactive testing

  7. Motivation • Processing Speed • Invariant of geometric complexity: Typical 2k element completes in under 1 minute • Many speed/quality controls for interactive testing • Art Direction • Tweak shutter per-frame without rerendering • Vary motion blur within a frame shutter/2 shutter shutter x 2 shutter x 3

  8. Motivation Diffuse Reflection Shader • In-Compositor Shading • For static frames: • Renderer outputs images ofnormals, positions, etc.;compositor does shading Normals Output

  9. Motivation Diffuse Reflection Shader • In-Compositor Shading • For static frames: • Renderer outputs images ofnormals, positions, etc.;compositor does shading • For motion-blurred frames: • Motion blurring data imagesin renderer is incorrect Normals Output Ground truth

  10. Motivation Diffuse Reflection Shader • In-Compositor Shading • For static frames: • Renderer outputs images ofnormals, positions, etc.;compositor does shading • For motion-blurred frames: • Motion blurring data imagesin renderer is incorrect • Using pixmotor allows doingshading on static images Normals Output Pixmotor Ground truth

  11. Pixmotor Data Flow Color Input Motion Input Pixmotor Settings Depth Input

  12. Computing Motion Vectors • When shading a triangle at barycentric point (,,):Compute NDC motion vector asPNDC(,,){ShutterClose} - PNDC(,,){ShutterOpen} Shutter Open Shading Moment Shutter Close

  13. Implementation:Overview • Discretize motion into a set of slices • Each slice = particular moment within shutter span • Total number of slices depends on maximum motion vector • User sets maximum pixel separation between slices • Compute each slice and add it to accumulation buffer

  14. Implementation:Overview • Discretize motion into a set of slices • Each slice = particular moment within shutter span • Total number of slices depends on maximum motion vector • User sets maximum pixel separation between slices • Compute each slice and add it to accumulation buffer 1 slice 3 slices 7 slices 52 slices

  15. Implementation:Main Loop • For each slice: • Copy each pixel from source image to a work buffer, shifting its coordinates based on motion vector

  16. Implementation:Main Loop • For each slice: • Copy each pixel from source image to a work buffer, shifting its coordinates based on motion vector • Resolve occlusion for overlappingpixels using depth info

  17. Implementation:Main Loop • For each slice: • Copy each pixel from source image to a work buffer, shifting its coordinates based on motion vector • Resolve occlusion for overlappingpixels using depth info • Holes form due to divergent motion Use hole-filling heuristics

  18. Implementation:Main Loop • For each slice: • Copy each pixel from source image to a work buffer, shifting its coordinates based on motion vector • Resolve occlusion for overlappingpixels using depth info • Holes form due to divergent motion Use hole-filling heuristics Pixmotor Single-slice interpolation Single-res WBno hole fill Double-res WB,no hole fill Double-res WB,with hole fill Static Image

  19. Implementation:Hole-Filling Heuristics • Before Main Loop: • Blur motion image (optionally) • Filter kernel must ignore empty pixels

  20. Implementation:Hole-Filling Heuristics • Before Main Loop: • Blur motion image (optionally) • Filter kernel must ignore empty pixels • Compute motion image gradient • Force high gradient at image edges

  21. Implementation:Hole-Filling Heuristics • Before Main Loop: • Blur motion image (optionally) • Filter kernel must ignore empty pixels • Compute motion image gradient • Force high gradient at image edges Motion Gradient Result Blur = 0 Blur = 3 pixels Blur = 10 pixels Blur = 60 pixels

  22. Implementation:Hole-Filling Heuristics • During Main Loop: • Identify holes as empty pixels near high-gradient regions • Fill holes with nearby solid neighbors along motion path

  23. Implementation:Hole-Filling Heuristics • During Main Loop: • Identify holes as empty pixels near high-gradient regions • Fill holes with nearby solid neighbors along motion path • Solid pixels at image edges are always presumed holes

  24. Implementation:Hole-Filling Heuristics • During Main Loop: • Identify holes as empty pixels near high-gradient regions • Fill holes with nearby solid neighbors along motion path • Solid pixels at image edges are always presumed holes StaticImage Hole Fill: off [demo] MotionGradient Hole Fill: on

  25. Implementation:Hole-Filling Heuristics • After Main Loop: • Option to fix matte for remaining holes:  Unpremultiply partial pixels with solid neighbors(distance along motion vector times user constant) • Helps create a solid matte where needed for compositingBut: can harden some soft edges

  26. Implementation:Hole-Filling Heuristics • After Main Loop: • Option to fix matte for remaining holes:  Unpremultiply partial pixels with solid neighbors(distance along motion vector times user constant) • Helps create a solid matte where needed for compositingBut: can harden some soft edges FixMatte: 0.0 FixMatte: 1.0

  27. Results • No overlap or transparency: excellent 3D Motion Blur Pixmotor

  28. Results • No overlap or transparency: excellent • With overlap or transparency: not so good 3D Motion Blur Pixmotor

  29. Results: Linear 3D MB (easy case)

  30. Results:Pixmotor (easy case)

  31. Results:Linear 3D MB (hard case)

  32. Results:Pixmotor (hard case)

  33. Motion-Blurred Shadows:Our Approach • Need to store partial occlusion in depth map use matte channel Matte Depth

  34. Motion-Blurred Shadows:Our Approach • Need to store partial occlusion in depth map use matte channel • Process depth, matte channels using Pixmotor Motion Pixmotor Matte Depth

  35. Motion-Blurred Shadows:Our Approach • Need to store partial occlusion in depth map use matte channel • Process depth, matte channels using Pixmotor • Use unpremultiplied values for depth • Depth map camera settings are unmodified Motion Pixmotor Matte Depth Matte Depth

  36. Motion-Blurred Shadows:Results

  37. Motion-Blurred Shadows:Results

  38. Frustrations & Future Work • Transparency, occlusion artifacts  use more fragments per pixelBut: speed will suffer • Nonlinear motion (e.g. propeller)  Use acceleration + velocity for motion • Memory-hungry on deep inputs  Process channels in batchesBut: counteracts multithreading • Apply to reflections, refractions • Compute motion vectors for these

  39. Demo Video • Real-time capture of interactive Icy session on aquad-CPU Opteron 2200demo

  40. Thank You • Jonathan Robbins • Martin Ryan www.rhythm.com/~ivan/pdfs/sketch2007.pdfwww.rhythm.com/~ivan/pub/sketch2007.ppt

More Related