Electronic Projection Mapping using Warp & Blend

Electronic Projection Mapping using Warp & Blend Introduction by: Tim Brooksbank, CEO, Calibre UK Ltd. Technical Presentation by: Paul Carey, President, MSI Inc.

Modern projectors used fixed pixel numbers and positions To shape the picture to fit a screen or to overlay an object it is necessary to modify the shape of the image content in that signal To seamlessly blend multiple projectors to create very large or different shape images, the intensity of overlapping pixels must be accurately controlled This presentation discusses how this can be achieved Background

Projectors should be central and perpendicular to a flat screen Projectors should be the correct distance from the screen such that the picture just fills the screen The screen should be flat Screen aspect ratio should match projector’s native aspect ratio One projector is bright enough with suitable optics to generate the whole image In real world use this is often not the case The Ideal World

The screen is not always flat Projection is used in many applications other than cinemas and traditional lecture theaters: Immersive Caves and Domes, for Simulation, Themed Entertainment, Museums and Architect Designed Spaces as well as for Creative Use on Stage/Set. Screen aspect ratio and shape is often a creative decision of the presentation or experience designer The Real World

Warp or Projection Mapping is the manipulation of the projected signal to correct for distortions due to screen geometry and off axis projection. This is the mapping of the projector onto the surface. Blend is the term used to describe the lightness or intensity mapping of multiple projectors such that the array appears as a single projector. This is the mapping of several projectors into one image. Important note - Warping does not change the optical characteristics of the projector Depth of field is an optical function, the image needs to remain focused, this cannot be corrected electronically Warp & Blend or Projection Mapping

Warping by small amounts examples are overlaying one projected image on top of another for extra brightness or 3D alignment Two projectors cannot be in the same space so will have some errors in overlay Warping can be extreme an example from a London museum is making the image overlay an object shaped to look like a stained glass window in a church Why warp or map

A steeply angled and off center mount warped so that the image projected shows the circle correctly Why warp or map

When using two 4:3 projectors for a 16:9 screen or two 16:9 projectors for a 2.35:1 film application Architects often curve a screen to fit the building Here is an example in a London bank where the screen at the front of the theatre continues around two curves to the side wall Why warp and blend

Designers of public interactive exhibits are not restricted by convention In a London museum visitors walk on the exhibit, the screen is both on a wall and on the floor. There are five projectors, the three on the wall project at extreme angles Why warp and blend

Another example in a museum: 3 projectors used, 2 project on wall, 1 on floor All located high in the ceiling producing an image that interacts with physical scenery Why warp and blend

Multiple projectors are indicated when: - Aspect ratio of the design differs significantly from the aspect ratio of the projector - Projector placement (short throw or position) does not cover the surface design with a single projector - Brightness is desired The brightness of the image on the screen is a function of the amount of light produced by the projector(s) divided across the area of your screen Tiling an array of projector images is a method of getting more light on the screen, each projector covers a small area thereby increasing the overall brightness Blending the tiled array of projectors makes the image look like one image Multi-Projector Arrays

Create dedicated source material or modify content at source Modify the source material using an external processing box Modify the source material within the projector Blend and Warp - Three Options

Blending relies on feathering the light from each projector in the overlapping areas This process is achieved by controlling light intensity, this differs from video level as it must take account of projector gamma Matched projectors are important to achieve good results, including matching lamps and lamp-burn hours Contrast and color enhancing projector effects such as Vivid Color and “Brilliant Color” must be disabled as these cause gamma errors Blend and Warp – How it Works

Create dedicated source material Used in permanent displays Costly and inflexible Inappropriate where content is changed regularly Alternate content cannot be presented May be harder or more expensive to re-calibrate as that process is interactive with content producer Can be advantageous with screen shapes which cannot be mapped using curve fitting or are discontinous Blend and Warp – Option 1

Modify the source material between source and projector using and external box Flexible Can be used with any projector Any content can be shown Allows extremes of adjustment, provided the surface is continuous without hard angles – uses curve fitting Works with any source, doesn’t require application specific video servers Blend and Warp – Option 2

Modify the source material within the projector Some high end projectors now offer this option Option stays with projector Requires expensive projector options Limited projector choice Hard to transition to alternative projectors Range of adjustment usually limited User interface to control warping is usually rudimentary Blend and Warp – Option 3

Multiple projector arrays can improve projector utilization on the screen to improve image performance If the overall aspect ratio of the display is for example 3.5:1 then a single 16:9 projector would only utilize 1.78/3.5 of the imager. 50% of the light and pixels would be lost (not contribute to the image.) If that 3:5:1 aspect ratio display was formed with 3, 4x3 projectors then 3 x 1.33 ~ 4:1 88% of the imager is utilized (12% overlap to equal 3.5). The latter maximizes lumens per sq foot. Projector Choice and Utilization

For more detail the example being used is two dedicated processors to two projectors Let’s assume The projectors are mounted off angle Two 4:3 projectors are available A 16:9 image is required How to blend and warp

Align the projectors optically Maximize the number of pixels on the screen area but use the same amount of zoom on each projector if possible, this ensures the lumens per square foot per projector are equal Ensure the optical image covers the screen, warping can only operate within the projected area. Choose a suitable amount of overlap, this should be at least 10%, preferably 20% or more to get good results. In the special case of making a 16:9 image from two 4:3 projectors it is 2/3. Ensure focus is accurate before you start to warp and blend - many projection lenses change the size of the projected image as focus is adjusted How to blend and warp 2

Choose the native resolution of the projector The projector has a fixed display resolution A dedicated scaler will generally do a better job of scaling than an inbuilt scaler How to blend and warp 3

Choose the position of the projectors in the matrix. This defines which blend regions are appropriate It defines the section of image to be displayed by each projector when ‘Auto Zoom’ mode is used How to blend and warp 4

Choose the test alignment pattern This highlights the blend region to make it easy to see This is also an adjustable cursor positioned at the edge of the blend (follows ‘blend width adjustment’) How to blend and warp 5

Choose the Cross Hatch test pattern The cross hatch is scaled to fit the tiled image The ‘F’ confirms the projection orientation Cross hatch is an easy way to align the warp Colour is alternates to aid alignment How to blend and warp 6

Warp the images to fit the screen or projection target object with either four corner adjustment or via the warp map application How to blend and warp 7

There are two methods available to achieve the warp Four Corner adjustment is very efficient for alignment on a flat screen as linearity is automatically compensated when implemented well Independent X and Y adjustment of each corner, sometimes called four corner pinning How to blend and warp 8

Warp Map PC drag and drop mesh application This is very efficient for complex shapes & curved surfaces as well as for correcting lens distortion Ease of use depends on quality of configuration tools How to blend and warp 9

Auto-calibration software systems also exist for some hardware systems: These rely on camera feedback Fast and efficient, ideal for systems which need accurate alignment or need re-calibrating regularly A must for stacked projectors if sub-pixel precise alignment is required, e.g. 3D & 2D compatible stereo Expensive, usually priced per channel (projector) Require controlled ambient lighting conditions to achieve reliable results How to blend and warp 10

Choose S-Curve blend The alignment shading disappears The brightness of the blended regions each follow an S shaped curve to maintain 100% signal brightness across the blended region, crossing in the middle each at 50%. The S curve is performing lightness mapping such that at any point in the overlapping region the sum of the two contributing projectors is unity (equal lumens per sqft) How to blend and warp 11

Choose how to display image: ● preserve original content aspect ratio ●fill the available screen, ignoring content aspect ratio ●fill the screen but crop image (zoom) to preserve content aspect ratio – this is often desired Display input signal

Black level correction of a tiled image: Where projectors are tiled and the ambient light level is well controlled black level leakage” is visible All projectors pass some light even when the signal is black Some processors offer compensation for this effect Black levels are raised in individual areas to provide a smooth overall image Requires very high processing bit depth if greyscale banding and quantization noise are to be avoided, 48 bits/pixel typically Reduces image contrast ratio drastically so should be avoided except where essential, e.g. in dark viewing environment. Very Important: Before correcting black level, the black balance must be confirmed on all projectors. If there is an R,G or B bias, no amount of correction can be applied that will achieve an acceptable blend or seamless image. Additional options

Black level correction of a tiled image Note central projector requires 9 separate correction zones! Additional options

Any questions? Job done!

Thank you A copy of this presentation can be down-loaded from: http://www.calibreuk.com/downloads/infocomm2012.ppsx For further detailed help, come see us on the show-floor, booth N1153

Electronic Projection Mapping using Warp & Blend