THE-I: SOLVING an overlooked problem OF stereo based solely on binocular disparity

1. THE-I: SOLVING an overlookedproblem OF stereobased solely onbinocular disparity Rob Black

2. Advantages of binocular disparity based stereo • Greatly enhanced visual detail and vividity. • Accuracy based on mathematical derivation • Ability for this cue to function in isolation But, in conventional and recent 3D glasses systems this very problem is the undoing. Disparity is phenomenally accurate at dictating that the screen being viewed is totally flat.

3. So what if we know perceptually that the screen is flat? • Evolution designed us to have two laterally separated eyes, to help foreground objects stand out more in space. • This advantage converts to a loss when we look at a computer screen or a flat canvas/paper • There is nothing to see round, and all of our life’s experience tells us that the surface is flat, and that pictures on flat surfaces are flat. • We are so conditioned that pictures are flat that we don’t acknowledge another possibility...

4. Hold on, why not just close one eye, wouldn’t that be the same? • Partially. Closing one eye does remove binocular disparity information. • However, the eye still makes vergence movements as if matching the other closed eye. • Plus, one closed eye dictates an ambiguous depth percept – could be flat or deep. • Two eyes in parity present an unambiguous depth percept – the visual information is deep. • Besides, have you tried holding one eye closed for extended periods? Very uncomfortable...

5. The effect of other cues • Convergence is nowhere near as accurate as binocular disparity for determining depth. • However, it is somewhat accurate • In that, if the viewer is looking at a screen less than 10m away, there will be a noticeable degree of visual convergence • And ditto for anywhere between 100 and 400m away for binocular disparity • This tells the brain that the objects are nearby • This may be why 3D film-makers often used hyper-stereo. Now, they tend to use hypo.

6. Any other problems with binocular disparity stereo? • It virtually never works consistently across the board. There is inevitably one + factor amiss. • There is almost always a degree of ghosting, processor slowdown, resolution loss, misrendered foreground objects, inappropriate scaling, camera misalignment, parallax etc. • This is being resolved by the industry daily. • PC games can be scaled to IPD but movies can’t • This is until multi-viewer glasses-free and depth-map scaled systems which maintain roundness for near objects are implemented

7. So what is the alternative? • Binocular disparity is a very good cue to depth, but also an incredibly good cue to flatness. • In the first instance, removing real binocular disparity information results in the surface being unspecified, or specified as far away. • However, crucially, the convergence cue must be removed by encouraging vision to look straight ahead, not focus on a near point. • Accommodation needs correction to infinity. • By combining these factors we can fool the eyes

8. Fooled How? • Fooled into perceiving the visual information as being extant in three dimensional space. By removing binocular disparity and convergence cues to flatness, the brain can perceive any image, movie, photograph, artwork or video game with it’s natural implicit depth map. Anything, ever created can look more three dimensional, just by removing real depth info.

9. How in the world did no one discover this effect before? • They did! In 1874 Jentzsch observed non-stereoscopic depth in microscopes. • 3D visionaries such as Helmholtz, Gregory, Gibson, Wheatstone and Brewster all acknowledged monocular/plastic relief. • In 1903-7, Moritz Von Rohr (Zeiss) patented a synopter device to be marketed to art galleries which failed due to cost and ergonomic issues. • The only remnant is the binoviewer attached to telescopes. People frequently comment that the space objects look more 3D and more detailed.

10. But the optical configuration is so simple, why wasn’t it done? • The device exists in various guises with a single beam splitter. However this gives it a large footprint and a very small FOV • A large FOV is critical for the effect to work. • The double beam splitter device requires the alignment accuracy to be almost perfect, cutting prisms to fit the correct IPD a • There must be no visible seaming or mis-scale. • Such prisms do exist already in LCD projectors, but are tinted with RGB colour filters • Manufacture is simple, affordable and intuitive, however not implicitly obvious.

11. Surely an HMD or globally offset 3D glasses can do the same? • Conventional 3D glasses can be given a global offset which does bring the picture forward. However, the convergence and binocular disparity flatness information remain, changing only the screen content, not context • It is not the added depth, so much as the removed flatness which aids this effect to work. • Only the very best HMDs have a sufficiently large field of view and pixel density to pass off as normal. So it is out of reach but to a very few (eg users of the £500,000 piSight etc.)

12. But still, couldn’t we make this device with an HD helmetcam and two high resolution TFTs? • Yes, it is theoretically possible to make this device using electronic components. • It would require 400 DPI+ screens and an extremely good, tiny camera • However, only the very best camera viewfinder screens have sufficient DPI and contrast ratios, • (THE-I would have an RRP of under €100). • The contrast and resolution of the real world is still orders of magnitude better than the best electronics commercially available today.

13. What is the perceptual experience of THE-I? • Effortless natural depth produced by the eyes being completely relaxed. • We are so used to viewing pictures with our normal eyes that there is quantitative shift. • Motion parallax is greatly enhanced (perhaps because zero disparity and parallel convergence both specify infinite depth) • Monocular depth cues also look more salient, strong perspectives can look very realistic. • There is massively more visual detail evident compared to normal viewing.

14. So in practical terms... • BINOCULAR DISPARITY 3D • Computationally and economically costly • System specific • Requires custom hardware (monitors) • Requires software tuning profiles for each game • BINOCULAR PARITY 3D • No electronics, profiles, flickering • No intervention with source material • Moderately priced • Compatible with virtually everything

15. What practical use can it be? • PC games and applications lack a consistent, specialised setting to make them look 3D. • Fragmented environment-specific drivers & wrappers that only work with some programs. • What if I want to use a graphic design package, see photos in more 3D, play an FPS at 1080P (frame rate barely normal) & play my console? • No existing 3D system currently does all these things, and certainly can’t handle the range.

16. Is there any possible overlap? • Yes! 3D shutter or polarising glasses are independent technology from THE-I • Combining the two results in a perfect, stable perception with no visual strain of identically placed physical images in both eyes, filtered either by flickering or polarising. • The cancellation of distracting surrounding information and removal of flatness information result in a dramatically more vibrant and convincing 3D setup.

17. In Summary • THE-I works as a standalone device. Cancelling the disparity and convergence cues to flatness results in powerful and unexpected increased depth in the picture or moving picture. • THE-I can also be combined with all existing disparity-based 3D technologies creating a much more powerful effect than before. • THE-I is a unique device, drawing 150 years of sidelined visual theory back to the fore.