Projector Calibration of Interactive Multi-Resolution Display Systems

1. 互動式多重解析度顯示系統之投影機校正 Projector Calibration of Interactive Multi-Resolution Display Systems Presenter: 邱柏訊 Advisor: 洪一平 教授

2. Outline • Introduction • Related Work • Calibration of Interactive Tabletop Displays • Calibration of Interactive Cylindrical Displays • Experiments and Error Analysis • Conclusion

3. Introduction • Multi-Resolution Display System • Features • A future personal desk • Cost-effective large high-res display

4. Introduction • i-m-Top Mirror & PTU IR Camera IR LEDs Peripheral Projector Fovea Projector

5. Introduction • Motivation • Disadvantages of Manual Calibration • Tedious and inefficient • Technician dependent accuracy • Goal • Automatic Calibration • Fast and accurate

6. Outline • Introduction • Related Work • Calibration of Interactive Tabletop Displays • Calibration of Interactive Cylindrical Displays • Experiments and Error Analysis • Conclusion

7. Related Work • Automatic Projector Calibration • Smarter Presentations • Camera-assisted approach -1 P = CT T C R. Sukthankar, R. Stockton, and M. Mullin, “Smarter Presentations: Exploiting Homography in Camera-Projector Systems,” In Proceedings of International Conference on Computer Vision (ICCV), 2001.

8. Related Work • Automatic Projector Calibration • Lee et al. • Optical sensors embedded • Gray code patterns Gray Code Pattern Lee, J., Dietz, P., Aminzade, D., Raskar, R., and Hudson, S. "Automatic Projector Calibration using Embedded Light Sensors", Proceedings of the ACM Symposium on User Interface Software and Technology, October 2004.

9. Outline • Introduction • Related Work • Calibration of Interactive Tabletop Displays • Hardware Configuration • Implementation • Calibration of Interactive Cylindrical Displays • Experiments and Error Analysis • Conclusion

10. Calibration of Interactive Tabletop Displays Color Camera • Hardware Configuration

11. Calibration of Interactive Tabletop Displays • Implementation • Step1.Peripheral Projector Calibration • Step2. Fovea Projector Calibration • Step3. Infrared Camera Calibration

12. Implementation H S PP • Peripheral Projector Calibration Camera( C ) Surface( S ) Our goal Peripheral Projector ( PP )

13. Implementation H S C • Peripheral Projector Calibration White Pattern

14. Implementation Visual Pattern H H Option 1: Circle Pattern S PP C C • Peripheral Projector Calibration Option 2: Concentric Circle Pattern Option 3: Middle Line Pattern

15. -1 Implementation H H H H H S S S PP PP = C PP C C C • Peripheral Projector Calibration Gray Code Pattern

16. Implementation • Peripheral Projector Calibration

17. Calibration of Interactive Tabletop Displays • Implementation • Step1.Peripheral Projector Calibration • Step2. Fovea Projector Calibration • Step3. Infrared Camera Calibration

18. Implementation H S FP(θ, ϕ) • Fovea Projector Calibration Cam( C ) Surface( S ) Our goal Fovea Projector ( FP )

19. Implementation Visual Pattern H H Option 1: Circle Pattern S FP(θ,ϕ) C C • Fovea Projector Calibration Option 2: Concentric Circle Pattern Option 3: Middle Line Pattern

20. -1 Implementation H H H H H FP(θ,ϕ) S S S FP(θ,ϕ) = FP(θ,ϕ) C C C C • Fovea Projector Calibration Gray Code Pattern

21. Implementation • Fovea Projector Calibration • Problem : Project to anywhere desired • Range of PTU angle : pan(-3087~3087), tilt(-908~604) • Calibration for all PTU angles is impractical • Solution • Sample PTU angles with fixed interval • Interpolate mapping function (f1and f2 )

22. Implementation • Mapping Function Interpolation • Record trajectories of fovea projection

23. Implementation • Mapping Function Interpolation • Record trajectories of fovea projection

24. Implementation • Mapping Function Interpolation • Interpolate f1 for one given surface coordinate ps Surface X Y

25. Implementation • Mapping Function Interpolation • Interpolate f2 for one given PTU angle (θ,ϕ) Surface X Y

26. Implementation • Fovea Projector Calibration

27. Calibration of Interactive Tabletop Displays • Implementation • Step1.Peripheral Projector Calibration • Step2. Fovea Projector Calibration • Step3. Infrared Camera Calibration

28. Implementation H H S PP Surface( S ) IRC S • Infrared Camera Calibration Source Image Warped Image IR Camera(IRC)

29. Implementation • Infrared Camera Calibration

30. Calibration of Interactive Tabletop Displays • Summary Calibrate peripheral projector Calibrate IR cameras Obtain valid projector angle Repeat for all sampled PTU angles Calibrate fovea projector : Peripheral Projector Calibration : Fovea Projector Calibration Interpolate mapping function : Infrared Camera Calibration

31. Outline • Introduction • Related Work • Calibration of Interactive Tabletop Displays • Calibration of Interactive Cylindrical Displays • Hardware Configuration • Projector Calibration • Experiments and Error Analysis • Conclusion

32. Calibration of Interactive Cylindrical Displays • Hardware Configuration Front View Back View 90 cm Hardware Component Surface 120 cm Surface 85 cm Projector Projector

33. Calibration of Interactive Cylindrical Displays • Projector Calibration • Piecewise Planar Mapping • Texture mapping • Using corresponding points between surface & projector • Problem • Projector intrinsics estimation

34. Calibration of Interactive Cylindrical Displays P4 P3 P2 P1 P0 • Projector Calibration • Projector Intrinsics Estimation • Step1. Initial guess • Step2. Measure corresponding points m and M a = [fx, fy, cx, cy] : m P9 : M P8 P7 Marker P6 P5 Surface Projection

35. Calibration of Interactive Cylindrical Displays • Projector Calibration • Projector Intrinsics Estimation • Step1. Initial guess • Step2. Measure corresponding points m and M • Step3. Estimate extrinsicsE with a, m, M • Step4. Update withE,m,M • Step5. Repeat step 3, 4 until a = [fx, fy, cx, cy] a = a + △a ||△a || < ε

36. Calibration of Interactive Cylindrical Displays • Projector Calibration Source Image Warped Image Pre-Warping

37. Outline • Introduction • Related Work • Calibration of Interactive Tabletop Displays • Calibration of Interactive Cylindrical Displays • Experiments and Error Analysis • Conclusion

38. Experiments and Error Analysis - Calibration of Tabletop Displays • Comparison of Calibration Time • Manual vs. Auto Unit: minute FP :Fovea Projector PP : Peripheral Projector

39. Experiments and Error Analysis - Calibration of Tabletop Displays • Error Analysis • Calibration Error of Peripheral Projector

40. Experiments and Error Analysis - Calibration of Tabletop Displays • Error Analysis • Calibration Error of Fovea Projector • Without mapping function interpolation

41. Experiments and Error Analysis - Calibration of Tabletop Displays • Error Analysis • Calibration Error of Fovea Projector • With mapping function interpolation

42. Experiments and Error Analysis - Calibration of Tabletop Displays • Applications are run after Calibration

43. Outline • Introduction • Related Work • Calibration of Interactive Tabletop Displays • Calibration of Interactive Cylindrical Displays • Experiments and Error Analysis • Conclusion

44. Conclusion • We propose an automatic projector calibration method of interactive tabletop displays, which is both efficient and accurate • The fovea projector is able to correctly project over the whole tabletop surface • Calibration of cylindrical displays is achieved with semi-automatic approach

45. Thanks for your listening