Autonomous 3D Reconstruction From Irregular LiDAR and Aerial Imagery

1. Autonomous 3D Reconstruction From Irregular LiDAR and Aerial Imagery Nicholas Shorter BSEE, May 2005; MSEE, Aug. 2006 PhD Committee: Dr. Takis Kasparis (Chair) Dr. Georgios Anagnostopoulos, Dr. Michael Georgiopoulos, Dr. Andy Lee, Dr. Wasfy Mikhael Email: nshorter@cfl.rr.com Website: http://www.nshorter.com http://www.nshorter.com

2. LIDAR Overview Problem Statement Present Art Proposed Solution Completed Work Future Work Potential Implementation Obstacles Milestones Presentation Layout http://www.nshorter.com

3. LIDAR Overview • Data Collection • Plane Equipped with GPS, INS & LIDAR • LIDAR – Light Detection and Ranging (active sensor) • Collection of 3D points • Laser sent out from Emitter, reflects off of Terrain, Returns to Receiver • Receiver measures back scattered electromagnetic radiation (laser intensity) • Time Difference Determines Range to Target http://www.toposys.com/ http://www.nshorter.com

4. LIDAR Captured Characteristics • Range to Target (elevation from INS & LiDAR) • Longitude and Latitude (GPS) • First and Last Return Pulses • First – shrubbery, vegetation, power lines, birds and buildings • Last – buildings (unless vegetation is really dense, then vegetation too) • Returned Laser Intensity http://www.nshorter.com

5. LiDAR Noise • Geolocation results from LiDAR, GPS and INS sensor systems • Accuracy Limitations • Offset and Drift in both GPS and INS • Misalignment between INS and LiDAR • Atmosphere – Intensity and Path Distortion • Shadowing Effect from Tall buildings • Artifacts from non uniform sampling from multiple strips http://www.nshorter.com

6. Problem Statement • Input • LIDAR (Light Detection and Ranging) Data • Collection of Irregularly Distributed 3-Dimensional Points • Aerial Photograph • Output • Semi to complete Automatic (minimal user intervention) development of 3-Dimensional Virtual Model http://www.nshorter.com

7. Military Applications Automatic Target Recognition Reconstructed Models of Opponent Terrain (UAV?) Tourism/Entertainment Virtual Walkthrough of Theme Park Commercial Change Detection (Natural Disasters) Network Planning for Mobile Communication Noise Nuisance (Universal Studios, 408 Expressway - walls) Urban Planning Applications for 3D Reconstruction http://www.nshorter.com

8. Ideal Attributes Handle Complex (non planar) buildings Irregular Points Multiple Source as Available Reconstruction exists as membered simple entities Building Isolation Complete Automation Aerial Image Projection Dissertation Implementation Yes Yes Only LiDAR and Aerial Imagery Yes Yes Yes Yes Ideal Algorithm Features http://www.nshorter.com

9. Existing 3D Reconstruction Research • Data Sources • LIDAR • Aerial Imagery • GIS Ground Plans • Model Based Reconstruction • Pre-defined models with parameters • Minimize error between models and data • Data Driven • Group Coplanar Pts • Identify Break Lines • Derive Model to Minimize Error http://www.nshorter.com

10. Triangulation Based Methods • Triangular Irregular Network (TIN) • Series of Non-Overlapping Triangles Modeling given Surface • TIN 3D Reconstruction Methods • Clustering approach • Spherical Normal Vectors of Triangles • TIN region growing approach • Merge Triangles to Same Region if Normal Vectors within Threshold http://www.nshorter.com

11. Existing 3D Reconstruction Methods • Most still under development • Most Methods Use ‘Grided’ (Interpolated) LIDAR Data • Advantages • Less Computationally Complex • DTM & DSM Thresholding to distinguish Building from Non Building • Use of additional conventional methods • Disadvantages • Decrease in Accuracy • Uncertainty from Building and Ground Interpolation http://www.nshorter.com

12. Masters Research • Greedy Insertion Triangulation • Implemented Noise Filtering Technique • Proposed FSART normal vector clustering • Proposed Planar Regression to combat Category Proliferation • Realized simple planar reconstruction algorithm (MSEE Thesis) http://www.nshorter.com

13. Post Masters, Pre-Candidacy Research • Implemented FA, GA, and FSART Clustering for reconstruction performance comparison • Developed Category Proliferation and Clustering Performance measures • Ran comparison on 4 buildings http://www.nshorter.com

14. Proposed Algorithm – System Level • Anticipated Challenges: • Automatic Image and LiDAR Registration • Matlab Rendering Reconstructed City Blocks • Matlab mapping Images to Models http://www.nshorter.com

15. Building Detection • Anticipated Challenges: • Morphological Filtering needs a priori window size http://www.nshorter.com

16. Wall Tri. And Gnd Pt. Identification • Wall Triangle = (Max diff. in elev. > 2m) & (pitch > 60 degrees) • Gnd Pt. = Wall Tri. Lowest elevation pt http://www.nshorter.com

17. Building Extraction • Clustering will automatically identify and separate individual buildings http://www.nshorter.com

18. Building Features • Vegetation Points • Significant first and last return elevation diff. • Corresponding green aerial image color • Nearest points have diff. elev. or adjacent triangles have significantly diff. norm vectors • Common Building Points • Spatially close in terms of long. and lat. • Bounded by aerial img. edges and exterior wall tri. • Bounded building does not contain terrain pts – or - • Triangulation of all points, building points connected http://www.nshorter.com

19. Building Reconstruction http://www.nshorter.com

20. Building Reconstruct Cluster Features • Surface change - normal vector orientation difference between adjacent triangles • (X,Y,Z) - Longitude, Latitude, Elevation • Edge - Aerial imagery edge detection • Color - Aerial imagery corresponding color (building surface differs from clutter) • Triangle planar coef, pt. height diff., same normal vector, or planar equation • Feedback – difference between reconstruction and raw LiDAR and Aerial Image Edges http://www.nshorter.com

21. Algorithm Novelty • Novel Building Detection Method– Irregular LiDAR • Uses raw pts and triangulation • Novel Automated Building Extraction • Clustering for automated extraction • Novel Clustering for Reconstruction • Capable of handling complex building structure • Proposing Clustering instead of Existing (ART, K-Means, etc.) • Automated LiDAR and Image Registration http://www.nshorter.com

22. Implementation Obstacles • Matlab Obstacles • Matlab Rendering of Large City Sets • Matlab displaying Large City Models with Mapped Images • Will have to program Open GL or other rendering solution (learning curve) • Additional reader development for different data sources (Fairfield mostly simple buildings) http://www.nshorter.com

23. Future Tasks and Milestones – Su 07 • Investigate systematic noise removal • Continue Debugging and Testing of GIT for 150k+ pts • Currently works for 10k pts in > 10 seconds • Development of Building Detection • Investigate no a-priori max build size for morphological filtering http://www.nshorter.com

24. Future Tasks and Milestones – Fa 07 • Develop Automatic LiDAR & Image Registration • Development of Building Extraction • Investigate which clustering approach • Publish Detection and Extraction Procedure http://www.nshorter.com

25. Future Tasks and Milestones – Spr 08 • Develop Reconstruction Algorithm • Propose Clustering Method • Automatically map images to constructed models • Develop OpenGL to render multiple building subsets (neighborhood block sizes) http://www.nshorter.com

26. Future Tasks and Milestones – Su 08 • Publish Reconstruction Algorithm • Write Dissertation (~July 21 deadline) • Defend Dissertation (~July 10 deadline) http://www.nshorter.com

27. Acknowledgements • Data Contributors • Dr. Simone Clode, Dr. Franz Rottensteiner, AAMHatch • Mr. John Ellis, AeroMap • Mr. Steffen Firchau, TopoSys • Mr. Paul Mrstik, Terra Point • Advisor (Committee Chair) • Dr. Takis Kasparis • Committee Members • Dr. Michael Georgiopoulos , Dr. Georgios Anagnostopoulos, Dr. Andy Lee, and Dr. Wasfy Mikhael http://www.nshorter.com

28. Accomplishments • WSEAS International Conference on Systems Theory and Computation – August 2006 • WSEAS Transactions on Signal Processing – August 2006 • MSEE – August 2006 • Invited Harris Talk • UCF Graduate Research Forum Poster Board Presentation (Spring 2006) • UCF Graduate Research Forum Oral Presentation (Spring 2007) http://www.nshorter.com

29. Thank You for your Attendance Additional Questions? http://www.nshorter.com