PragMap

1. PragMap Pragmatic tactical rapid mapping Group 1

2. Team Chris Jones – MEng Computer Engineering Ben Roberts – MEng Computer Science Dave Sansome – MEng Computer Engineering Chris Scutcher – MEng Computer Engineering John Carter – Supervisor

3. Industry Sponsor Headquarters in Portugal with offices in UK, Romania, Brazil and US Focus on safety-critical development and safety assurance work with an emphasis on complex and leading-edge projects Clients include: NASA, ESA, Airbus, GE, BAE, EADS, Thales, EC, AgustaWestland, MBDA, ...

4. Introduction Our problem and how we intend to solve it

5. Scenario • Disaster or conflict scenario • Terrain altered drastically; • Fallen trees, buildings • Craters • Roads destroyed • Dams burst • Flooding • Need to get people out and supplies in!

7. Our Solution Our customer has put emphasis on a pragmatic software solution

8. Team Management Organisation and planning

9. Management • Chris Scutcher managing • Sub Team 1: Chris Scutcher, Chris Jones • Data acquisition • Terrain Identification • Path Finding • Sub Team 2: Dave Sansome, Ben Roberts • Projection Correction • 3D Reconstruction • Depth maps

10. Data acquistion Data considerations and possible acquisition methods

11. Considerations Realism – Is it close to scenario Coverage – Image overlap required Cost – Can we afford it Feasibility – Is it possible given the resources Metadata – GPS, Tilt, etc First and second hand data is considered

12. First hand acquisition techniques • Unpowered flight – Balloon • Powered flight – Hiring plane, model aircraft, helicopter • Building height – Cranes, ziplines, standing on top of zepler! • Additional considerations; • Safety • Equipment

13. Second hand acquisition techniques • Buying data • Looking for similar projects • Google Earth and similar • Additional considerations; • Difficult to get coverage • Difficult to meet our specific needs

14. Current Best Going up in an airplane this month Getting hold of images taken from a helicopter Requests for quotes out with various data providers

15. Projection Correction 3D reconstruction, Orthorectification and depth maps

16. Stitching • Identify key features in each frame • Detect motion in the scene • Transform the images such that they share the same perspective • Stitch the images together into a seamless panorama of the original scene

17. Correspondence

18. Lens Distortion Straight lines become distorted by the lens As the camera moves, an object will be distorted a different amount in each frame We need to calibrate the amount of distortion in the camera Undo the distortion in each frame

19. Orthorectification The acquired images are perspective projections, we can see the sides of buildings. But what we want to produce is a top-down view, like a map.

20. Stereo Correspondence Having two eyes means we see two separate images of objects in the world. Our brain detects depth by looking at the disparity between the images. In computer vision we normally set up a stereo rig where the two cameras are held a fixed distance apart.

21. Stereo Correspondence With one moving camera One moving camera

22. Stereo Correspondence With one moving camera One moving camera

23. Stereo Correspondence Rectification tries to transform the “left” and “right” images so that disparity occurs along horizontal lines. The problem is now the same as that of a fixed stereo rig.

24. Stereo Correspondence Disparity map

25. Triangulation 2D point on the first image (X)corresponds to a ray of possible 3D points. This ray is shown as a line in the second image. If we can find a matching point in the second image (x’) we can work out the original position in 3D space. Requires; Focal length (available in EXIF headers) Camera absolute position and orientation.

26. Terrain analysis Detection of Buildings, Roads, Foliage and Water

27. SVM Classification of Foliage Support Vector Machine Classification Machine learning technique that can be used for image classification. SVM is trained using sample data which has similar characteristics to the real data. SVM can then be used to classify the real data.

28. Road Detection Algorithm Developed by Emmanuel Christophe and JordiInglada Other Classification Methods SVM Classification can be insufficiently accurate when differentiating between buildings, roads and water. Other classification methods will be used to augment the SVM classification results. Depth map will also be used to help differentiate between roads and buildings.

29. Path Finding Computing an optimal route from the terrain Accessibility map

30. An example of A star (A*) algorithm in action. A* Search Algorithm Method: Traverses network searching all neighbours. Sort discovered paths by cost. Follow path with lowest cost until its cost rises above that of the next in the list. Continue until goal is reached.

31. Conclusion Summary, progress and questions

32. Progress Pursuing multiple avenues for data acquisition. Established which key steps are necessary to process the data. Prototyped software to do several key steps in this processing.

33. Next steps Acquire data Complete the image processing chain Create a terrain accessibility map from the processing output. Implement route finding algorithm

34. Thanks for listening Any questions?