Project 2 Presentation & Demo

1. Project 2 Presentation & Demo Course: Distributed Systems By Pooja Singhal 11/22/2011

2. Outline • Requirement • Requirement Analysis • Challenges • Design • Data Structures • 3-Tiered Model • Algorithms • Implementation • Learning and Experience • Summary • Acknowledgement • Demo

3. Requirement • Design and Implement a 3-Tiered Client Server Model. • Given a City and State, find 5 nearest Airports. • Using RPC • System: Linux machine, Sun RPC, Language: C++/C

4. Requirement Analysis 3) 3-Tiered Model Requirement 4) Requirement of Algorithm To search Nearest Neighbors • Client • Places Server • Airport Server 1) Parsing of Places2k.txt – Fast and Efficient. Which DS? 2) Parsing of airport-locations.txt - Spatial Partitioning. Which DS?

5. Challenges • 3-Tiered Client Server Model. • Spatial Partition: Did not know much about it! • How to Search 5 nearest Airports ? Again No idea! 1) Parsing of Places2k.txt 2) Design and Implement Client 3) Design and Implement Places Server Test the code So far 4) Design and Implement 3-Tiered Model 5) Design and implement Spatial Partitioning of data in airport-locations.txt 6) Design and implement N nearest Airports Search My Approach: Divide and Conquer

6. Design

7. Design Tactics Weekly Submissions made the job easy! • First week Design: Parse both the files, Data Structures • Second Week Design: • IDL Design : Structure Design, Function Design • Client Design and Logic: • Places Server Design: Server for Client • Final Week Design • 3-Tiered Model Design • Change of Data Structure for airport-locations.txt records • Nearest Neighbor Search design

8. 3-Tiered Model CLIENT PLACES SERVER AIRPORT SERVER

9. Client Design • Client • Gets the location from User in the form of CITY and STATE. • Pass it to Places Server • Display the results

10. Places Server Design Phase 1 • As a Server to Client: • On start up, parse places2k.txt in hash table • Hash Table key is combination of “CITY” and “STATE” • Latitude and Longitude are stored as DATA along with the key • Gets the inputs (CITY and STATE) from Client • Make the Key: Apply Hash Function • Search Hash Table • If Found: Get Latitude and Longitude Phase 2: • As a Client to Airport Server: • Pass on Latitude and Longitude to Airport Server • Get the result back from Airport Server • Return the result to Client.

11. Airport Server Design • Act as a Server to Places Server • On start up, parse airport-location.txt • Creates a K-D Tree in memory • Gets the latitude and location from places server. • Search Nearest Neighbor • Calculate the Distance • Sort the results on Distance • Return 5 neared neighbor back to places server

12. Design - Data Structures

13. Data Structures (1) • Hash Table • Store places2k.txt records • Key is combination of CITY and STATE • Data: Latitude and Longitude • Advantages: Fast • K-D Tree • Spatial Partitioning of 2 Dimensional Points consist of Latitude and Longitude • Node consists of 2 Dim points (Latitude, Longitude) • Node Data: Airport Code, Airport Name, State • Linked List • Store Results consisting of 5 nearest airports • Since, pointers do not get passed over RPC, needed to store the address of the next record

14. Data Structures (2) • K-D Tree • Space Partitioning Data Structure for storing a finite set of points in a k-dimensional space • Invented by J Luis Bentley in 1975 • Is a Binary Tree: Special example of Binary Space Partitioning Trees • Applications in wide areas: Neural Networks, searching multidimensional data 1. X Plane Division 2. Y Plane Division 3. X Place Division (2,3), (5,4), (9,6), (4,7), (8,1), (7,2) Source: http://pointclouds.org/documentation/tutorials/kdtree_search.php

15. Design - Algorithms

16. Algorithms Nearest Neighbor Search • Starting with the root node, the algorithm moves down the tree recursively • Once the algorithm reaches a leaf node, it saves that node point as the "current best“ • The algorithm unwinds the recursion of the tree, performing the following steps at each node: If the current node is closer than the current best, then it becomes the current best. • The algorithm checks whether there could be any points on the other side of the splitting plane that are closer to the search point than the current best • done by intersecting the splitting hyperplane with a hypersphere around the search point that has a radius equal to the current nearest distance. • Since the hyperplanes are all axis-aligned this is implemented as a simple comparison to see whether the difference between the splitting coordinate of the search point and current node is less than the distance (overall coordinates) from the search point to the current best. • If the hypersphere crosses the plane, there could be nearer points on the other side of the plane, so the algorithm must move down the other branch of the tree from the current node looking for closer points, following the same recursive process as the entire search. • If the hypersphere doesn't intersect the splitting plane, then the algorithm continues walking up the tree, and the entire branch on the other side of that node is eliminated. • When the algorithm finishes this process for the root node, then the search is complete.

17. Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

18. Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

19. Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

20. Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

21. Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

22. Source: http://en.wikipedia.org/wiki/File:KDTree-animation.gif

23. Implementation

24. Implementation Weekly Submissions made the job easy! • First week Implementation: • Parsing of places2k.txt and airport-locations.txt, Storage in Hash Tables • Second Week Implementation: Make Client and Places Server work Properly • IDL implementation: client.x • Implementation of Client Logic : client.c • Places Server Implementation: places_server.c , client_svc.c • Final Implementation • Phase 1 : 3-Tiered Model Implementation • Phase 2: Implementation of K-D Tree • Phase 3: Implementation of Nearest Neighbor Search

25. Implementation • Phase 1 : 3-Tiered Model Implementation • 2nd IDL : placesclient.x created • places_server.c modified to call airport server • Ist IDL client.x modified to include “host” input • Phase 2 : K-D Tree Creation • Airport server creates K-D tree and stores airport records. • Libkdtree++ is used: kd_create(), kd_insert3() • Phase 3: Nearest Neighbor Search • kd_nearest_range(tree, point, radius) • Calculation of Distance of all the points inside the circle with the Point • Top 5 Nearest Airports were selected and returned • Change of Resultant Structure: Made as Linked List

26. Learning and Experience GREAT LEARNING EXPERIENCE !! • 3-Tiered Client - Server Model • Data Distribution on different Servers • Space Partitioning of multi dimensional data • Search in Multi Dimensional data – Practical Approach • Working with Hash Tables, K-D Trees, Linked List, Sort

27. Summary • Implemented 3-Tiered Client Server Model • Use of Hash Table to store places2k.txt • Use of K-D Tree to store airport-locations.txt • Use of Nearest Neighbor search algorithm • Use of Linked List to return Result containing nearest airports

28. Acknowledgements • Martin Krafft, Paul Harris, Sylvain Bougerel • Library: libkdtree- Open Source STL Like implementation of K-D Trees

29. Demo

30. Thank you