Computational Geometry

1. Computational Geometry Piyush Kumar (Lecture 2: NN Search) Welcome to CIS5930

2. Our First Problem • Nearest neighbor searching • Applications? • Pattern Classification • Graphics • Data Compression • Document retrieval • Statistics • Machine Learning • …

3. Similarity Measure • In terms of Euclidean distance (4,5) (2,3)

4. Similarity Measure Similar?

5. Similarity measure • Other similarity Measures

6. The dimension • Lets assume that our points are in one dimensional space. ( d = 1 ). We will generalize to higher dimension ( When d = some constant ).

7. Fixed radius near neighbor problem Question • Given a set of points S on the real line, preprocess them to answer the following question : • Find all the pair of points (p,q) such that distance of (p,q) < r . q Points in S

8. Nearest neighbor search Question • Given a set of points S on the real line, preprocess them to answer the following query : • Given a query point q, find the neighbor of q which is closest in S. q nn(q) Points in S

9. Answers? • Fixed NN Search • O( n2 ) ? • O(nlogn + k) ? • O(n + k) ? • NN Search • O( n ) ? • O( log n) ?

10. Answers? • NN Search • O( n ) ?  Brute Force [ Trivial ] • O( log n) ?  Binary Search Tree • Fixed NN Search • O( n2 ) ?  Brute Force • O(nlogn + k) ?  Sorting • O(n + k) ?  Hashing?

11. NN search

12. NN Searching : Balanced binary tree O( log n ) q nn(q) Points in S

13. K-nearest neighbor search • Problem: Given a set of points P on the real line and a query point q, find the k-nearest neighbors of q in P. • O(nlogn)  Trivial bruteforce • Do you see how? • Thought Problem: How do we do this in O(n) time? • (Hint: Median finding works in O(n) time).

14. Fixed NN Search

15. Brute Force implementation • min = + • for( i = 0; i < n; ++i ) for( j = 0; j < n; ++j ) • (D = dist (pj, pi )) < r ) ? ( cout << (pj, pi ) ) : (); • float dist ( point p, point q ) • sum = 0; • for ( i = 0; i < d; ++i ) sum += (pi – qi)2 • return sqrt(sum) What can we speed up here?

16. Brute Force implementation • min = + • for( i = 0; i < n; ++i ) for( j = 0; j < n; ++j ) • (D = dist (pj, pi )) < r ) ? ( cout << (pj, pi ) ) : (); • float dist ( point p, point q ) • sum = 0; • for ( i = 0; i < d; ++i ) sum += (pi – qi)2 • return sqrt(sum) How do we speed this up?

17. Fixed NN Search: By Sorting • Once we sort the points on the real line, we can just go left and right to identify the pairs we need. • Each pair is visited at most twice, so the asymptotics do not change.

18. Total work done after sorting

19. Total work done after sorting • ki denotes the pairs generated when visiting pi • With this approach, we need at least Ω(nlogn) (for sorting).

20. Fixed radius near neighbor searching • How do we avoid sorting? • How do we get a running time of O(n+k) ? • How do we use hashing to simplify our problem.

21. b= -2 b=0 { r 0 Solution using bucketing • Put points in infinite number of buckets (Assume an infinite array B) Interval b is [ br, (b+1)r ] x lies in b = floor (x/r)

22. Solution using bucketing • Only n buckets of B might get occupied at most. • How do we convert this infinite array into a finite one: Use hashing • In O(1) time, we can determine which bucket a point falls in. • In O(1) expected time, we can look the bucket up in the hash table • Total time for bucketing is expected O(n) • The total running time can be made O(n) with high probability using multiple hash functions ( essentially using more than one hash function and choosing one at run time to fool the adversary ).

23. The Algorithm • Store all the points in buckets of size r • In a hash table [ Total complexity = O(n) ] • For each point x • b = floor(x/r); Retrieve buckets b, b+1 • Output all pairs (x,y) such that y is either in bucket b or b+1 and x < y and ||xy|| < r x 0

24. Running Time • Let nb denote the number of points in bucket b of the input pointset P. • Define Note that there are nb2 pairs in bucket b alone that are within distance r of each other.

25. Observation • Since each pair gets counted twice :

26. Running Time • Depends on the number of distance computations D. Total Running Time = O(n+k)

27. Higher Dimensions • Send (x,y)  (floor(x/r),floor(y/r)) • Apply hash with two arguments • Running time still O(n+k) • Running time increases exponentially with dimension

28. Introduction: Geometry Basics • Geometric Systems • Vector Space • Affine Geometry • Euclidean Geometry • AG + Inner Products = Euclidean Geometry

29. Vector Space • Scalar ( + , * ) = Number Types • Usual example is Real Numbers R. • Let V be a set with two operations • + : V x V  V • * : F x V  V • Here F is the set of Scalars

30. Vector Space • If (V , +, * ) follows the following properties, its called a vector space : • (A1) u + (v + w) = (u + v) + w for all u,v,w in V. • (A2) u + v = v + u for all u,v in V. • (A3) there is unique 0 in V such that 0 + u = u for all u in V. • (A4) for every u in V, there is unique -u in V such that u + -u = 0. • (S1) r(su) = (rs)u for every r,s in R and every u in V. • (S2) (r +s)u = ru + su for every r,s in R and every u in V. • (S3) r(u + v) = ru + rv for every r in R and every u,v in V. • (S4) 1u = u for every u in V. Note: Vectors are closed under linear combinations A basis is a set of n linearly independent vectors that span V.

31. Affine Geometry • Geometry of vectors • Not involving any notion of length or angle. • Consists of • A set of scalars • Say Real numbers • A set of points • Position specification • A set of free vectors. • Direction specification

32. Affine Geometry • Legal operations • Point - Point = Vector • Point +/- Vector =Point • Vector +/- Vector = Vector • Scalar * Vector = Vector • Affine Combination • ∑Scalar * Points = Point • such that ∑Scalar = 1 • Note that scalars can range from –Infinity to +Infinity

33. Affine Geometry Affine Combination Convex Combination

34. Affine Combinations • [ Affine Span or Affine Closure ] The set of all affine combinations of three points generates a plane. • [ Convex Closure ] The set of all convex combinations of three points generates all points inside a triangle.

35. Euclidean Geometry • One more element added • Inner Products • Maps two vectors into a scalar • A way to `multiply’ two vectors

36. Example of Inner Products • Example : Dot products • (u.v) = u0v0+ u1v1+…+ ud-1vd-1 • Where u,v are d-dimensional vectors. • u = (u0,u1,…, ud-1); v = (v0,v1,…, vd-1) • Length of |u| = sqrt(u.u) (Distance from origin) • Normalization to unit length : u/|u| • Distance between points |p - q| • Angle (u’,v’) = cos-1(u’.v’) • where u’=u/|u| and v’=v/|v|

37. Dot products • (u.v) = (+/-)|u|(projection of v on u). • u is perpendicular to v  (u,v) = 0 • u.(v+w) = u.v + u.w • If u.u not equal to zero then u.u > 0 • positive definite

38. Some proofs using Dot Products • Cauchy Schwarz Inequality • (u.v) <= |u||v| • Homework. • Hint: For any real number x • (u+xv).(u+xv) >= 0 • Triangle Inequality • |u+v|<=|u|+|v| • Hint: expand |u+v|2 and use Cauchy Schwarz. v u

39. Next Lecture • Orientation and Convex hulls Due Wed Homework: (Programming) Play with dpoint.hpp and example.hpp (Implement orientation in 2D). Implement your own dvector.hpp, dsegment.hpp using metaprogramming Make sure you understand how things work. Due Monday Homework: (Theory) Cauchy Schwarz Triangle Inequality dot n cross prod. Reading Assignment: Page 1-10, Notes of Dr. Mount Page 1-2 and Section 1.3 of the text book. Sources for this lecture: Dr. David Mount’s Notes. WWW.

40. Crash course on C++“dpoint.hpp” • Namespaces • Solve the problem of classes/variables with same name • namespace _cg {code} • Outside the namespace use _cg::dpoint • Code within _cg should refer to code outside _cg explicitly. E.g. std::cout instead of cout.

41. Object Oriented Programming • Identify functional units in your design • Write classes to implement these functional units • Separate functionality for code-reuse.

42. Class membership • Public • Private • Always : Keep member variables private • This ensures that the class knows when the variable changes • Protected

43. Inheritance • ‘is a’ relationship is public inheritance • Class SuperDuperBoss : public Boss • Polymorphism : Refer an object thru a reference or pointer of the type of a parent class of the object • SuperDuperBoss JB; • Boss *b = &JB; • Virtual functions

44. Templates • Are C macros on Steroids • Give you the power to parametrize • Compile time computation • Performance “The art of programming programsthat read, transform, or write other programs.” - François-René Rideau

45. Generic Programming • How do we implement a linked list with a general type inside? • void pointers? • Using macros? • Using Inheritance?

46. Templates • Function Templates • Class Templates • Template templates * • Full Template specialization • Partial template specialization

47. Metaprogramming • Programs that manipulate other programs or themselves • Can be executed at compile time or runtime. • Template metaprograms are executed at compile time.

48. Good old C • C code • Double square(double x) { return x*x; } • sqare(3.14)  Computed at compile time • #define square(x) ((x)*(x)) • Static double sqrarg; #define SQR(a) (sqrarg=(a), sqrarg*sqrarg)

49. Templates • Help us to write code without being tied to particular type. • Question: How do you swap two elements of any type? How do you return the square of any type?

50. Function Templates • C++ • template< typename T > inline T square ( T x ) { return x*x; } • A specialization is instantiated if needed : • square<double>(3.14) • Template arguments maybe deduced from the function arguments  square(3.14) • MyType m; … ; square(m);  expands to square<MyType>(m) Operator * must be overloaded for MyType