1 / 34

Indexing Multi-Dimensional Uncertain Data with Arbitrary Probability Density Functions

Indexing Multi-Dimensional Uncertain Data with Arbitrary Probability Density Functions. Yufei Tao, Reynold Cheng, Xiaokui Xiao, Wang Kai Ngai, Ben Kao, Sunil Prabhakar City University of Hong Kong Hong Kong Polytechnic University University of Hong Kong Purdue University.

gus
Download Presentation

Indexing Multi-Dimensional Uncertain Data with Arbitrary Probability Density Functions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Indexing Multi-Dimensional Uncertain Data with Arbitrary Probability Density Functions Yufei Tao, Reynold Cheng, Xiaokui Xiao, Wang Kai Ngai, Ben Kao, Sunil Prabhakar City University of Hong Kong Hong Kong Polytechnic University University of Hong Kong Purdue University

  2. Multi-dimensional Uncertain Data • Moving objects • An object sends its location to a server whenever its distance from the previously reported location is larger than certain threshold. • Sensor readings • Each sensor reports the temperature, humidity, UV index, …, in its neighborhood periodically. • Querying the (uncertain) data stored in the server directly is meaningless.

  3. Uncertainty Modeling An object’s location is described by a probability density function.

  4. Probabilistic Range Search Find theclients that are currently in CityU with at least 50% probability (probabilistic range query)(probability threshold)

  5. Appearance Probability E.g., uniform pdf: apperance probability:

  6. Appearance Probability must be calculated numerically Calculation time of an appearance probability in 2D space: 1.3ms Time for a random access: 10ms

  7. A good solution should… • Support any pdf. • Minimize the number of page accesses. • Minimize the number of appearance probability calculations. • Minimize the total cost (I/O + CPU)

  8. Main Idea • Pre-compute some “auxiliary information” that can be used to • efficiently decide whether an object appears in a region with at least a certain probability • without calculating its actual appearance probability.

  9. Quick Examples pq=20%

  10. Probabilistically Constrained Regions (PCR)

  11. Probabilistically Constrained Regions (PCR) For a query q with search region rq and probability pq= 0.2 • Observation 1.1 (pruning) an object o can not satisfy q if rqdoes not intersecto.pcr(0.2)

  12. Probabilistically Constrained Regions (PCR) (=1 – 0.2) For a query q with search region rq and probability pq= 0.8 • Observation 1.2(pruning) an object o can not satisfy q if rqdoes not fully containo.pcr(0.2)

  13. Probabilistically Constrained Regions (PCR) A query q with search region rq and probability pq= 0.2 • Observation 1.3 (validating) an object o definitely satisfies q if rq fully contains the part of o.MBR on the left of l1-(or on the right of l1+or below l2-or above l2+)

  14. Probabilistically Constrained Regions (PCR) A query q with search region rq and probability pq= 0.8 • Observation 1.4 (for validating) an object odefinitely satisfiesq if rq fully contains the part of o.MBR on the left of l1+ (or on the right of l1-or below l2+or above l2-)

  15. Probabilistically Constrained Regions (PCR) A query q with search region rq and probability pq= 0.6 =(1 – 2 * 0.2) • Observation 1.5(for validating) an object o must satisfy q if rqfully contains the part of o.MBR between l1-and l1+(or between l2-and l2+)

  16. Probabilistically Constrained Regions (PCR) o.pcr(0.2)provides 5 heuristics to reduce CPU cost In general, for a prob-range query with probability threshold pq • if pq <= 0.5 • o may be pruned using o.pcr( pq ) observation 1.1 • o may be validated using o.pcr( pq ) observation 1.3 • o may be validated using o.pcr( (1 - pq)/2 )observation 1.5 • if pq > 0.5 • o may be pruned using o.pcr( 1 - pq ) observation 1.2 • o may be validated using o.pcr( 1 - pq ) observation 1.4 • o may be validated using o.pcr( pq /2 ) observation 1.5 pq in [0, 1]→ infinite number of pq → infinite number of PCRs Impractical! It is possible to use a finite number of PCRs to achieve pruning and validating.

  17. Using PCRs in a Conservative Way E.g., U-catalog: { 0, 0.1, 0.2, 0.3, 0.4, 0.5 } for a query q with search region rq and probability pq= 0.25 • Observation 1.1 an object o cannot satisfy q if rqdoes not intersect o.pcr(0.25) • Observation 2.1 an object o cannot satisfy q if rqdoes not intersect o.pcr(0.2)

  18. Using PCRs in a Conservative Way U-catalog: { 0, 0.1, 0.2, 0.3, 0.4, 0.5 } for a query q with search region rq and probability pq= 0.75 • Observation 1.2 an object o cannot satisfy q if rq does not fully contain o.pcr(0.25) • Observation 2.2 an object o cannot satisfy q if rq does not fully contain o.pcr(0.3)

  19. U-catalog Size m {0, 0.5}, m = 2 {0, 0.25, 0.5}, m = 3 {0, 0.1, 0.2, 0.3, 0.4, 0.5}, m = 6 … larger m→ more PCRs→ greater pruning/validating power → less CPU cost larger m → higher space consumption → larger I/O cost m = 9

  20. Conservative Functional Boxes (CFB) U-catalog: { 0, 0.1, 0.2, 0.3, 0.4, 0.5 } o.pcr : 2m values for each dimension o.cfbout : 4 values for each dimension o.cfbin : 4 values for each dimension total : 8 values m = 9 8 : 18

  21. Conservative Functional Boxes (CFB) for a query q with search region rq and probability pq= 0.25 U-catalog: { 0, 0.1, 0.2, 0.3, 0.4, 0.5 } • Observation 1.1 an object o cannot satisfy q if rqdoes not intersect o.pcr(0.25) • Observation 2.1 an object o cannot satisfy q if rqdoes not intersect o.pcr(0.2) • Observation 3.1 an object o cannot satisfy q if rqdoes not intersect o.cfbout(0.2)

  22. Conservative Functional Boxes (CFB) for a query q with search region rq and probability pq= 0.75 U-catalog: { 0, 0.1, 0.2, 0.3, 0.4, 0.5 } • Observation 1.2 an object o cannot satisfy q if rqdoes not fully contain o.pcr(0.25) • Observation 2.2 an object o cannot satisfy q if rqdoes not fully contain o.pcr(0.3) • Observation 3.2 an object o cannot satisfy q if rqdoes not fully contain o.cfbin(0.3)

  23. Comparing CFBs with PCRs • CFBs have weaker pruning/validating power than PCRs • But CFBs require less space than PCRs

  24. Finding Conservative Functional Boxes • goal: minimize • for the i th dimension, minimize with the following constrains: • Linear Programming: Simplex Method

  25. More in Our Paper • The U-tree a dynamic index designed to accelerate prob-range queries.

  26. Experimental Results • data space: [0, 10000]d • uncertainty region shape: circle (sphere) • uncertainty region radius: 250 • data set: • Long Beach County (LB): 53k 2D objects, uniform pdf • California (CA): 62k 2D objects, Gaussian pdf • Aircraft: 100k 3D objects, uniform pdf • query set: 100 queries for each data set with various sizes of rq and different pq

  27. Experimental Results

  28. Experimental Results Query performance vs. search region size (LB, pq = 0.6)

  29. Experimental Results Query performance vs. search region size (CA, pq = 0.6)

  30. Experimental Results Query performance vs. search region size on (Aircraft, pq = 0.6)

  31. Experimental Results Query performance vs. probability threshold on (LB, qs = 1500)

  32. Experimental Results Query performance vs. probability threshold on (CA, qs = 1500)

  33. Experimental Results Query performance vs. probability threshold on (Aircraft, qs = 1500)

  34. Summary • A fast method for answering probabilistic range search queries.

More Related