1 / 21

Clustering Uncertain Data Items

Clustering Uncertain Data Items. TEAM 3. Outline. Problem Statement Our Contributions Description of Methods Comparisons Trends in clustering Future work Conclusions References. Problem Statement. In many applications data contains inherent uncertainty.

varick
Download Presentation

Clustering Uncertain Data Items

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Clustering Uncertain Data Items TEAM 3

  2. Outline • Problem Statement • Our Contributions • Description of Methods • Comparisons • Trends in clustering • Future work • Conclusions • References

  3. Problem Statement • In many applications data contains inherent uncertainty. • The problem of clustering data objects whose locations are uncertain are addressed here. • The goal of clustering is to group the objects into k-clusters such that total expected distance from objects to a representative point of their corresponding clusters is minimized.

  4. Our Contributions  The main classes of clustering algorithms for uncertain data are as follows: • Mixture-Modeling Algorithms • Density-based Methods • Partitional Methods • Streaming Algorithms • High Dimensional Algorithm

  5. Mixture model clustering • Mixture model clustering is a popular method for clustering of deterministic data, and it models the clusters in the underlying data in terms of a number of probabilistic parameters • A popular method in order to determine these model parameters is the EM algorithm, which uses an expectation maximization approach to iteratively update the parameters with the observed data instances.

  6. Density based methods • Density based methods are very popular in the deterministic clustering literature ,because of their ability to determine clusters of arbitrary shapes in underlying data. • Density based methods used here are FDBSCAN and FOPTICS

  7. Clustering uncertain data streams • The data may have uncertainty, because of errors in the readings of the underlying sensors. • This may result in uncertain streams of data. Uncertain streams pose of special challenge because of the dual complexity of high volume and data uncertainty.

  8. Uncertain data in High dimensionality • Main effect of uncertainty is the impact on the distance computations. • Different dimensions in the data can have very different levels of uncertainty. • In some cases, if the uncertainty data is not used in the mining process, this may result in a clustering which does not truly reflect the underlying behavior

  9. Clustering with the possible world • A general-purpose method for performing data analytics in such scenarios is to use Monte-Carlo sampling to generate different instantiations of the database, and then apply the algorithms to each sample. The final step is to merge these M different clustering into a single clustering.

  10. Clustering with the possible world • Many techniques can be used in order to perform the clustering: • It is possible to use the probabilities as the weights on the edges. However, such an approach does not explicitly account for the connectivity of the underlying network and its interaction with the combinatorial nature of underlying graph. • The possible worlds model has been used in order to perform the clustering. The edit distance is used on the underlying network in order to perform the clustering. • The problem of graph clustering is explicitly connected to the problem of sub graph reliability . The work determines methods for finding “reliable” sub graphs in uncertain Graphs. These sub graphs are those which are hard to disconnect, based on a combination of the combinatorial structure of the graph, and the edge uncertainty probabilities.

  11. Contrast and compare • We consider the algorithms categorized in four groups based on their clustering method: partitional, hierarchical, density-based and grid-based algorithms. • The type of the data that an algorithm supports (numerical, categorical), • The shape of clusters, • Ability to handle noise and outliers, • The clustering criterion and, • Complexity

  12. Characteristics of partitional clustering algorithm

  13. Characteristic of hierarchical clustering algorithm

  14. Characteristics of density based clustering algorithm

  15. Characteristics of grid based clustering algorithm

  16. Trends in clustering • Discovering and finding representatives of arbitrary shaped clusters • Non-point clustering • Handling uncertainty in the clustering process and visualization of results • Incremental clustering • Constraint-based clustering

  17. Future work • Future work includes uncertainty against various applications such as spatial queries, dominating queries, data streams in xml data, graph data ,statistical estimation, time series etc.

  18. Conclusions • The problem of clustering is significantly affected by the uncertainty, because different attributes may have different levels of uncertainty embedded in them. • This survey provides different algorithms for clustering uncertain data. Most of the conventional classes of deterministic algorithms such as mixture modeling, density based algorithms, partitioning algorithms, streaming algorithms and high dimensional algorithms have been extended to the case of uncertain data

  19. References • [1] C.C. Aggarwal, and P. S. Yu. A Survey of Uncertain Data Algorithms and Applications.IEEE Transactions on Knowledge and Data Engineering, 21(5), pp. 609–623, 2009. • [2] S. Abiteboul, P. Kanellakis, and G. Grahne. On the Representation and Querying of Sets of Possible Worlds. In ACM SIGMOD Conference, 1987. • [3] C. C. Aggarwal and P. S. Yu. A Framework for Clustering Uncertain Data Streams. In ICDE Conference, 2008. • [4] C. C. Aggarwal, C. Procopiuc, J. Wolf, P. Yu, and J.-S. Park. Fast Algorithms for Projected Clustering. In ACM SIGMOD Conference, 1999 • [5] C. C. Aggarwal, and P. S. Yu. Outlier Detection with Uncertain Data. In SDM Conference, 2008. • [6] C. C. Aggarwal. On High-Dimensional Projected Clustering of Uncertain Data Streams. In ICDE Conference, 2009. • [7] C. C. Aggarwal. Managing and Mining Uncertain Data, Springer, 2009. • [8] C. C. Aggarwal, J. Han, J. Wang, and P. Yu. A Framework for Clustering Evolving Data Streams. In VLDB Conference, 2003.

  20. References(Cont..) • [9] R. Agrawal, and R. Srikant. Privacy-Preserving Data Mining. In ACM SIGMOD Conference,2000. • [10] C. C. Aggarwal. On Density Based Transforms for Uncertain Data Mining. In ICDE Conference Proceedings, 2007. • [11] C. C. Aggarwal. On Unifying Privacy and Uncertain Data Models. In ICDE Conference Proceedings, 2008. • [12] M. Ankerst, M. M. Breunig, H.-P. Kriegel, and J. Sander. OPTICS: Ordering Points to Identify the Clustering Structure. In ACM SIGMOD Conference, 1999 • [13] N. Bansal, A. Blum, and S. Chawla. Correlation clustering, Machine Learning, vol. 56, no.1-3, pp. 89–113, 2004. • [14] D. Barbara, H. Garcia-Molina, and D. Porter. The management of probabilistic data. IEEE Transactions on Knowledge and Data Engineering, 4(5), pp. 487–502, 1992. • [15] D. Burdick, P. Deshpande, T. Jayram, R. Ramakrishnan, and S. Vaithyanathan. OLAP Over Uncertain and Imprecise Data. In VLDB Conference Proceedings, 2005.

More Related