Graduate seminar-2

1. Graduate seminar-2 Web structural delta mining Name: Kuchukulla Raghavender Reddy St id 0540554.

2. Previous presentation: • Web data mining: Data mining is the process of mining the data from the data set using different logical quarries. • Three types of data mining: • Web content mining • Web structure mining & • Web usage mining

3. Abstract : • With the progress of World Wide Web (WWW) technologies, more and more data are now available online for web users. The most of web data is simultaneously changes according to the certain situations, changes can be like deleting, updating or inserting the web pages. the information regarding the changes on web pages or web structures can be retrieved using the web structural delta mining.

4. Issues.. • Web structure mining has been a well researched area during recent years. Based on the observation that data on the web may change at any time in any way, some incremental data mining algorithms have been proposed to update the mining results with the corresponding changes. However, none of the existing web structure mining techniques is able to extract useful and hidden knowledge from the sequence of historical web structural changes.

5. Example.. Gray boxes deleted Black boxes newly inserted Bolded boxes updated

6. Issues.. • The existing web structure mining algorithms focus only on the in degree and out degree of web pages. They do not consider the global structural property of web documents. • Global properties such as the hierarchy structure, location of the web page among the whole web site and relations among ancestor and descendant pages are not considered.

7. First Invention.. • HITS: Computing Hubs and Authorities • The algorithm identifies only two kinds of web in web structure data they are • Hubs: • Pages with good sources of links • Authorities: • Pages with good sources of content

8. HITS • A good hub is a page that points to many good authorities; • A good authority is a page that is pointed to by many good hubs. • HITS associates a non-negative authority weight x<p> and. • A non-negative hub weight y<p>. • The weights of each type are normalized so that their squares sum to 1.

9. HITS : Figure 1: hubs and authority sites

10. HITS : Numerical Definition • if p points to many pages with large x-values, then it should receive a large y-value; if p is pointed to by many pages with large y-values, then it should receive a large x-value. • For Given weights x<p>,y<p>:

11. HITS :Drawbacks • Sometimes a set of documents on one host point to a single document on a second host, or sometimes a single document on one host point to a set of document on a second host. • The set of authoritative and hub pages computed at time t1 may change at time t2. That is some of the previously authoritative pages may not be authoritative any more. Similar cases may happen to hub pages. Thus, the mining results of the HITS algorithm may not be accurate and valid any more with the changes of web data.

12. Web structure delta mining • Related work: • HITS algorithm • Web Community Algorithms • community is one of the applications based on the analysis of similarity and relationship between web sites or web pages. The main idea of web community algorithms is to construct a community of web pages or web sites that share a common interest.

13. Related work • Change detection: According to format of web documents, web data change detection techniques can be classified into two categories. • HTML change detection. • XML change detection.

14. Related work • HTML change detection: • Currently, most of the existing web documents are in HTML format, which is designed for the displaying purpose. • An HTML document consists of markup tags and content data, where the markup tags are used to manipulate the representation of the content data.

15. Related work • The changes of HTML documents can be changes of the HTML markup tags or the content data. The changes can be sub page level or page level. • The AT&T Internet Difference Engine (AIDE) • It can detect changes of insertion and deletion.

16. Related work • WebCQ: • is a system for monitoring and delivering web information. It provides personalized services for notifying and displaying changes and summarizations of corresponding interested web pages.

17. Related work • Change Detection for XML Document: • XML documents are becoming more and more popular to store and exchange data in the web. Different techniques of detecting changes for XML documents are. • XyDiff technique. • is used to detect changes of ordered XML documents. It supports three types of changes: insertion, deletion and updating.

18. Web structure delta mining • Problem statement: • The goal of web structural delta mining is to extract any kind of interesting and useful information from the historical web structural changes. As the object of web structural delta mining can be structures of web sites, structures of a group of linked web page and even structures within individual web page, we introduce the term web object to define such objects.

19. Web structural delta mining Fig 2:Web structural delta mining

20. Web structure delta mining • Architecture of web structure delta mining

21. Web structure delta mining • Definition 1. • Let O={w1,w2……wn} can be a set of web pages. O is a web object if it satisfies any one of the following constraints: • 1) n=1; • 2) For any 1<=i <=n, wi links to or is linked by at least one of the pages from{w1,….wi-1,wi+1,…, wn}

22. Web structure delta mining • From the definition, we can see that a web object can be either an individual web page or a group of linked web pages. Thus, the structure of a web object O refers to the intra-structure within the web page if web object O includes only one web page, otherwise it refers to the inter-structure among web pages in this web object. The web object is defined in such a way that each web object corresponds to an instance of a semantic concept.

23. Web structure delta mining • With respect to the dynamic property of web data, we observed that some web pages or links might be inserted into or deleted from the web object and for individual web page the web page itself may also change over time. Consequently, the structure of a web object may also change. Our web structural delta mining is to analyze the historical structural changes of web objects.

24. Web structure delta mining • Definition 2 • Let (s1, s2, s3,….sni) be a sequence of historical web structural information about a web object, where Si is i-th version of the structural information about the web object O at time ti. (s1,s2,s3…sni) are in the order of time sequence. Assume that this series of structural information records all versions of structural information for a period of time. The objective of web structural delta mining is to extract structures with certain changes patterns, discover associations among structures in terms of their changes patterns, and classify structures based on the historical change patterns using various data mining techniques.

25. Web structure delta mining • According to the definition it incorporate the • Temporal: • Different versions of web structures as a sequence • Dynamic & • the changes between different versions of web structures • Hierarchical property of web structural data • hierarchy structures of web sites

26. Web structure delta mining • based on the dynamic metric, global metric and temporal metric, different types of interesting structures can be defined based on their historical change patterns. • Based on these definitions, the desired structures can be extracted from the sequence of historical web structural delta by using some data mining techniques.

27. Result.. • The algorithm extracts the historical data from the web page which is conceptual and in novel mode. • It also extracts the historical data from the single or individual web documents. • It can also retrieve the data from the invalid web pages.

28. References • Qiankun Zhao, Sourav S. Bhowmick, and Sanjay Madria. • 1) School of Computer Engineer, Nanyang Technological University, Singapore. qkzhao@pmail.ntu.edu.sg, assourav@ntu.edu.sg • 2) Department of Computer Science, University of Missouri-Rolla, USA. madrias@umr.edu • Arasu Arvind and Hector Garcia-Molina . Extracting structured data from web pages. • In The 2003 ACM SIGMOD International Conference on Management of Data, pages 337–348, 2003.

29. Thank you