Ad hoc data integration for mobile GI S a pplications

1. Ad hoc data integrationfor mobile GIS applications Ramya Venkateswaran (ramya@geo.uzh.ch)

2. Contents • Scenario • Research Objective • Introduction: Overview of the GenW2 project • Motivation: Why is Ad hoc Data Integration needed? • State of the Art • Research Questions: Discuss 3 research questions • Methods: TourGuide and friends • Next Steps:Data Enrichment and Quality control

3. Scenario 1

4. Scenario of Usage I will be vacationing in Paris and I want to visit some of the famous palaces, History related places and other tourist locations in Paris Tourist & Travel Websites ? Other Sources Recommendations from Albums & Images People Tourist Guides

5. I’d still like to go to Paris.. Scenario of Usage Tourguide ? Other Sources Recommendationsfrom Tourist & Travel Websites Albums & Images People Tourist Guides

6. Research Objective 2

7. Objective of my research Ad hoc Data Integration • Data quality control • Completeness • Correctness • Credibility • User feedback • Data Integration • Flavour Based integration • Ad hoc DI vs. Traditional DI • TourGuide • Data enrichment • POI Enrichment • Website credibility

8. Overview and Introduction 3

9. Overview of the GenW2 Project • Short for: Generalization for portrayal in Web and Wireless mapping • Develop new methods for web and wireless mapping • Focus on • ad hoc integration of heterogeneous information • on-the-fly map generalization in a mobile context.

10. The GenW2 Framework

11. The GenW2 Framework

12. The GenW2 Framework

13. Types of Data sources Web services Web pages Image metadata Static datasets MRDB Facts DB

14. Motivation - Why is Ad hoc Data Integration needed? 4

15. Motivation • So many data sources and so little structure • Web as a database – Too much information to ignore! • Ad hoc integration – Need based according to scenario and flavour, unlike search engines. • Importance of recording certain facts that can enrich the MRDB and the integration process.

16. State of the art 5

17. Relevant Domains Information Filtering Information Retrieval Collaborative Filtering Recommendation Systems Ad hoc Data Integration

18. State of Art Ad hoc Data Integration • Data quality control • Completeness • Correctness • Credibility • User feedback • Data Integration • Flavour Based integration • Ad hoc DI vs. Traditional DI • TourGuide • Data enrichment • POI Enrichment • Website credibility

19. Integration, IR and decision systems • Different concepts and methods in Data Integration • Data Integration from multiple sources • Geospatial data mining and integration. (Knoblock et al. 2001, Michalowski et al., 2004) • Mashup web data for overall importance of landmarks. (Grabler et al., 2008) • SPIRIT – Design, techniques and implementation (Purves et al., 2007, Jones et al., 2002, Bucher et al., 2005) • Geo parsing, geo coding and IR techniques (Clough et al., 2005)

20. Integration, IR and decision systems • Methods for marking tourist locations and a guide that is 'context aware'. (Abowd et al., 2004) • Activity based model of decisions that are affected based on activity-travel behavior and also predict the activities. (Arentze and Timmermans, 2004) • Voluntary information from a community, collaborative semantics, recommendation systems (Schlieder , 2007)

21. Data Enrichment • Methods and algorithms for the provision of auxiliary data and its use for controlling an automated adaptive generalization process (Neun, 2007)

22. Data quality and assessment • Framework for efficient and accurate integration of geospatial data from a large number of sources • Positional accuracy, completeness (Thakker et al., 2007) • VGI (Volunteered Geographic Information) Trust models for Gazetteers (Keßler et al., 2009)

23. Observations from literature • Considerable work and methods for traditional data integration, variety of methods in IR and GIR • Lesser work and methods for data integration from multiple and dynamic sources (Focus on semantics rather than data and context) and recording reusable facts. • Considerable work on user modeling, activities and activity recommendation • Data enrichment work for improving generalization

24. Challenges • Datasets are not static and are dynamic and heterogeneous • Auxiliary data • Determining parameters (user categories, activities habits etc, not a single user or set of preferences) • Point of complete integration • Methods to test and evaluate the effectiveness

25. Research Questions ? 6

26. RQ1 – Flavour Based Integration • Given an activity and unrelated data that is heterogeneous and dynamic, what is an effective method of data integration, so that the results are streamlined towards information about events and places for a set of users? • Flavour based data integration from various sources • Ad hoc DI vs. Traditional DI • Tour guide – An example of web data integration

27. RQ2 – Data Enrichment • How can the Generalization for portrayal in Web and Wireless mapping (GenW2) framework record and exploit valuable reusable information, obtained from the preceding data integration? • Facts DB • Activity-Location pairs • Data source credibility (Keßler et al., 2009) • User feedback

28. RQ3 – Quality of data • What are the different metrics that can be used to control and/or assess the quality of the integrated data? • Measurement of Quality? • Quality of data by completeness (Thakkar et al., 2007) • Quality of data by correctness (Thakkar et al., 2007) • Another metric for Quality Assessment • Quality of data by collective user feedback • Credibility rank of information sources (Keßler et al., 2009) • Evaluation Methodology

29. Methods 7

30. Flavour Based Data Integration Information Filtering Information Retrieval Collaborative Filtering Recommendation Systems

31. Definition - Flavour Based Data Integration Information Filtering Information Retrieval Collaborative Filtering Recommendation Systems “a field of study designed for creating a systematic approach to extracting information that a particular person finds important from a larger stream of information” (Canavese, 1994). “the goal of an information [retrieval] system is for the user to obtain information from the knowledge resource which helps her/him in problem management” (Belkin, 1984) “use the opinions of a community of users to help individuals in that community more effectively identify content of interest from a potentially overwhelming set of choices” (Resnick and Varian 1997). “The central idea here is to base personalized recommendations for users on information obtained from other, ideally likeminded, users.” (Billsus and Pazzani, 1998).

32. Definition - Flavour Based Data Integration Information Filtering Information Retrieval Collaborative Filtering Recommendation Systems “a field of study designed for creating a systematic approach to extracting information that a particular person finds important from a larger stream of information” (Canavese, 1994). “the goal of an information [retrieval] system is for the user to obtain information from the knowledge resource which helps her/him in problem management” (Belkin, 1984) “use the opinions of a community of users to help individuals in that community more effectively identify content of interest from a potentially overwhelming set of choices” (Resnick and Varian 1997). “The central idea here is to base personalized recommendations for users on information obtained from other, ideally likeminded, users.” (Billsus and Pazzani, 1998).

33. Definition - Flavour Based Data Integration Information Filtering Information Retrieval Collaborative Filtering Recommendation Systems “a field of study designed for creating a systematic approach to extracting information that a particular person finds important from a larger stream of information” (Canavese, 1994). “the goal of an information [retrieval] system is for the user to obtain information from the knowledge resource which helps her/him in problem management” (Belkin, 1984) “use the opinions of a community of users to help individuals in that community more effectively identify content of interest from a potentially overwhelming set of choices” (Resnick and Varian 1997). “The central idea here is to base personalized recommendations for users on information obtained from other, ideally likeminded, users.” (Billsus and Pazzani, 1998).

34. Definition - Flavour Based Data Integration Information Filtering Information Retrieval Collaborative Filtering Recommendation Systems “a field of study designed for creating a systematic approach to extracting information that a particular person finds important from a larger stream of information” (Canavese, 1994). “the goal of an information [retrieval] system is for the user to obtain information from the knowledge resource which helps her/him in problem management” (Belkin, 1984) “use the opinions of a community of usersto help individuals in that community more effectively identify content of interest from a potentially overwhelming set of choices” (Resnick and Varian 1997). “The central idea here is to base personalized recommendations for users on information obtained from other, ideally likeminded, users.” (Billsus and Pazzani, 1998).

35. Definition - Flavour Based Data Integration Information Filtering Information Retrieval Collaborative Filtering Recommendation Systems “a field of study designed for creating a systematic approach to extracting information that a particular person finds important from a larger stream of information” (Canavese, 1994). “the goal of an information [retrieval] system is for the user to obtain information from the knowledge resource which helps her/him in problem management” (Belkin, 1984) “use the opinions of a community of users to help individuals in that community more effectively identify content of interest from a potentially overwhelming set of choices” (Resnick and Varian 1997). “The central idea here is to base personalized recommendations for users on information obtained from other, ideally likeminded, users.” (Billsus and Pazzani, 1998).

36. Flavour Based Data Integration Information Filtering Information Retrieval Collaborative Filtering Recommendation Systems “a field of study designed for creating a systematic approach to extracting information that a particular person finds important from a larger stream of information” (Canavese, 1994). “the goal of an information [retrieval] system is for the user to obtain information from the knowledge resource which helps her/him in problem management” (Belkin, 1984) “use the opinions of a community of usersto help individuals in that community more effectively identify content of interest from a potentially overwhelming set of choices” (Resnick and Varian 1997). “The central idea here is to base personalized recommendations for users on information obtained from other, ideally likeminded, users.” (Billsus and Pazzani, 1998).

37. Keyphrases in FBDI • Systematic approach to extracting information • Obtain information from one or many knowledge resource/s • Recommendations for user groups or user categories • Opinions of a community of users • Keyword, flavour or activity such as tourism, history, sport, culture, shopping etc

38. Definition of FBDI • FBDI is an activity based, systematic approach to extract and integrate information from multiple knowledge sources depending on habits of certain user groups or user categories, capable of learning over time. • Flavour = typical activities of a certain user group • Examples – Tourism, Shopping, Sports, Historical excursions, Cultural excursions etc

39. Demo Click me!

40. The GenW2 Framework

41. The GenW2 Framework

42. Adaptive tour guide for Paris • Flavour Based Integration with web as datasource • Only web as the database (Grabler et al.,2008 ) • Integration of data on • Tourism • Transport • User feedback • User Rating • Facebook profile • Dopplr profile • Scheduler

43. Data Integrator • Example of web data integration • Functional components (Baumgartner et al., 2009) • Web interaction component • Lonelyplanet, wikitravel, virtualtourist, tripadvisor and official tourist website • Wrapper generator • OpenKapowRobomaker • Data transformer • DOM parser for RSS and XML formats

44. The GenW2 Framework

45. Data Integrator • Example of web data integration • Functional components (Baumgartner et al., 2009) • Web interaction component • Lonelyplanet, wikitravel, virtualtourist, tripadvisor and official tourist website • Wrapper generator • OpenKapow Robomaker • Data transformer • DOM parser for RSS and XML formats

46. The GenW2 Framework

47. Web data Extraction • Semi automatic wrappers • Automatic wrapper Induction • WIEN (Kushmerick et al., 1997) • Stalker (Muslea et al., 2001) • DEBye (Laender et al., 2000) • Commercial • RoboMaker (Kapow Technologies) • WebQL(QL2 Software Inc.) • Academic • XWARP (Liu et al., 2000) • Lixto(Baumgartner et al., 2001) • Wargo(Pan et al., 2002)

48. Data Integrator • Example of web data integration • Functional components (Baumgartner et al., 2009) • Web interaction component • Lonelyplanet, wikitravel, virtualtourist, tripadvisor and official tourist website • Wrapper generator • OpenKapow Robomaker • Data transformer • DOM parser for RSS and XML formats

49. The GenW2 Framework

50. Data Integrator • Example of web data integration • Google as a first part of integration • Second Part - Functional components (Baumgartner et al., 2009) • Web interaction component • lonelyplanet, wikitravel, virtualtourist, tripadvisor and official tourist website • Wrapper generator • OpenKapow Robomaker • Data transformer • DOM parser for RSS and XML formats