Activity-Based Serendipitous Recommendations with the Magitti Mobile Leisure Guide

1. Activity-Based Serendipitous Recommendations with the Magitti Mobile Leisure Guide V. Bellotti, B. Begole, et al. CHI 2008 Proceedings, pp. 1157-1166

2. Motivation & Introduction • Motivation • Traditional city guide • “Time Out” in London and New York, and “Tokyo Walker” in Tokyo • Location-based services • Search for local restaurants, movies, stores and so on • Discovery of activities and venues in context-aware computing • Magittiproject • Sponsored by Dai Nippon Printing Co., Ltd. (DNP) • DNP, one of Japan’s largest printing companies • Development of a service to replace printed city guides • An activity-centered mobile leisure-time guide • Delivering timely and personally relevant recommendations about nearby venues • Predicting future activity based on the user’s context and models of past behavior • Target • People : 19~25 year-olds • Locations : Japanese cities that have so many venues

3. Understanding Leisure Time Priorities • Dearth of English literature on Japanese leisure time activities • Previous time-based survey → too coarse activity for specific recommendation • Field exercises for following questions • How do young Japanese spend their leisure time? • What resources do they use to support leisure time? • What needs exist for additional support by a new kind of media technology? • Methods for field exercises • Interviews and Mockups (IM) • 20 semi-structured interviews with 16~33 year olds • 12 interviews with 19~25 year olds • Online Survey • A survey on a market research web site to get statistical information • 699 responses from 19~25 year olds • Focus Groups • 6~10 participants for each group • Presentation about a walkthrough of the Magitti mock-up and its functions • Gathering detailed feedback on the concept

4. Understanding Leisure Time Priorities • Methods for field exercises • Mobile Phone Diaries (MPD) • Daily activities of 19~25 year olds • Two mobile phone diary studies • First study → 12 people for one Sunday • Second study → 21 participants for a seven-day week • Street Activity Sampling (SAS) • 367 short interviews with people in target age range • Reporting three activities from their day • Choosing one as a focal activity • Classifying the activity into one of a number of pre-determined types • Expert Interviews • Three experts on the youth market in the publishing industry • Information commonly published to inform and support their activities • Informal observation • Observing young adults in popular Tokyo neighborhoods at leisure

5. Critical Findings from Field Exercises • How young people in Tokyo spend their leisure time ? • Shopping >going out with friends > dining out > going on a date > doing sport • Activity frequency in SAS interviews • Dining (31.8%), shopping (24.6%), browse/explore/look (7.5%) • Dining and shopping are major activities that involve going out • What resources are used to support leisure time ? • Friends and family, TV, Internet, and Magazines • Online survey respondents → Internet • IM interviewees → Friends and family • Information based on personal experiences of friends and family • Most trusted but not very extensive • What needs exist for additional support ? • 58.8% of SAS interviewees : interest for more information to support focal activity • Requests for information • Maps and venue locations (14.6%) • Customers’ and friends’ opinions (8.2%) • Prices (7.8%) • Store/venue contents (6.8%)

6. Design Requirements • Relaxation, Serendipity and Spontaneity • Relaxation • Busy schedules, often with multiple occupations (e.g., student and part-time-worker) • Serendipity • Attraction to serendipitous information • Avoidance of Information Overload • Reducing leisure information to only the most relevant • Minimal size • Particular preference of the younger generations • As small as possible in a pocket • One-handed operation • Strong requirement for one-handed operation by interviewees • Focusing on relaxation, serendipity, and spontaneity • Generating recommendations automatically using activity inference

7. Magitti Design • Magitti • Context filtering to reduce overload of leisure time in dense urban areas • No requirement of explicit definition of a user’s profile or preferences • Inference of interests and activities from learned models • Using data such as places visited, web browsing, and communications with friends • Magitti’s three key features • Context Awareness • Using current time, location, weather, store hours, and user patterns • Activity Awareness • User’s inferred or specified activity based recommendation • Eating, Shopping, Seeing, Doing, or Reading • Serendipitous, relaxing experience • Not necessary for profile, preferences, or queries • Activity inference for Magitti using context

8. Related Activity-Detection Research • Lamming and Newman’s activity-based information-retrieval system • One early system related to Magitti • Presentation of information that was generated in contexts • Impossible to infer activity with effective accuracy • Other activity detection approaches • Begole et al. : sensor-based availability detection • Inference of human activities from use of objects with RFID tags • Inference of human activities by using video and audio data analysis • Froehlich et al. : finding correlations between place preference and data • Activity modeling research • Liao et al. using location-based sensing with Relational Markov Networks • ‘AtHome’, ‘AtWork’, ‘Shopping’, ‘DiningOut’, and ‘Visiting’ • Previous works • Detection of a person’s current activity • Magitti guide system : predicting a person’s future activities

9. Related Mobile City Guide Applications • Location-based information recommendation system • Similar in spirit to Magitti: location-aware tourist guides • Some systems to recommend venues based on the user’s state • No prediction of the user’s activities • Cyberguide • A mobile tourist guide for the Georgia Tech campus • Awareness of its time, location, and history • Matching information on venues and special events to the data • MobyRec • A context-aware mobile tourist recommender system • Hotels, restaurants, etc. • Improvement of recommendations over time

10. Related Mobile City Guide Applications • GUIDE • Providing tour routes and accesses ticket reservation services • Dynamically recomputing routes based on location and time • Targeting for touring unfamiliar areas • COMPASS • A tourist guide service covering a wide range of venue types • Using profile and goal information entered by its user • Using location, speed, user profile, schedule, shopping list, and recent visit • Filtering by the user’s stated goal and preferences • CRUMPET • Providing tips, tour suggestions, maps and other information on a range of tourist-related venues (restaurants, movies, shows, etc.) • Learning user preferences over time

11. Magitti : User Interface • Main Screen • A scrollable list of up to 20 recommended items in Main Screen • Matching the user’s current situation and profile • Automatic list update to show items relevant to new locations • Detail Screen • Viewing Detail Screen by tapping each recommendation • Initial texts of a description, a formal review, and user comments • Rating the item on a 5-star scale by a user

12. Magitti : User Interface • Partial map on the Main Screen • Showing the four items currently visible in the list • Minimal size and one-handed operation requirements • Large buttons on the screen to enable the user to operate Magitti with a thumb • Marking menus on touch screens to operate the interface • Menu buttons at the bottom of the Main Screen • Adjusting the recommendation list if needed • Five modes of user activity; Eat, Buy, See, Do, or Read • Recommendations from just one category • Bookmarking recommended items

13. Magitti : System Architecture • Client-server architecture • Mobile client UI on a handheld device • Providing data for the Context Sensing Module • Gathering data about user’s physical context and data context • User’s physical context • GPS, time of day, user inputs, weather • Data context • Content of emails sent/received, calendar, web pages and documents viewed, applications used

14. Magitti : Activity Prediction Module • Data for probabilistic modeling • Using data collected on Magitti’s target demographic in the fieldwork • Japanese Survey on Time Use and Leisure Activities • Modeling the frequency of each mode by tracking user behavior • Visiting a retail store → Buy • Visiting a restaurant or café → Eat • Visiting theater or museum → See • Gym or park → Do • Reading of content on Magitti itself → Read

15. Magitti : Recommender System • Computation of the utility of each content item • Combining results from a variety of recommendation models • After computing scores of all items, top results are allocated in the slot • Computing score for an item in Magitti • Combination of Eight Model

16. Computing score for an item in Magitti • Collaborative filtering • Computing similarities between users • Determining scores each item based on how other similar users rated it • Stated Preferences • Scoring items according to how closely they match the user’s stated preferences • Learned Preferences • Learning from observed behavior rather than explicitly stated preferences • Content preference • Measuring the similarity of an item’s content to a user’s profile • Distance • Items within a distance range (either entered or inferred from location traces) • Reading • Using a model of users from the fieldwork • Boredom Buster • Reducing scores of items that have previously been seen • Future Plans • Temporarily raising scores based on future plans derived from the Content Analysis

17. Data Context Detection • Detecting the user’s physical context • Calendar appointments, viewed documents, and messages to extract information about the user’s plans • Leisure activity plans with friends using mobile email and SMS • Test for the potential usefulness of SMS • 10,000 SMS messages by students at the National University of Singapore • 11% of the messages related to leisure activities • Prototype Content Analysis module • Only Eat and See activity planning • Other activities planned for future work

18. Field Evaluation • 11 volunteers with Magitti in the Palo Alto, California area between one and four times each over several days • Participants, who were company employees not working on the project, ranged in age from mid-20s to late-50s, and averaged 37 • Visiting a total of 60 places over 32 outings, averaging 1.9 places per outing. • About half the outings (16) accompanied by a family member or friend

19. Supporting Serendipity • Try to find a new place(such as restaurant) • Very successful at discovering new places • Over half new places (53%) • Including 38% that they had never heard of • Including 15% they had heard of but never been to • Places visited once or twice (25%) • Places visited many times (23%) • People’s expression about finding new places • “Cool! I like that. I would never have found that place if it wasn't for this.” • “I think it makes life more interesting. It allows you to get out of your daily routine, almost as if you’re going to a different city.” • Magitti’s overall usefulness • 4.1 on a scale of 1-5 (5=very helpful) • Useful for residents and not just tourists or newcomers

20. Predicting User Activity • User activities in the experiment • Visiting 30 places to Eat, 27 to Buy, and 3 to Do • Some most frequent activities • Changing activity type : an average of 5.1 times per outing • Eat (1.8 times per outing), Buy (1.4), Do (0.7), See (0.5), and Read (0.1) • “Any” mode : an average of 0.7 times per outing • Wrong inference : easy to switch to a different activity

21. Context-Aware Recommendations • Relevant and interesting recommendations • Average rating of 3.8 (1=rarely, 5=almost always) • A little less than “usually” • A person’s opinion • “Most of the time, the list contained a mix of useful and not so useful recommendations” • Several factors that affected people’s confidence in the system • Omission • “the list did not represent what downtown has to offer” • Small omissions or inaccuracies reduced people’s trust • Distance • People expect that the closest places would be at the top of the list • Poor recommendation if it required driving • First Item • More weight on the first item recommended • Reasonable first item → good recommendations • Guide vs. Recommender • Relatively less loss of confidence for recommendation of a closed place • Some people : location information guide (closest place) • Other people : recommender (similar place)

22. Context-Aware Recommendations • Several factors that affected people’s confidence in the system • Transparency • Some users try to understand how Magitti decided which activities and venues to list • A complex set of algorithms based on many factors • Location, time, preferences, similar users’ opinions, prior behavior • Lack of transparency of the algorithm • sometimes confusing or even frustrating users • Need for offering more cues to help users develop an appropriate user model

23. Issues & Conclusion • User Control • Desire to have more control in managing the recommendation list • Ability to sort the items by factors such as rating, price, or distance • Ability to remove items from the list • Social Use • Outings involved two or more people • Incorporation into a social setting • Conclusion • Predicting the user’s current and future leisure activity • Modeling the user’s preferences, to filter and recommend relevant content • An interface with a novel one-handed, thumb based interaction