Thoughts on Tagging & Search

1. Thoughts on Tagging & Search Marti Hearst UC Berkeley

2. Talk Outline: Two Main Points • Massive user behavior is aiding search algorithms in interesting ways. • Going deeper: An examination of social tagging: • The controversy • Research questions • Our work on automating creation of metadata structure

3. User-contributed content is exploding!

4. Social Information & Search • Trend: human behavioral information is getting “baked in” to search algorithms. • In many cases, the actions of the many is more useful than the actions of the individual. • Three examples follow.

5. Actions of the Many vs. Individual • Anchor text for improved ranking. • vs author-supplied meta-tags

6. Actions of the Many vs. Individual • “Clickthrough” to improve ranking. • vs. an individual’s prior clicks • Joachims et al. and Agichtein et al. found that human selections of links from search results could improve rankings for popular queries. • Some surprising rules: • Assign negative weight to an unclicked link that appears above and below a clicked link

7. Actions of the Many vs. the Individual • Query auto-suggest based on other users’ queries • vs based on one one’s prior queries alone

8. Social Tagging • Metadata assignment without all the bother • Spontaneous, easy, low cognitive overhead • Usually used in the context of social media

9. Popular pages on del.icio.us

10. Visitor tagging at Powerhouse Museum

11. Tagging is Controversial! • Power to the people! • Easy! • Cheap! • Sloppy! • Disorganized! • Incorrect!

12. Professional Cataloguer: “Everything I know isn't in the picture!” Investigating social tagging and folksonomy in the art museumwith steve.museum", J. Trant, B. Wyman, WWW 2006 Collaborative Tagging Workshop

13. The Tagging Opportunity • At last! Content-oriented metadata in the large! • Attempts at metadata standardization always end up with something like the Dublin Core • author, date, publisher, .... • I think the action is in the subject metadata, and have focused on how to navigate collections given such data.

14. The Tagging Opportunity • Tags are inherently faceted ! • Multiple labels are assigned to each item • Rather than placing them into a folder • Rather than placing them into a hierarchy • Concepts are assigned from many different content categories

15. See how this author attempts to compensate: Tagging Problems • The haphazard assignments lead to problems with • Synonymy • Homonymy • Unpredictability

16. Tagging Problems • Some tags are fleeting in meaning or too personal • toread todo • Tags don’t “cover” all the concepts • Tags are disorganized • Tags are not “professional” • (I personally don’t think this matters)

17. Research Questions for Tags & Search • How to improve tag convergence? • How to group tags meaningfully? How to eliminate uninteresting tags? • What is the role of user interface on tag convergence? • Preliminary evidence suggests there is a big effect • There are some good ideas out there • More experimentation is needed. • What algorithms can we use to clean up the tags after they are assigned? • There is some work here, much more can be done. • TagAssist: Automatic Tag Suggestion for Blog Posts, Sood et al., ICWSM 2007

18. Interface for adding tags on del.icio.us

19. Effects of Interface On the Structure, Properties and Utility of Internal Corporate Blogs,Kolari et al. ICWSM 2007

20. Research Questions for Tags & Search How to get tag expertise? Who will identify the plant species in this image? office desk plants windows shadows

21. Research Questions for Tags & Search • What is the relationship of social tags to automated content extraction? • Are tags more informative, or differently informative, than other labeling methods?

22. Research Questions for Tags & Society • What motivates people to tag? • Who owns the tags? • Privacy and sharing of tags?

23. Research Questions for Tags & Search • How to use tags for browsing / navigation? • Currently most tags are used as a direct index into items • Click on tag, see items assigned to it, end of story • Grouping into small hierarchies is not usually done • del.icio.us now has bundles, but navigation isn’t good • IBM’s dogear and RawSugar come the closest • One solution: organize tags into faceted hierarchies, use faceted navigation.

24. Faceted Metadata & Navigation

25. The Idea of Faceted Metadata • Create INDEPENDENT categories (facets) • Each facet has labels (sometimes arranged in a hierarchy) • Assign labels from the facets to every item • Example: recipe collection Ingredient Cooking Method Chicken Stir-fry Bell Pepper Curry Course Cuisine Main Course Thai

26. Faceted Navigation • A flexible, dynamic way to allow everyday users browse & search large information collections. • We’ve been investigating and promoting this at UCB since 1999. • It’s now widely used for e-commerce sites; • Digital libraries, image collections, etc., are following. • Search verticals as well • Google co-op • More info: flamenco.berkeley.edu Next Generation Web Search: Setting Our Sites, M. Hearst, IEEE Data Engineering Bulletin, Special issue on Next Generation Web Search, Sept. 2000

27. Faceted Nav in eBay Express

28. Faceted Nav in Digital Libraries • NCSU has a start at it

29. Faceted Nav in eBay Express

31. Facets in Google Co-op

32. Advantages of Faceted Navigation • Gives users control and flexibility • Can’t end up with empty results sets • (except with keyword search) • Helps avoid feelings of being lost. • Easier to explore the collection. • Helps users infer what kinds of things are in the collection. • Evokes a feeling of “browsing the shelves” • Is preferred over standard search for collection browsing in usability studies. • (Interface must be designed properly)

33. Advantages of Faceted Metadata • Helps alleviate the metadata wars: • Allows for both splitters and lumpers • Is this a bird or a robin • Doesn’t matter, you can do both! • Allows for differing organizational views • Does NASCAR go under sports or entertainment? • Doesn’t matter, you can do both!

34. Our Approach: Castanet (Stoica, Hearst, & Merichar, HLT-NAACL ’07) How to Create Facet Hierarchies?

35. Example: Recipes (3500 docs)

36. Castanet Output (shown in Flamenco)

37. Castanet Output (shown in Flamenco)

38. Example: Biology Journal TitlesCastanet Output (shown in Flamenco)

39. Build tree Compress tree Select terms Get hypernym paths WordNet Divide into facets Castanet Algorithm • Leverage the structure of WordNet Documents

40. Will Castanet Work on Tags? • Class project by Simon King and Jeff Towle, 2004 • 1650 captions captured from mobile phones • Wanted to organize them. • Used the CastaNet algorithm • Had to first remove proper names

41. Example Photos & Captions (King & Towle) very scary x-mas tree Hp presentation chasing a cat in the dark My cat

42. instrumentality, (112) vehicle (26) car (9) bike (8) vessel, watercraft (4) mayflower (2) ferry (1) gig (1) truck (3) airplane (2) device (20) machine (7) computer (4) laptop (1) sander (1) container (16) vessel (7) bottle (5) water_bottle (2) jug (1) pill_bottle (1) bath (2) bowl (1) can (2) backpack (1) bumper (1) empty (1) salt_shaker (1) furniture, piece of furniture, article of furniture (12) seat (8) bench (2) chair (2) couch (2) lounge (1) bed (4) desk (1)

43. Conclusions • The actions of the many are a boon for improving search algorithms. • Social tagging is, in my view, a terrific way to get good content metadata. • I think automated techniques can do a lot to help clean them up and organize them. • They are an inherently social phenomenon, part of social media, which is a really exciting area.

44. For more information: flamenco.berkeley.edu Thank you!