Evolving Folksonomies Complexity Theory & Folksonomies

1. Evolving Folksonomies Complexity Theory & Folksonomies Tom ReamyChief Knowledge Architect KAPS Group Knowledge Architecture Professional Services http://www.kapsgroup.com

2. Agenda • Introduction • What are Folksonomies? • Advantages and Disadvantages of Folksonomies • Complexity Theory and Folksonomies • Environment, Evolutionary Mechanisms • Intelligent Design: Universe of Discourse • Conclusion • Possible Scenario’s for Evolving Order • Research Directions • Benefits

3. KAPS Group • Knowledge Architecture Professional Services (KAPS) • Consulting, strategy recommendations • Knowledge architecture audits • Partners – Convera, Inxight, FAST, and others • Taxonomies: Enterprise, Marketing, Insurance, etc. • Taxonomy customization • Intellectual infrastructure for organizations • Knowledge organization, technology, people and processes • Search, content management, portals, collaboration, knowledge management, e-learning, etc.

4. What are Folksonomies? • Wikipedia: A folksonomy is an Internet-based information retrieval methodology consisting of collaboratively generated, open-ended labels that categorize content such as Web pages, online photographs, and Web links. • A folksonomy is most notably contrasted from a taxonomy – done by users, not professionals, • Example sites – Del.icio.us and Flickr (not really – no feedback) • It is just metadata that users add • Key – social mechanism for seeing other tags

5. Advantages of Folksonomies • Simple (no complex structure to learn) • No need to learn difficult formal classification system • Lower cost of categorization • Distributes cost of tagging over large population • Open ended – can respond quickly to changes • Quality – “compare favorably with professional”? • Relevance – SME generated, close to content • User’s own terms

6. Advantages of Folksonomies • Aboutness – qualitative judgments • Reflect user’s perspective • Multiple dimensions – “communities” of like minded taggers • Support serendipitous form of browsing • Easy to tag any object – photo, document, bookmark • Better than no tags at all

7. Disadvantages of Folksonomies • They don’t work very well – polysemy, synonyms, etc. • Focus on easy tagging, not finding • Compare favorably with no tags, not controlled vocabularies • No structure, no conceptual relationships • Flats lists do not a onomy make • Jargon – SME’s talking to themselves or each other • Multiple communities – different terms • SME’s are not info professional – different skill • Based on popularity only, no quality control

8. Disadvantages of Folksonomies • Issues of scale – popular tags already showing 10,000’s of hits • Limited applicability – only useful for non-technical or non-specialist domains • Either personal tags (other’s can’t find) or popularity tags – lose interesting terms (Power law distribution) • Errors – misspellings, single words or bad compounds, single use or idiosyncratic use • Wikipedia article – very shallow, “wrong”? – not a taxonomy at all

9. Complexity Theory (abridged)History • An interdisciplinary method • Applied to math, model systems, economics, ecology, etc. • Initial Hype Period – 1980’s-1990’s • Chaos theory, Catastrophe theory, AI, etc. • Current – half way between hype and practical • Beware articles that focus on one aspect – self-organizing • Santa Fe Institute, social research, our Keynote • The Center for Complex Systems Research

10. Complexity Theory (abridged)Examples • Complex Systems (not complicated) • Large number of independent relatively dumb elements interact according to a small set of rules. • Self-organizing – order emerges • Local rules, local interactions – global order emerges • Definition by Example • Ant Colonies – clear tunnels with no idea of how to clear a tunnel • Neighborhoods – create a structure with no central planning

11. Complexity Theory (abridged)Essential Features • Large numbers of elements • Local Interactions • Emergence – global from local • Feedback • Self-organization • Key idea – often over-hyped • Importance of the environment • Often overlooked

12. Complexity Theory and FolksonomiesIntelligent Design – Universe of Discourse • Complexity – need right level of structure and disorder • No evolution without: • Initial complex structure • Evolutionary mechanisms – feedback with consequences • Level of structure = value of order that emerges • Color clumps, ants, neighborhood stores • Need to pay attention to initial organization structures • - “taxonomies” and metadata • Communities – users and designers

13. Complexity Theory and FolksonomiesDesigning the world • Wikipedia – fast cheap encyclopedia • Not really just mass of workers making local decisions • Role of initial environment and sets of rules • Feedback everywhere - including on structure and rules • Still have designers – administrators, councils of administrators • Editor Team – like Wikipedia • Create the structured environment • Create the rules and feedback system • Tweak the evolution of the system • Analyze data, paths to monitor • Develop initial candidates – interviews, search log analysis, ethnographic studies

14. Complexity Theory and Folksonomies:Two domains – Internet and Internal – Intranet, databases • Internet – this is the normal domain for folksonomies • More content, more users/taggers • Wilder environment • Less specific targets – web sites, topical articles • Large general sites best, not specialty sites • Internal • More initial structure, more similar content • More resources for tagging • More rewards for categorization • Need authority – corporate policy • More precise targets

15. Complexity Theory and Folksonomies:Two domains of evolution • Structure of the folksonomies • Bottom up – create clusters – based on co-occuring terms, other, groups of people? • Social – have people categorize the tags and then have people rank the appropriateness of categories • One possibility – community based agreement – more people rank a category as good – they become a community • Appropriate tags on documents • Social - Wikipedia style – everyone can tag any document • Include evolutionary rank for taggers • Everyone can rank the appropriateness of tags • Develop rank of taggers – super-editors, editors, authors

16. Complexity Theory and Folksonomies:Evolutionary Mechanisms • Feedback with Consequences • If an ant fails to follow a rule, it dies • Set up minimum number of good category or good tag ranks by everyone – if below, it dies. • Death of tag/category – deleted or moved to unranked primordial goo • Filter to top – popularity, Tag Clouds – Del.ici.ous • Very high level – “Blog, photography” • Mutation – keywords into other categories, categories into other categories • Success within a category – popularity, other criteria

17. Complexity Theory and Folksonomies Evolutionary Mechanisms • Ranking Methods • Explicit – people rank directly (roles) • Categories, tags, taggers • Good tags, best bets for terms or categories? • Implicit – software evaluation, reverse relevance • Who will rank? • Interested people, folksonomy advocates • Intranet – rewarded employees • Internet – community sites, aggregators, search engines • What will we call them? • Force of Nature, Deities, Intelligent Designers

18. Complexity Theory and Folksonomies Scenario One: Del.icio.us Plus • Social tags – only real criteria is popularity – Tag Cloud • Evolve quality of tags and emerging structure of tags • Preferred term = popular (Blog/blogs – Books/book) • Add broad general taxonomy of most popular tags • Tags as natural categories – build up and down • Add mechanisms – rank tags, taggers, categories • Flickr – facets are natural structure – date, people, events • Start – evolve a simple 2 level taxonomy • People assign tags to a category, build numbers • Evolutionary phase shift – spasm of structure

19. Popularity Structure of FolksonomyGood, Bad, and Natural

20. Complexity Theory and Folksonomies Scenario Two – Intranet/Content Aggregators • Buy/Build starter taxonomies – more structure than Del.icio.us • Create a team of designers/rules/mechanisms • Develop reward structures • Feedback – about tags – from employees or customers • Feedback about taggers, categories – from central team with input from selected SME’s. • Add evolution to best bets – compete with management selected

21. Complexity Theory and FolksonomiesResearch and Theory • Research Ideas • Uncover the effects of “interwingledness” • Monitor how people tag (and categorize) – historical patterns • Who would do this? Institute? Flickr? Other commercial? • Design new metrics and reports? • Discover natural category levels • Take terms as candidates for natural level • Build structure up and down • Apply communities – different natural levels • Simplicity – expose taxonomy at natural level - keywords

22. Conclusions: Evolutionary Model • Need to design a complex system, not complicated or free form • Different design for four domains: Internet-Internal, Categories-Tags • Advanced feedback is necessary • Editor Function is necessary • Develop infrastructure, analyze feedback, facilitate • Order is grown – from a combination of bottom up and design • Management is suggesting rules and testing and gathering feedback about usefulness, not dictating correct terms

23. Complexity Theory and FolksonomiesBenefits • Add the onomy to folk • More structure at low cost • Benefits of research • Investigation itself yields ideas • Internal Domain – supplemented by traditional methods, taxonomies, controlled vocabularies • If it fails, at least could kill the term, foksonomy (It’s metadata)

24. Questions? Tom Reamytomr@kapsgroup.com KAPS Group Knowledge Architecture Professional Services http://www.kapsgroup.com