Measuring Information Architecture Quality

1. Measuring Information Architecture Quality The Nebulous Science Nick Ragouzis, Enosis Group nickr@enosis.com ACM SIGCHI, CHI2001 Seattle, Washington April, 2001 (Extended version)

2. Why are we asking about measuring IA quality? • Basis: • Not: about the research/science domain • Rather: about commercial/applied domain • In significant, challenging, environments • Important, relevant? • What do we mean? To what extent? • Assumptions, Causes, Consequences? • Or is it merely symptomatic … of IA’s lost place? April 2001, Nick Ragouzis, nickr@enosis.com

3. Information will have an Architecture • Quality is an attribute of value • An IA is not intrinsically valuable • Value, in terms of • Creation, continuity • Users; time, often money • Relative to competing forces April 2001, Nick Ragouzis, nickr@enosis.com

4. Just Wasted Time? • A sea of assumptions • High dimensionality • High volatility • … in aspects, degrees, time frames, etc. • Quality of one IA over another • Just another design job • Good-enough: Just favor one of several IA • Assumptions more directly addressed by increasing and sifting focus to innovation April 2001, Nick Ragouzis, nickr@enosis.com

5. The IA, Defined • Information Architect [1996, noun, archaic]: • Theindividual who organizes … the patterns inherent in data … making the complex clear • Aperson who creates … structure or map … allows others … personal path to knowledge • The 21st century professional occupation addressing needs of the age [, who is] focused on • clarity, • human understanding and • the science of the organization of information (Editorialized from Richard Saul Wurman) April 2001, Nick Ragouzis, nickr@enosis.com

6. Information Architect’s Main Job (Editorialized from Rosenfeld & Morville) April 2001, Nick Ragouzis, nickr@enosis.com

7. IA, Re-Defined • Information Architect [2001, verb]: • A facet of the capabilities and responsibilities of the professionals participating in ainterdisciplinary collaborative design system • Executive committee, Marketing, Sales, Strategic planning, OD/Comm, Finance, Independent divisions and departments, Product management, IS management, Project management, interaction designers, experience designers, graphics designers, programmers … • … and all other interested parties, constituents, and users directly or by convenient proxy and surrogate. April 2001, Nick Ragouzis, nickr@enosis.com

8. Information Architect’s Main Job? April 2001, Nick Ragouzis, nickr@enosis.com

9. 1996 Only an Information Architect could know what’s best 2001 Just another design job IA-in-a-box Automatic; Prescription Heard throughout the interdisciplinary team: “I’ll do an info architecture on that by Friday.” Learn what a [website] customer wants and direct them to it April 2001, Nick Ragouzis, nickr@enosis.com

10. Meanwhile … Our “Users” evolve: Parker, Day, The Dawn of Man, Burlington, 1992 wf.carleton.ca/Museum/man/evnman.html April 2001, Nick Ragouzis, nickr@enosis.com

11. Users as People • Social-Psychological, Arousal • Reeves & Nass • Persuasion • Captology, BJ Fogg et al • CHI2001 Website Credibility • Trust • CHI2001 Relational Agents • Choice • Surfing Regularities, Information Scent (CHI2001) • Huberman, Pirolli, Card, et al, April 2001, Nick Ragouzis, nickr@enosis.com

12. The Main Jobof an Interdisciplinary, Collaborative, Design System Innovate, or follow strategically April 2001, Nick Ragouzis, nickr@enosis.com

13. Innovation is? • Which of these flashy, money-hogging, schedule-overrunning, customer-frustrating features would you like to add to your website project today? • Perhaps you also have some of your own to suggest? I’d prefer we spend the money on a sweepstakes… April 2001, Nick Ragouzis, nickr@enosis.com

14. Innovation is? • Do?, When?; Which combination? • Both in basic navigation & interaction structure, and so-called value-adding services e.g., personalization, chat, real-time svcs • Risks in investment, and not • Wasted time and money on one, another, or more? • Missed something crucial today, next week? Back button as opportunity? April 2001, Nick Ragouzis, nickr@enosis.com

15. Innovating Strategically • Deliver verifiable improvements in the customer’s perceived value That’s for more than just the website or whatever, right? April 2001, Nick Ragouzis, nickr@enosis.com

16. Following Strategically • Choose, in order to maximize: • Differentiation from most serious rivals • Affinities with competitors and partners having similar differentiation requirements • Competitive power of the explicit or tacit market group to which the follower belongs • De facto considerations, where differentiation isn’t relatively valuable • Follower-specific pathologies April 2001, Nick Ragouzis, nickr@enosis.com

17. A call from the past … April 2001, Nick Ragouzis, nickr@enosis.com

18. How much innovation? “In this age companies have lostthe privilege of standing still.”† relative to rivals †1962, Philip Kotler, Marketing Management April 2001, Nick Ragouzis, nickr@enosis.com

19. Create customers†more efficiently‡than your important rivals †1962, Peter Drucker, “The first task of a company … create customers.” although my meaning in this presentation is not limited to commercial companies and their consumers. Neither do I interpret Drucker as intending such limitation. ‡Efficiency in its fully-strategic implications April 2001, Nick Ragouzis, nickr@enosis.com

20. Customer-Oriented Focus † Benefits to us: • Customer needs are more basic than particular offerings • Attention to customer needs helps us spot new opportunities more quickly • In return we are more effective at satisfying customers • In return we bring our own interests into greater harmony with society’s interests. †1957, Wendell Smith April 2001, Nick Ragouzis, nickr@enosis.com

21. In other words… We must apply out creativeness more intelligently to people, and their wants and needs, than to products. † †1959, Charles Mortimer, italics in original April 2001, Nick Ragouzis, nickr@enosis.com

22. A Change? Simple Growth? Basic metric of innovation Compound rates of growth …early, and sustained • EBIT, NCF/CAO • Return on reinvestment of customer’s investment (their time, operations on their financial gains) matched to customer’s ability to further benefit April 2001, Nick Ragouzis, nickr@enosis.com

23. Efficiently creating customersRelative advantage to rivals Not So Good Much Better Requires sustained above-average performance April 2001, Nick Ragouzis, nickr@enosis.com

24. The sounds of average performance • Parity, more or less • Do what 90% of other companies do • Implement de facto practices • Follow 75%-likely recommendations • Improved conversion rates on launch and… • The quality of an information architecture The Average Surfer? April 2001, Nick Ragouzis, nickr@enosis.com

25. Choosing: a wealth of opportunities • Profile of • Customers perceived value mechanisms and • Value perception history; • Including vis-à-vis important rivals • Real-options-based valuation • Achievable rates of growth in perceived value • Achievable relative advantage to rivals • Volatility: in opportunities, actions by rivals, own organization’s ability to perform • Information returns to organization’s systems • Including in (re)designing the design systems April 2001, Nick Ragouzis, nickr@enosis.com

26. Choosing; a wealth of opportunities • Synergy Effects, Interdependence • Extended View of Opportunities • Flexibility in delay • Multi-staged, mixed • Multi-pronged, with late run-off • Wait-and-see; emphasis on See • Flexibility to abandon • Preemption; establishing, responding • Restructuring rivalries; switching barriers • Unilateral alliances April 2001, Nick Ragouzis, nickr@enosis.com

27. Summary • Justifying IA? It’s the wrong question • Times have changed • What was IA is, rather, a design job • For the organization’s entire design system • Deliver verifiable improvements in the customer’s perceived value • Innovate or follow, strategically • Do so persistently; rates of growth, compounded • The fittest wins • Beware short-term, false, maxima! April 2001, Nick Ragouzis, nickr@enosis.com

28. Thank You Nick Ragouzis nickr@enosis.com April 2001, Nick Ragouzis, nickr@enosis.com

29. ----Further Notes---- April 2001, Nick Ragouzis, nickr@enosis.com

30. But IA is important! • Measurements we would have seen: • Depletion/decay of a info architecture’s integrity • Average onset time • Rate of undone effort, investments • Remediation, costs, effort, lost opportunity • Rival-comparison of an architecture’s effectiveness • Consistent maps establishing dependency of results on features, activities, or responsibilities • Half-life of an information architect’s efforts • Factor for valuation and volatility for IA’s efforts in overall project valuations April 2001, Nick Ragouzis, nickr@enosis.com

31. But we need the name! • Fine • In some domains • But in commercial applied domain • It will inevitably fall into use as a verb • There, it’s superfluous • Interdisciplinary, collaborative, “zone” approach • Project managers have more influence on an IA Is the name the problem? April 2001, Nick Ragouzis, nickr@enosis.com

32. Inspecting the false parallel of “Architect” in IA Structure, function, utility vs Experience Design & usability analysis vs Perceived value analysis Construction engineers vs Architects Information architects vs Experience/Interaction designers Return to: We Need The Name! April 2001, Nick Ragouzis, nickr@enosis.com

33. The IA as Bridge From Research to Applied Domain • Shaping the Design System • Empirically disconfirming propositions • Challenging the status quo • Harvesting information gain • Options on further innovation • Options on organization change Design Research Reconciliation April 2001, Nick Ragouzis, nickr@enosis.com

34. Aspects of an IA’s Role as Bridge† • Problem Driven • Client Centered • Challenges the Status Quo • Presents Empirically-Disconfirmable Propositions • Systems Theory-based • Usable in Everyday Life † Loosely Adapted from Action Science: Lewin, Susman, Argryis, … April 2001, Nick Ragouzis, nickr@enosis.com

35. Damaged Merchandise? • Teaming the academic and the industrial • Addressing the limits of academic peer review in cross-disciplinary environments • Addressing the poor quality of methodology coupled with the naïve claims of validity • Improving the ability of those working throughout the design continuum to make valid inferences Gray and Salzman, HCI 1998, 13(3) Return to IA as Bridge April 2001, Nick Ragouzis, nickr@enosis.com

36. Center for Design Research Reconciliation “Research on such real-world problems is difficult, and this is probably an ideal problem to be tackled by academic- industrial partnerships” Olson, Moran, introductory editorial to HCI 1998 13(3) April 2001, Nick Ragouzis, nickr@enosis.com

37. Center for Design Research Reconciliation • Ground other’s past and newly published work in other’s past and current (and future) media and design disciplines • To relate the published research to business agenda and personal experience agenda • To reconcile conflicting or disjoint work • To help raise the level of understanding of the qualities in research that is most helpful and productive • To help raise understanding in organizations regarding what they can do to incorporate such research into their design systems April 2001, Nick Ragouzis, nickr@enosis.com

38. Center for Design Research Reconciliation • Enlist sponsors: Corporations (clients, systems houses, developers, researchers, etc.) academe, associations, and foundations. From many disciplines and having many interests. • Guide a grant process following a community-developed agenda • Integrate researchers (basic and applied) and practitioners into the effort • Hold in appropriate escrow the “sacred” data • Explore and evolve standards for the work • NOT doing the resulting peer-review publishing; CDRR-supported research published as grantees desire plus appropriate CDRR-supported releases. Return to IA as Bridge April 2001, Nick Ragouzis, nickr@enosis.com

39. Backtracking as Success • Strength of the “prior” page relative to followers • Power of exploratory enticement • User confidence in space, in performance • Strength of “home base” • Who has investigated user-set “checkpoints” in browser history path pruning services Return to Innovation is? April 2001, Nick Ragouzis, nickr@enosis.com

40. Crucial, Yet Rarely Incorporated … in design … in applying, extending results … in selecting subjects Mode: P(1.5 clicks) = .3 …so most common experience is 1 click Mean: 2.98 clicks; σ = 2.06 …but by then almost 2/3 are gone Strong Regularities in WWW Surfing, Huberman et al, Science V280, 3 April 1998, pp. 95-97; Case: AOL Return to Average Performance April 2001, Nick Ragouzis, nickr@enosis.com

41. The unexpected may happen: NASDAQ Composite Index April 2001, Nick Ragouzis, nickr@enosis.com