How Do We Prove the Value of Museums?

1. How Do We Prove the Value of Museums? AAM annual meeting Friday, May 1, 2009 at 2:15pm Philadelphia, PA

2. Today’s Panel • Marsha Semmel, Deputy Director for Museums and Director for Strategic Partnerships, IMLS • Carol Scott, Renaissance London Programme Manager for 2012 • Jane Legget, Associate Director, New Zealand Tourism Research Institute at Auckland University of Technology • Barbara Soren, University of Toronto Museum Studies/Independent Consultant • Mamie Bittner, Deputy Director for Policy, Planning, Research and Communications, IMLS

3. Questions to Focus our Thoughts • If museums did not exist, what would our society miss? Give one example. • What is one example of evidence that indicates the value of museums to communities? • What is one example of evidence that indicates the value of museums to individuals? We will collect your responses following the session.

4. Session Goals • Increase awareness of studies that explore the value of museums across different dimensions (national, community, individual). • Identify areas for further investigation.

5. IMLS Mission • The Institute’s mission is to help build the capacity of libraries and museums: To connect people to information and ideas • The Institute is the federal voice for the nation’s 123,000 libraries and 17,500 museums.

6. IMLS Activities • Through grant-making, convenings, research, and publications, the Institute builds the capacity of museums and libraries to serve the public. We: • Sustain Heritage, Culture, and Knowledge; • Enhance Learning and Innovation; • Build Professional Capacity.

7. IMLS and Public Value • Government Performance and Results Act, 1993 • Perspectives on Outcomes Based Evaluation for Libraries and Museums, 2000 • Shaping Outcomes online course on outcomes-based planning and evaluation (shapingoutcomes.com)

8. Grant Application Requirements • Mission and strategic plan • Community needs assessment • Proposed outcomes and impact • Dissemination and Prospects for Sustainability

9. Grants for Research • National Leadership Grants • National impact • “Raise the bar” • Move the field forward • Up to $1,000,000

10. Exploring the Evidence Base for Museum Value Dr. Carol Scott Renaissance London Programme Manager for 2012 London Museums Hub London, UK Friday 1st May 2009 AAM Philadelphia

11. Format • Background • Values typology • Value indicators • Value evidence • A call to action

12. Context …measuring and articulating the value and impact of the sector is more than an academic exercise: given the policy, financial and business structures in which most cultural organizations operate…, rightly selecting, rigorously measuring and powerfully articulating the value and impact of the sector is one of the key pre-requisites for its sustainability (Stanziola 2008, 317)

13. What do we mean by ‘value’? Noun • Worth • Importance • Significance • Meaning • Merit • Use Verb • To appreciate • To treasure • To cherish • To attach importance to…. • To set great store by……

14. What value and whose values? Instrumental outcomes Social cohesion and inclusion Regeneration Access Knowledge and creative economies Problems imposed external dominant hard to assess

15. Instrumental value going beyond function and having aspirations to a wider agenda of social change Intrinsic value inherent qualities of things, often intangible, described in affective language, without a utilitarian dimension Institutional value processes and practices that agencies adopt to create value for the public; rooted in the ethos of public service; Public Value Use value direct use, indirect use and non-use value of museums A holistic concept of value

16. A values typology

17. Value indicators Can we translate a values typology into assessable indicators?

18. Use value indicators 1) Direct use (physical visits) 1a) Number of visitor attendances to museums annually 2) Indirect use (use of outreach services) 2a) Number of users of outreach programs ie. number of participants to traveling exhibitions outreach programs including lectures and workshops 2b) Number of unique visits to museum websites 3) Engagement 3a) Number of volunteers 3b) Total number of volunteer hours per annum 3c) Number of members 3d) Number of unpaid hours contributed by Boards of Trustees, fundraising groups, etc. 3e) Number of visits per visitor per year 4) Non –use 4a) Willingness to pay irrespective of direct engagement

19. Institutional value indicators 1) Recognition of trusted expertise 1a) Number of public enquiries annually 1b) Number of external projects for which museum expertise has been requested 2) Building relationships 2a) Number of local, national and international partnerships involving museums and other government agencies 2b) Significance of these projects in terms of $, number and type of major stakeholders 3) Attracting investment 3a) Value of government grants (capital and recurrent) 3b) Number and value of sponsorships (cash and in kind) 4) Capacity to bequests and donations 4a) Number and value of donations 4b) Number and value of bequests

20. 1) Providing educational resources 1a) Number of school students visiting 1b) Number of partnerships with education bodies 1c) Number of adult education programs/participants 2) Knowledge building 2a) Number of research publications based on collections 2b) Number and value of museum/university projects funded by Research Grants 3) Contribution to tourism 3a) Number of domestic tourists annually 3b) Number of international tourists annually 3c) Number of museums that win tourism awards annually 4) Contribution to local economy 4a) Number of EFT employed staff 4b) Value of local services purchased 5) Social inclusion 5a) Number and percentage of visitors by ethnicity 5b) Number and percentage of visitors by socio-economic status Indicators of instrumental value

21. Evidenceof value Does the evidence exist to support value?

22. Where is the evidence? • Central government agencies eg National Bureaux of Statistics, Tourism Research Centres, Time Use Surveys • Government Departments • National Museum Bodies • Individual Museums

23. Issues • Evidence is dispersed; central point to collect and collate data on a regular basis • Absence of consensus on a set of values indicators • A sector-wide approach to research -intrinsic value -contingent valuation studies • Intentional planning for long term social impact is the exception rather than the norm • Methodological suite of evaluation methods is limited

24. A call to action • A set of shared indicators around a holistic values framework • Centralised agencies charged with data collection • Coherent national programmes for ongoing research (a) sector-wide studies that examine the intrinsic value of museums (b) contingent valuation studies ( c) long term social impact • Acceptance that there are implications for our practice

25. Thank you Carol Scott carolannscott@fastmail.co.uk +44 787041 7079

26. American Association of Museums 2009/Philadelphia – 1st May 2009 Treasuring and Measuring – Evidence From Community Stakeholdersa case study from New Zealand Dr Jane Legget Auckland University of Technology In cooperation with The New Zealand Tourism Research Institute NZTRI I Private Bag 92006 I Auckland 1142 I New Zealand I Ph (+64 0) 21 109 8884 I Jane.Legget@aut.ac.nz I www.nztri.org

27. Outline of Presentation Introduction Case-Background Methodology Community Stakeholder Findings Concluding Remarks Outlook

28. Introduction What is it about museums that matters to stakeholders? How do stakeholders make their assessments of their museum’s performance in the context of public accountability? Identifying where community stakeholders locate their museum's value

29. Stakeholders in the Museum Stakeholder Categories Involved in the Research Board Friends of the Museum English Language Schools Researchers School Users Visitors Iwi Maori Staff - Paid/Unpaid Media Public Enquirers Corporate Sponsors Lenders & Borrowers Consultants Local/Regional Museums Tertiary Education Tourism Operators Donors Special Interest Groups Suppliers Benefactors Ratepayers Local/Regional Central Government Community Stakeholders Museum Sector Stakeholders Governance Stakeholders

30. Case Study Concept Mapping Process (I) Development of Focus Group Question Stakeholder Identification and Focus Group Selection Stakeholder Focus Groups (SFGs) Workforce Board Media School Users Ohaki Donors TangataWhenua Tertiary Users Special Interests Local Authority Other Museums Generate ‘possible performance statements’  310 Statements

31. Case Study Concept Mapping Process (II) 1st shift & edit  207 Statements 2nd shift & edit  140 Statements Prepare Statements for follow-up tasks Sorting Tasks Rating Tasks Analysis of data via Concept System Concept Maps Pattern Matching Interpretation of Results Generate ‘possible performance statements’  310 Statements

32. Focus Group Question • How would you know how your museum is doing? In other words: • How would you know if your museum is performing effectively? Responses effectively complete the following sentence: • We would know how our museum is performing by/because/if….

33. Examples of 140 ‘Possible Museum Performance Indicators’generated by community, governance and sector stakeholders • Museum’s share of the total visitor market • Success in applications for grants from central government • How culturally safe Maori feel in the museum • The time taken to respond to public enquiries • Number of repeat visits by schools and other educational groups • Quality of touring exhibitions loaned to the museum • Relevanceof the museum to its diverse local communities • Courtesy with which staff respond to offers of objects • Level of spending per visitor in shop and café • Range of visitors (age, gender, nationality etc.) visiting the museum

34. Sorting the ‘Possible Performance Statements’

35. C3. Maori Confidence C9. Maori Values C2. Collection Management C6. Utilization of Collections C8. Management Effectiveness C4. Education in the Community C7. Reputation C10. Visitor Demographics Concept Map Showing Aspects of Museum Performance Areas Identified by Community Stakeholders: C1. Staff & Operations C5. Public Interactions C11. Visitor Response

36. Collections - Quality and Management Considered a safe place for Maori artefacts and human remains Appreciation of the significance and value of items given State of cleanliness of the displays – no dusty birds! Display of taonga (Maori treasures) taking into account contemporary Maori perspectives Security systems' in place Confidence of donors in offering items for the collections Collection’s representation of, and relevance to, the local community Proper care and management of the objects, taonga and specimens

37. Staff – Calibre & Management of the Staff Whether staff can keep up with their workload Whether staff are outward-looking, promoting the museum in the community Whether staff share a common purpose & goals Whether staff are well-trained in their respective fields Maori representation on the staff Quality of research undertaken by museum staff Staff giving talks, lectures on request Museum’s ability to attract & keep high quality staff & volunteers Level of staff satisfaction

38. Relative Importance of Concepts to Community Stakeholders Community Stakeholders • Staff management & operations • Quality of collection management • Maori confidence in museum • Education in the community • Public interactions • Utilisation of collections • Reputation • Management effectiveness • Maori values • Visitor demographics • Visitor response

39. Comparing Perspectives: Governance, Sector & Community

40. Community KRG Ratings – Correlated with Governance and Sector Stakeholder Ratings Governance KRG Community KRG Sector KRG 4.17 4 4.07 Staff Management & Operations Reputation in Community Quality of Collection Management Public Interaction Visitor Response Utilisation of Collections Visitor Demographics/Trends Maori Confidence in Museum Education in the Community Maori Values Management Effectiveness 3.21 3.6 3.42 r = .50 r = .61

41. Towards a Framework for StakeholderDetermined Performance Assessment

42. Future Research • Undertake similar exercise with two or three other museums of different scale in different parts of the country • Refine the performance assessment framework to articulate the value and means of assessing museums’ effectiveness in maintaining and enhancing this • Develop menu of practical performance indicators in consultation with NZ museum directors

43. Thank You! With grateful thanks to the US Embassy, Wellington, NZ, for assistance towards attending this conference, and the Board, Director & stakeholders of Canterbury Museum, NZ, for their participation, and Professors Kerr Inkson and Mason Durie of Massey University for their guidance. Contact Details: JANE LEGGET, PHD, FMA Associate Director (Cultural Heritage) New Zealand Tourism Research Institute at AUT www.nztri.org jane.legget@aut.ac.nz NZTRI I Private Bag 92006 I Auckland 1142 I New Zealand I Ph (+64 0) 21 109 8884 I Jane.Legget@aut.ac.nz I www.nztri.org

44. Museum experiences that change visitors --------------------------- evidence that indicates the value of museums to individuals American Association of Museums 2009/Philadelphia – 1st of May 2009

45. Pipilotti Rist: Pour Your Body Out • (7354 Cubic Meters), Museum of Modern Art • November 19, 2008–February 2, 2009 • ... A printed text at the entrance asks visitors to explore the space and themselves within it, to stretch and even sing, to – yes – pour their bodies out.... after a couple of visits over the weekend I'm still under its calming influence: My body never before realized museums could be so physically rapturous and transformative. • Globe and Mail, Houpt, 2009 Barbara J. Soren, PhD AAM Annual Meeting 2009

46. Outline of presentation • an exploration of the meaning of ‘transformational experiences’ • ‘triggers’ for transformational museum experiences • two case studies describing how visitors articulate change they have experienced and actions they may take as an outcome of a museum program or exhibition visit Barbara J. Soren, PhD AAM Annual Meeting 2009

47. Introduction The nature of transformational experiences: • provide new opportunities to invent knowledge and explore new ideas • create challenges to discover the interconnectedness of ideas • transform experiences into knowledge, skills, attitudes, values, emotions, beliefs, and senses • change individuals by giving ‘cognitive hooks to the hookless’ • become more inclusive, discriminating, emotionally capable of change, and reflective Barbara J. Soren, PhD AAM Annual Meeting 2009

48. Triggers for transformational museum experiences Barbara J. Soren, PhD AAM Annual Meeting 2009

49. Case study 1: Summer InstitutesRoyal Ontario Museum, 1995-1998 A Moving Photo Album Barbara J. Soren, PhD AAM Annual Meeting 2009

50. Matrix to identify and predict interactions between museum experiences and transformation Barbara J. Soren, PhD AAM Annual Meeting 2009