ISO 2789 and ISO 11620: standards as reference documents in an assessment process

1. ISO 2789 and ISO 11620: standards as reference documents in an assessment process

2. Preliminary precautions, when speaking about standards and evaluation • An acknowledged naïvety. Consider the difference between the reality as-seen-through-the-standards and what-really-happens-in-libraries. • An hypothesis of honesty. You need to be two to play the game of assessment. • A national context. My point of view is necessarily oriented by my french field of experimentation.

3. Library statistics running after reality • Today • Yesterday (overdone !) ? ? ? People / using / book users / doing things / library stuff Need for data Easiness of harvesting

4. Standardization running after the libraries reality • While libraries evolve : • Quickly • In different ways • Experimentaly • Standardization requires : • Time • Consensus • Stand back, feedback What can actually be measured and evaluated (at a reasonable cost) What should be known about impact and missions fulfilment

5. ISO 2789 : a standard for defining, classifying and counting in libraries • What you decide to count and the way you classify says something about how you see the world. • example : indicator number of visits • external example : how to measure « well-being ». • “Classifications are theories about the basis of natural order, not dull catalogues compiled only to avoid chaos.” Stephen J. Gould, Wonderful life : the Burgess Shale and the nature of history

6. What to find into ISO 2789 ? • ISO 2789 International library statistics • Definitions • Libraries • Collection • Use and users • Access and facilities • Expenditure • Library staff • Uses, benefits and limitations of statistics • Reporting statistical data • Collecting statistical data • Same divisions as definitions • Annex A : measuring the use of electronic library services • Annex B : recommended categories for further analysis

7. OPAC Library website Electronic collection Electronic document delivery Electronic reference services User training on electronic services Internet access Databases Electronic serials Digital documents Computer files Full text Ebooks Electronic patents Abstract and indexing Networked AV documents other other The question of electronic services :a clear classification of services

8. The use of electronic services :a small set of data • Three core datasets have been identified which should, if possible, be collected for all services, separately for each service as well as summed for all services : •  number of sessions; •  number of downloads (split up into content downloaded and record downloaded); •  number of virtual visits; • In addition to these core datasets which provide basic information on the use of electronic services, some additional data have been found relevant and should be collected when possible and appropriate: •  session time; •  number of rejected sessions (turnaways); •  number of searches; •  number of Internet sessions.

9. ISO 2789 & 11620 and other management tools Public Policies & evaluation Management tools ISO 9000 series Information Management standards ISO 11620 ISO 2789 Connectors to be found

10. The delimited area of action of ISO 2789and ISO 11620 General missions, as defined at a political level Objectives, aims Indicators : ISO 11620 Measure : ISO 2789

11. What does underpin ISO 11620 ? goals • A classical performance assessment model relevance effectiveness means résults efficiency • 4 families of indicators, from an adapted balanced scorecard approach • resources, access and infrastructure • use • efficiency • potentials and et development

12. Content of ISO 11620 : building robust indicators • Definitions • Criteria [of admissibility for the indicators] and descriptive framework • Uses of performance indicators • How to present indicators • Objective ; Scope ; Definition ; Method ; Interpretation and factor affecting the indicator ; Related indicators • List of indicators • Description of each indicator • Each indicator has been tested and published before

13. To sum up : relevance of the standards, today • They do exist ! • They express what can be surely and clearlydefined and calculated today • As international standards, they give a real weight to the works which rely on them, especially during political or budgetary discussions • They offer a common language • You don’t need to re-invent definitions, indicators, etc. • Use them as reference guide, not as roadmaps

14. Path for the future : toward a family of standards • Other standards/technical reports are developped into the same ISO sub-committee • performance indicators for national libraries (regarding their special missions) ; • library buildings (design and uses) • Possibility : merge or reoganize the different standards, including • Meta-standard allowing connection with quality standards and public policies evaluation • Definitions • Collection of data • Building indicators • …

15. Path for the future ? The question of intangible assets • A new field for states, a new way to measure the value of assets which are not material, which have no direct market price • Example : the Louvre trademark sold to Abu Dhabi for some years for M€ 400 • Intangible assets are, for example : • the staff’s skills, the know-how, … • the content of databases, patents, trademarks, … • the quality of information systems • Note that ISO 2789 & 11620 can help in measuring some of these items = € ?

16. Path for the future ? The question of intangible assets • It’s also a new way for states to consider : • the national and cultural heritage • The intellectual and scientific value of academic organisations • ISO 2789 and 11620 can help libraries : • measuring themselves as intangible assets’ sources ; • demonstrating their involvement in promoting intangible assets. • Libraries sould not be left apart from this new direction in public policies.

17. Path for the future ? Be imaginative ! • How to answer the question of social impact of libraries ? Two possibilities • work on synthetic indicators of library activity or « health » = imagine indicators as simple and as rich in information as the GDP (gross domestic product) ; • work from user’s perception and from differents specific services. • Consider a certification process ?

18. Post scriptum : assessment and dishonesty