Metadata and Digital Library Development

1. Metadata and Digital Library Development Introduction: Background, Goals, and Course Outline Course design: David Ruddy Cornell University Library 1

2. Cat21 Series Objectives • To equip catalogers to deal with new types of resources and to recognize their unique characteristics • To equip catalogers to evaluate competing approaches to and standards for providing access to resources • To equip catalogers to think creatively and work collaboratively with others inside and outside their home institutions • To ensure that catalogers have a broad enough understanding of the current environment to be able to make their local efforts compatible and interoperable with other efforts • To prepare catalogers to be comfortable with ambiguity and being less than perfect • To enable practicing catalogers to put themselves into the emerging digital information environment and to continue to play a significant role in shaping library services 2

3. Goals for this Course • Understand digital library design and development processes • Explore the relationship between data/metadata and system functionality • Understand the motivations and strategies for metadata conversion • Understand metadata workflow design • Appreciate the role of metadata and metadata work in digital library development projects • Recognize the role and contributions of the metadata specialist on a digital library project team 3

4. Outline of this Course Session 1: Digital Library System Objectives, Functionality, and Metadata Session 2: Understanding Functional Requirements Session 3: Metadata and Functionality Session 4: Metadata Conversion: Enhancement and Mapping Session 5: Metadata Workflows Session 6: Digital Library Development Project Exercise 4

5. Metadata and Digital Library Development Session 1: Introduction to Digital Library System Objectives, Functionality, and Metadata 1

6. Goals of Session • Understand the relationship between system objectives and metadata • Examine the objectives of the library bibliographic system and how those objectives impact system metadata • Explore the connection between digital library systems and digital library metadata • Underscore the importance of system objectives when working with metadata 2

7. The Library Catalog • Why do we describe library materials in the way we do? • Why do we catalog in the way that we do? • Why do we assemble certain information (metadata) about library materials, and record this metadata in such a highly defined way? 3

8. Cutter (1876) Objectives of a bibliographic system • To enable a user to find a book if the author, title, or subject is known • To show what the library has by a given author, on a given subject, or of a given kind • To assist in the choice of a book based on its edition (bibliographically) or its character (literary or topical) 4

9. IFLA (1998) • To find entities that correspond to the user’s stated search criteria • To identify an entity • To select an entity that is appropriate to the user’s needs • To acquire or obtain access to an entity described 5

10. Svenonius (2000) • To locate • Known entity • Set of entities • To identify an entity • To select an appropriate entity • To acquire or obtain access to an entity • To navigate a bibliographic database 6

11. Exercise 1-A • How does MARC metadata support the objectives of the Library Bibliographic System? • To find/locate • To identify • To select • To acquire • What other system objectives can we detect from the system’s metadata? 7

12. The Library Bibliographic System • System objectives have led to specific practices in bibliographic description • Standards such as AACR2 • Uniform record creation is required by global bibliographic databases • Standard record formats (MARC21) • Desired functionality requires precise cataloging rules and conventions 8

13. Exercise 1-B • Exercise 1-B is XML encoded metadata used by some type of digital information system • What system objectives can we detect by examining this system’s metadata? 9

14. Digital Library Systems • No agreed upon definition or objectives • No agreed upon standards or formats • Very little interoperability • A huge number of players, many of whom are not librarians • What is a “Digital Library,” anyway? • Digital (Electronic) Information Systems? 10

15. Digital Library Systems • A different world from the Library Bibliographic System, but not an alternate universe • Digital library system development… • Still requires the articulation of objectives (desired system functionality) • And those objectives will depend upon certain characteristics of available or generated metadata 11

16. Digital Library System Objectives • To support… • Discovery • Navigation • Presentation, display • Access control • Administration, management • Preservation • Others? 12

17. System Objectives? • Who decides on the objectives of the digital library system? • Who decides what functionality to support? • Who are the players or stakeholders on digital library projects? 13

18. Digital Library Projects • Digital library stakeholders: • Project “owner” • Project manager • Subject specialist • System developer/programmer • Metadata specialist • Library administrator/manager • End-users • Others? 14

19. System Objectives? • How do the stakeholders decide on system objectives? • How is system functionality developed? • What’s the process by which decisions are reached? 15

20. Metadata and Digital Library Development Session 2: Understanding Functional Requirements 1

21. Goals of Session • Understand functional requirements and their usefulness • Recognize how functional requirements inform system metadata decisions • Understand “use cases” and how they define and record functional requirements • Learn how a use case should be “read” by a metadata specialist 2

22. Functional Requirements (1) • What are functional requirements? • In this context, functional requirements are those of a software system, not of bibliographic records (FRBR) • A more specific and detailed description of system objectives • They describe and define specific, required system behaviors • Ideally, they are developed through a requirements analysis process • They guide system implementation and programming work 3

23. Functional Requirements (2) • How do project stakeholders develop functional requirements? • Ideally, system designers use some reasonably formal design process • Examples of design processes: • Rational Unified Process (RUP) • User Centered Design • Agile Software Development 4

24. Software Design Processes • Systematic methods for generating and defining functional requirements • Different design processes emphasize different methodologies, but there are often many similarities among them • Most processes employ “use cases,” though they may exploit different methods to generate and develop them 5

25. Use Cases • Each use case describes a single function of the system • Each function is an interaction between the system and an external USER • Each use case describes functionality, but not how that functionality will be accomplished • The entire system may have dozens or hundreds of use cases • Taken altogether, the use cases define the system’s functional requirements 6

26. The USER in a Use Case • USERs are anything external to the system that will interact with it • A USER may represent a class of users • Data entry staff • System admins • General public users • A USER may represent another system • An OAI harvester 7

27. Sample Use Case • Exercise 2: Sample Use Case • Typical use case components: • Priority • Preconditions • Flow of Events (scenario) • Alternative Events (exceptions) • What in this use case will depend on or impact system metadata? 8

28. Generating Use Cases • The design process used will likely guide how use cases are generated • A typical approach is to enumerate all the possible USERs of the system (everyone and everything that will interact with it), and then list every interaction • Each of these interactions will become a use case 9

29. A Complete Set of Use Cases • Together, they define the functional requirements of the proposed system • Documented, they form a contract among stakeholders about what the system will do and not do • Requirements help in the inevitable “panic phase” of a project • Requirements inform our decisions about metadata, standards, software, vendors… 10

30. Build or Buy? • Build or buy decisions are typical in digital library development projects • Building a digital library system • Defining one’s own functional requirements • Hiring programmers to build the system • Testing, evaluation, maintenance, updates • Acquiring a pre-built digital library system • Finding a system with functionality that meets your requirements as nearly as possible 11

31. Build or Buy • Both cases require articulating and documenting desired objectives and functionality • If Build, these will develop into complete use cases • If Buy, they will be used in the RFP process, and later to evaluate competing systems 12

32. Requirements and Metadata • Certain functional requirements will depend upon or impact system metadata • The requirements will inform our decisions about system metadata • What data elements are required • What content value practices need to be adopted • Whether metadata standards can or should be used • If we have existing metadata, requirements will inform our analysis and conversion of it 13

33. Metadata and Digital Library Development Session 3: Metadata and Functionality 1

34. Goals of Session • Review or familiarize ourselves with concepts and vocabulary of metadata assessment and analysis • Explore the connection between metadata and functionality 2

35. Metadata Specialist Scenario • The typical digital library development situation facing the metadata specialist: • We have some functional requirements to meet, AND we have some metadata • BUT the metadata must be altered in some way (cleaned-up, augmented, enhanced, mapped…) so that it will meet our requirements 3

36. Metadata and Functionality • In order to match metadata with functionality, we need first to assess, or analyze, our existing metadata • Then we can begin to evaluate whether our metadata will or won't support particular functionality and how it will need to be converted 4

37. Metadata Assessment • If we look at existing metadata, how do we describe what we observe? • File Format • Type of metadata • Semantics • Content values • Structure • Use • Status 5

38. Metadata Analysis: File Format • File, or data exchange, formats: • SGML / HTML • XML / XHTML • MARC • “Delimited” plain-text file • Binary (not plain-text) formats, either open or proprietary 6

39. 7

40. 8

41. 9

42. Metadata Analysis: Type • Types of metadata • Descriptive • Structural • Administrative • Technical • Preservation • Access/Rights 10

43. 11

44. 12

45. 13

46. Metadata Analysis: Semantics • Metadata element sets (“schemes”) • MARC21 • Dublin Core (DC) • EAD • MODS • VRA Core • METS 14

47. 15

48. 16

49. 17

50. Metadata Analysis: Content • Does the metadata… • Adhere to any published content standards or best practices? • AACR2/RDA, EAD Best Practice (RLG), CCO • Use any known and shared vocabularies? • LCSH, AAT, TGM • Adhere to any application profiles? • Degree of conformance to any employed standards, practices, or vocabularies? 18