Library Collaboration and Digital Exploration

1. JUCC 35th Anniversary IT Conference on Campus Informatization: Challenges and Opportunities November 16-18, 2005, Hong Kong Library Collaboration and Digital Exploration Presented by Ki Tat LAM, Head of Library Systems and Edward Spodick, IT Manager The Hong Kong University of Science and Technology Library {lblkt,lbspodic}@ust.hk

2. Table of Contents • Introduction • HKUST Library’s IT Background • Digital Libraries • Library Collaboration • Digital Libraries Explorations at HKUST • Examples of Library Collaboration • HKIUG Unicode Initiatives • Resource Pool Web Site – Towards JULAC Software Sharing • Future Ideas

3. About HKUST Library • HKUST is a young institution having its first student intake in 1991

4. About HKUST Library • HKUST Library Mission • To support the University's teaching and research programs • To provide for and to promote the general education of HKUST Students • To contribute to Hong Kong and its regional development in information sharing and exchange

5. IT at HKUST Library • The Library has been embracing new technologies since its founding: • 1991 – rolled out a fully Chinese-capable multilingual integrated library system • 1991 – set up a large-scale campus-wide CD-ROM network • 1992 – mounted a large-scale networked full text image jukebox system • 1993 – helped design and implement a Course Reserve Image System

6. IT at HKUST Library [2] • 1995 – established the Library Web Server • 1997 – managed a regional mirror site for the Ovid database system • 1997 – participated in a consultancy service for the Open University of Hong Kong’s Electronic Library Project • 1998 – began digital library projects and developed the following database systems: Digital University Archives, HKUST Electronic Theses and HKUST News Clippings

7. IT at HKUST Library [3] • 2000 – conducted Wade-Giles to Pinyin Conversion • 2001 – implemented XML-based information infrastructure for future digital library projects • 2002 – developed an XML Name Access Control Repository System as a global prototype • 2003 – collaborated with HKIUG libraries to improve the Unicode support in the library catalog

8. IT at HKUST Library [4] • 2003 – implemented DSpace for Institutional Repository • 2004 – implemented DSpace for Digital University Archives • 2005 – developed the open source JURO – Journal Usage Report Online software • 2005 – created a resource pool web site to host software developed by HKUST Library for open downloading

9. Digital Libraries • Drastic IT advancement in the past few decades: • Increasing availability of technology for information processing • Cheaper and faster hardware • More capable software • Readily accessible network • Allowing smaller and less well funded libraries to participate in digital exploration

10. “Personally I think they went a step too far in making the library electronic” [Extracted from: http://www.ariadne.ac.uk/issue24/cartoon/] Digital Libraries [2] • Similar terms: “digital”, “electronic”, “virtual” libraries

11. Digital Libraries [3] • Three meanings of “digital libraries”: 1. Libraries providing: • Access to digital information using a variety of networks, including the Internet; and • Services in an automated environment 2. The result of projects to digitize library materials for network access 3. A discipline which refers to research on the theories and technologies for the building of digital libraries [extracted from the authors’ ICADL 2005 presentation]

12. Digital Libraries [4] • Digital Library reflects a fundamental shift in how libraries do what they do, rather than in what they do • Basic mission of a library to support its user community through enhanced access and services remains the same • Improvements in IT allow greater flexibility in all stages of these processes

13. Library Collaboration • With the increase in digital capabilities, more projects which previously relied on large regional or global entities can now be accomplished through local collaboration • Through the adoption of open-source software, libraries have become empowered with the tools needed to build the infrastructure for these collaborations

14. Library Collaboration [2] • Why collaboration? • Progressive budget cuts reduce the ability of a library to efficiently respond to new IT developments • Reductions in salary and positions make a library less competitive with the private sector for skilled staff • Growing need for resource-intensive activities in contrast to a library’s stagnant ability to perform them

15. Library Collaboration [3] • Librarians have a strong tradition of resource sharing • JULAC Libraries Collaboration • JULAC White Paper on collaboration to HUCOM early this year • HKALL System for joint universities book borrowing • Activities from various JULAC committees and task forces; CDC (Collaborative Development Committee) being the most active one.

16. Digital Library Exploration • 1994 - potential capability for the Web to become primary gateway to information • Once the web infrastructure and technology were in place, a progression of digital library projects began to take place • The HKUST Library was suddenly able to provide much more accessible information about its collections, programs and services for our users.

17. Digital Library Development • Web technologies • Web-enabled content and services • Mark-up options • Searching and harvesting • XML and its related technologies

18. Digital Library Development [2]

19. Content Digitization • In 1998, after more than four years of experience digitizing materials using the TIFF format for access from dedicated PCs, it was decided to base newer digitization projects on Adobe’s PDF document format with a Web interface. • HKUST Theses • Digital University Archives • HKUST-related News Clippings

20. Content Digitization [2] • PDF was more portable created smaller files than TIFF, and a freely available multi-platform viewer could be used as a Web browser plug-in. • Documents were digitized and hyperlinked to the bibliographic metadata. • Text was extracted from the PDF documents using Acrobat Capture and some Chinese OCR software.

21. XML Metadata • The Library Catalog contains bibliographic metadata for all print, multimedia and electronic resources • Metadata is exported from the Catalog to the database for indexing and display • The HTML format is limited by its lack of metadata options • eXtensible Markup Language (XML)

22. XML Metadata [2] • 2001 - An Electronic Journals database system was developed, using XML technologies to automate the process of metadata extraction from the Catalog • Real-time XML extraction was later available using a special vendor-provided URL • XSL style sheets were developed to transform the metadata into a format suitable for the Library’s applications

23. XML • Partially fulfills the old promise of having computers doing more of the work, through the development of more compatible database connectivity, SOAP-based and other Web Services and native XML database systems • A current example is RSS, used by libraries for everything from announcing current service hours to distributing “podcasts”

24. XML [2] • 2003 - HKUST Library’s first RSS “feeds” for new book and media acquisitions • A set of standards which greatly simplified collaboration efforts

25. Name Access Control • In cataloging, the use of names is governed by Authority Control, which ’authorizes’ one form of a name as an access point • Access Control provides a central place for looking up various forms of a name, allowing the linking of variant forms without declaring any of them to be the authorized form

26. Name Access Control [2] • 2002 - An XML Name Access Control Repository was developed to address the problems identifying personal authors whose names are in non-Latin scripts http://library.ust.hk/info/nac/ • Global prototype • Great collaborative potential, but only if the concept is widely adopted and becomes an actual or defacto standard

27. Journal Usage Report Online • Software to acquire electronic resource usage statistics from vendors, and allow online querying and generation of usage reports • Using COUNTER-compliant vendor usage data • Consolidates multiple partially duplicative efforts by different Library units

28. Journal Usage Report Online [2] • Meets a clearly defined need not yet being met by available software • Permits much more accurate analysis for Collection Development and Acquisitions efforts • Semi-integrated with the Library Catalog http://library.ust.hk/cgi/db/juro.pl

29. Journal Usage Report Online [3] • Enhancements: • Local usage statistics for products from vendors who do not provide usage data • Periodic surveys of manually compiled print journal usage data

30. HKIUG Unicode Initiatives • HKIUG – Hong Kong Innovative Users Group • Originally founded in 1996 by JULAC as a users group to facilitate communication among local libraries in using the INNOPAC library system, which is developed by Innovative Interface Inc. • Became an independent body in 2001, with membership expanded to include all INNOPAC libraries in Hong Kong and Macau

31. HKIUG Unicode Initiatives [2] • HKIUG Unicode Initiatives - one of the many successful library collaboration projects in which HKUST Library participated • Results: • Improved Unicode support in members’ library catalog • Vendor gaining better understanding of issues in Chinese information processing

32. HKIUG Unicode Initiatives [3] • INNOPAC CJK (Chinese, Japanese, Korean) background in 2003 • Local libraries were heavily using the BIG5 legacy Chinese character encoding system • INNOPAC adopted EACC (East Asian Character Code) for storing and indexing CJK data; and used an unreliable mapping table to display and search Chinese characters in BIG5

33. HKIUG Unicode Initiatives [4] • INNOPAC was in the transition towards Unicode support, with the development of the Millennium software • CJK support in the Millennium software was immature and buggy • Vendor did not understand the actual CJK needs and requirements • The EACC to Unicode mapping table was inaccurate and failed to handle multi-mappings, resulting in incorrect display, retrieval and storage of CJK data.

34. HKIUG Unicode Initiatives [5] • An example of multi-mappings

35. Retrieve 27462A 历 (Simplified form of 歷) Mapping Table EACC-based Database One-to-many Obtain 27462A ? or 274349 ? (Simplified form of 歷? or 曆?) Many-to-one Obtain U+5386 历 Save U+5386 历 Mapping Table Unicode-based Editor HKIUG Unicode Initiatives [6] • Multi-mapping causing failure of round-trip cross-walk

36. 系 (U+7CFB) 係 (U+4FC2) 繫 (U+7E6B) related HKIUG Unicode Initiatives [7] • TSVCC Linking Problem • Linking Traditional, Simplified and Variant Chinese Characters • For example, when searching 系 as the simplified form of 係 , it is desirable to retrieve both, but not 繫.

37. HKIUG Unicode Initiatives [8] • Individual efforts to resolve these problems – not working • Individual libraries communicated with the vendor – fixes were in piece-meal fashion • Some libraries conducted their own CJK/Unicode study with attempts to propose to the vendor how to tackle these problems – again without much progress • HKUST (April 2003) • City University of Hong Kong (July 2003)

38. HKIUG Unicode Initiatives [9] • Collaborative effort at HKIUG followed • 20 June 2003 – HKIUG Standing Committee met and agreed that a joint proposal was essential for gaining acceptance from the vendor • 11 July 2003 – A seminar was organized by CUHK to solicit ideas and comments from HKIUG colleagues

39. HKIUG Unicode Initiatives [10] • June-September 2003 – established a working group to study and develop a joint proposal to the vendor. Members consisted of catalogers and systems librarians from CITYU, CUHK, HKUST and HKU. • 16 September 2003 – working group completed the study and submitted the proposal to the vendor together with a HKIUG version of the EACC/Unicode Mapping Table • October 2003 – vendor accepted the proposal

40. HKIUG Unicode Initiatives [11] • November 2003 – the HKIUG code table was first implemented at HKUST as a pilot test and subsequently at other HKIUG libraries • 27-28 January 2004 – K.T. Lam of HKUST was invited to the vendor’s Headquarters to work with its staff on resolving outstanding CJK issues – a face-to-face opportunity to express HKIUG libraries’ concerns and to make them understanding the issues. Results: • Multi-mapping problem in Editor – fixed • TSVCC Linking – began development

41. HKIUG Unicode Initiatives [12] • 23 October 2004 – HKUST created the TSVCC linking tables and submitted to the vendor for testing • 30 November 2004 - the TSVCC tables were endorsed by HKIUG with more enhancements • 25 February 2005 – established HKIUG Unicode Task Force to maintain the Unicode and TSVCC code tables and to assist the vendor on Unicode migration; members from CUHK, CITYU, HKUST and HKU.

42. HKIUG Unicode Initiatives [13] • Lessons learnt: • Vendors are more comfortable and responsive to work with collaborative groups than individuals • Joint effort increases productivity and reduces redundancy • Deliverables produced in collaborative effort are more authoritative and attractive for adoption by others

43. Shared Resource Pool • HKUST Library is organizing a Website to host software packages developed and/or modified by the Library http://library.ust.hk/software/ • They are freely available for downloading by any interested parties • Open source codes are provided wherever possible

44. Shared Resource Pool [2] • We believe that it is highly beneficial that libraries which develop non-commercial software tools which might prove useful to other institutions make their efforts available to the larger Community of Libraries.

45. Shared Resource Pool [3] • The Open Source Software Movement • What is open source software? • “The basic idea behind open source is very simple: When programmers can read, redistribute, and modify the source code for a piece of software, the software evolves. People improve it, people adapt it, people fix bugs. And this can happen at a speed that, if one is used to the slow pace of conventional software development, seems astonishing.” http://www.opensource.org/ • Free Redistribution • Source Code • Derived Works

46. Shared Resource Pool [4] • The Open Source Software Movement • Customize, inspect, secure, repurpose, etc. • Required customization would not have been possible with closed-source software • Example: adding the ability to search and display only authorized documents in the Digital University Archives based on each individual user’s identity

47. Shared Resource Pool [5] • The Open Source Software Movement • We use open source software extensively, such as… • LAMP: GNU/Linux, Apache, MySql, PHP • Two of our digital library projects use DSpace • Meeting Room Booking System (MRBS) • Statistics programs (Analog, AWstats, etc.) • Sleepycat Berkeley DB XML • … • we could keep going for several screens… 

48. Shared Resource Pool [6] • Available packages on the Resource Pool website: • Pinyin Conversion Project • University Archives EAD XSLT • Room Booking System • Library Catalog Print2Mail Project • JURO: Journal Usage Report Online

49. Shared Resource Pool [7] • Wade-Giles -> Pinyin Conversion Project • Example of indirect collaboration • Feedback: “When using the Text2Marc programme to turn the reviewed files back to MARC format, we found that if the 008 tag ended in a blank, it was truncated to the last non-blank character. This would generally be the end of the language code in 008/35-37, resulting in some 008 tags that were 2 bytes too short” – McGill University

50. Future Ideas • Expanding the Shared Resource Pool to host JULAC downloadable software • Site-specific customization • Reducing redundant efforts • Distributed input yields improved software quality • A good project for JULAC Libraries’ Collaboration