Don t keep it under your hat A Fez Fedora Case Study

1. Don�t keep it under your hatA Fez Fedora Case Study Christiaan Kortekaas Lead programmer, Fez and eScholarshipUQ Testbed Australian Partnership for Sustainable Repositories Conference Paper Summary: In response to demand for a robust and scalable system to host and manage access to a range of electronic content, including theses, book chapters, articles and other research output and teaching materials, the University of Queensland has developed "Fez" - a new, open source, Fedora-based digital repository management and workflow system. Fez is based on PHP and MySQL and works as a front-end and administration and content mangement tool using Fedora 2.1.1 (http://www.fedora.info/). It is a highly flexible and configurable system for repositories. In 2006, the University of Queensland is undertaking a Research Assessment Exercise (RAE) in which academic staff from different schools and centres must nominate the three research works from the last five years that they consider their best. These works will then be assessed by different international panels of experts in each of the research fields. This case study explores the development and architecture of a Fez based repositories and looks at ways in which such a system could support the Research Assessment Exercise. Specific content models were developed to meet the reporting requirements of the RAE. These included a 'citation view', with links to either the object's Digital Object Identifier or to a locally housed version of the file to grant reviewers full access to materials. To simplify content creation Fez was integrated with data feeds from UQ central human resources systems to provide rich form controls based on cutting-edge AJAX technology. For example an 'Author Suggest' control which acts like Google Suggest. Fez is also being prepared for federated authentication and authorization based on the international standard eduPerson attributes, implemented with Shibboleth technology. This will assist future RAE processes by allowing external reviewers to authenticate using their own institutions central identify provider and Fez will be able to base access control rules for these external reviewers based on their individual eduPerson attributes.Conference Paper Summary: In response to demand for a robust and scalable system to host and manage access to a range of electronic content, including theses, book chapters, articles and other research output and teaching materials, the University of Queensland has developed "Fez" - a new, open source, Fedora-based digital repository management and workflow system. Fez is based on PHP and MySQL and works as a front-end and administration and content mangement tool using Fedora 2.1.1 (http://www.fedora.info/). It is a highly flexible and configurable system for repositories. In 2006, the University of Queensland is undertaking a Research Assessment Exercise (RAE) in which academic staff from different schools and centres must nominate the three research works from the last five years that they consider their best. These works will then be assessed by different international panels of experts in each of the research fields. This case study explores the development and architecture of a Fez based repositories and looks at ways in which such a system could support the Research Assessment Exercise. Specific content models were developed to meet the reporting requirements of the RAE. These included a 'citation view', with links to either the object's Digital Object Identifier or to a locally housed version of the file to grant reviewers full access to materials. To simplify content creation Fez was integrated with data feeds from UQ central human resources systems to provide rich form controls based on cutting-edge AJAX technology. For example an 'Author Suggest' control which acts like Google Suggest. Fez is also being prepared for federated authentication and authorization based on the international standard eduPerson attributes, implemented with Shibboleth technology. This will assist future RAE processes by allowing external reviewers to authenticate using their own institutions central identify provider and Fez will be able to base access control rules for these external reviewers based on their individual eduPerson attributes.

2. Fez Case Study � UQ Research Assessment Exercise How we used Fez and Fedora software to support a large, complex process consisting of many users, internal and external and thousands of records and specific requirements. What we learnt during the process, how did the software meet the requirements, what would we do better next time. What other things we are doing with the repository I am going to explain how the requirements of the RAE were met by Fez features Will explain what the RAE is shortly but first.. I am going to explain how the requirements of the RAE were met by Fez features Will explain what the RAE is shortly but first..

3. Firstly, what is Fez? A free, open source, flexible, highly configurable digital repository and workflow management system based on Fedora 2 Manages all kinds of documents publications, images, learning objects, spreadsheets, databases and datasets, course materials, exam papers, institutional records, documents or a mixture of all of the above � When assessing all the options of for a digital repository we liked Fedora the most due to its design, in that it was designed to be as flexible as possible. So when designing Fez we wanted to keep with that idea of flexibility and in the front end application and this has been the design philosophy behind Fez. What are the downsides? - Can be hard to install even for experienced IT administrators - Fairly new - Open source development model (hey that�s an upside!) - Open source blame model for management, might frighten small institutions who want a turn-key/in-a-box solution, no vendor support - However there is free email and forum support, happy to assist in further detail over phone or in person - Usually respond to emails within 24 hours or earlier depending on timezones - Community support should grow with adoption, Fez experts and patch submitters -> if you build it they will come? - Cost: Requires 1 IT admin (hopefully with some PHP skills) and at least 1 champion of the cause, and a Server to put it onWhen assessing all the options of for a digital repository we liked Fedora the most due to its design, in that it was designed to be as flexible as possible. So when designing Fez we wanted to keep with that idea of flexibility and in the front end application and this has been the design philosophy behind Fez. What are the downsides? - Can be hard to install even for experienced IT administrators - Fairly new - Open source development model (hey that�s an upside!) - Open source blame model for management, might frighten small institutions who want a turn-key/in-a-box solution, no vendor support - However there is free email and forum support, happy to assist in further detail over phone or in person - Usually respond to emails within 24 hours or earlier depending on timezones - Community support should grow with adoption, Fez experts and patch submitters -> if you build it they will come? - Cost: Requires 1 IT admin (hopefully with some PHP skills) and at least 1 champion of the cause, and a Server to put it on

4. Benefits Configure it easily for your specific content model, workflow and security needs using the administration web GUI Persistent identifiers, OAI and other core functionality in current and future releases of Fedora Base security and workflows on meaningful roles eg. Creator, Editor, Approver, Viewer, Lister

5. What does it do? Manages objects in a repository Object content model: Communities, e.g. a school or centre Collections, e.g. digital image sets, audio files, digitized scans Individual items, e.g. a single publication Objects can belong to more than one collection or community Dynamic aggregations coming in future versions.. Define your own grouping templatesDynamic aggregations coming in future versions.. Define your own grouping templates

6. Why did we develop it? 2002 � ePrints repository for research publications with eprints.org software 2004 � APSR Project (DEST grant) Identified multiple UQ digital data collections Developed ResearchFinder tool to point to distributed repositories at UQ 2005 � Single repository needed to house and preserve all kinds of (orphaned) UQ research output 2006 � UQ eSpace (based on Fez) UQ Research Assessment Exercise - Also because we assessed all the available options, thought Fedora was the best designed, and thought we could put together a front-end with our in house skills in programming RAE has been a driving factor for the last 6 months of development At this stage I would like to show you at least the front page interface as a picture tells a thousands words and will help to buy - Also because we assessed all the available options, thought Fedora was the best designed, and thought we could put together a front-end with our in house skills in programming RAE has been a driving factor for the last 6 months of development At this stage I would like to show you at least the front page interface as a picture tells a thousands words and will help to buy

7. Designed similarly to the ePrints interface, added recently added items, newsDesigned similarly to the ePrints interface, added recently added items, news

8. How does it work ? Handles different content models - defined by XSDs New models/XSDs can be created at any time XSDs trigger input screens Each object has layers of metadata and associated datastreams An eXtensible Schema Diagram is an XML expression of a Data Type Definition An eXtensible Schema Diagram is an XML expression of a Data Type Definition

9. Fez comes with these base level XSDs, matching closely with the ePrints document types. So to manage all these XSD and displays we designed a visual editor to match the XSD element against how it should be displayed in Fez -> so it is display metadata about template metadata which will instantiate FOXML objectsFez comes with these base level XSDs, matching closely with the ePrints document types. So to manage all these XSD and displays we designed a visual editor to match the XSD element against how it should be displayed in Fez -> so it is display metadata about template metadata which will instantiate FOXML objects

10. This is the Fedora Object XML XSD Display for Book templates At the xmlContent element level this is where it references other XML (X) based datastreams XSDs for their templatesThis is the Fedora Object XML XSD Display for Book templates At the xmlContent element level this is where it references other XML (X) based datastreams XSDs for their templates

11. UQ Research Assessment Exercise 2006 What? - RAE is a �dry run� for Research Quality Framework (RQF) 2007 Why? - RQF will determine Australian university research funding from DEST When? � Started Jan 06, Finished June 06 How? - Academic staff from different schools and centres nominate their 3 best works from last 5 years These works will then be assessed by different international panels of experts in each of the research fields

12. UQ Research Assessment Exercise 2006 � WorkFlow Close relations between Library with Office of Research and Postgraduate Studies (ORPS) organising the RAE ORPS organises which schools, authors will be involved in research programs Liaison Librarians assisting Schools to create documents for authors Unfinished Objects sit in My Fez submission buffer until ready to �publish� Authors login and check their published records Review Panel login and review authors� records in a �research program� area of expertise (by quality, impact) ORPS did much of the liaising with Heads of schools, RAE committees, research program panels Strengthened relationship between library and ORPS during planning and implementation process How is this process managed? With WorkflowsORPS did much of the liaising with Heads of schools, RAE committees, research program panels Strengthened relationship between library and ORPS during planning and implementation process How is this process managed? With Workflows

13. Workflows in Fez Configures the way a user manipulates digital objects Create procedures for reviewing and publishing objects Automatic processing of objects e.g. thumbnails, preservation metadata Can be role-restricted

14. Workflow components Triggers � Associate workflows with collections and contexts Behaviours � e.g. change metadata, generate thumbnails State Links � order of the steps Status tracks progress stores variables passed between steps

17. UQ RAE 2006 Content Model UQ RAE 2006 = Fez Community Lists of submitting schools = UQ RAE 2006 Collections Customisation of document types required for RAE (extra fields, ordering, naming) Journal Articles Conference Papers Patents Books Book Chapters

18. UQ RAE 2006 Content Model Customisation of Collection Type for RAE Associate collection with organisation structure for record entry/edit forms to get dynamic: Authors from the school/centre for Author Suggest (Ajax) control RAE Submitting Authors from the school/centre for drop down list List of Research Programs for drop down list Ajax is web 2.0 as per sandys web 2.0 slideAjax is web 2.0 as per sandys web 2.0 slide

19. UQ RAE 2006 Security Requirements All RAE records must be hidden from public view Only editable by designated data entry groups (liaison librarians and some assisting school staff, except for ITEE) Only viewable by RAE document authors and RAE assessment panels The School of information technology and electronic engineering requested that all their staff be able to create, edit and publish records in their RAE colelction The School of information technology and electronic engineering requested that all their staff be able to create, edit and publish records in their RAE colelction

20. Secure your non-OA documents Security is based on Fez Access Control Markup Language (Fez-ACML) Define your security rules with the same web based GUI (rather than manually editing XACML) Uses security inheritance from parent communities � can be set at any level (including per managed datastream) Rules grant a user Fez roles based on their user attributes

21. Fez-ACML Authentication/authorisation sources can be any combination of: Your organisations central LDAP/Active Directory and LDAP/AD user attributes Fez internal user details and user groups Your users eduPerson attributes (for universities)

23. Was the RAE a success? Current Stage: Most of the research program panels have completed reviewing author works � it�s nearly over! Review Panelist Dr. Jonathan Tittler from Rutgers �The system appears to work with remarkable ease, given the complexity of the task at hand.� ORPS Compared to first (single school) RAE, vast improvement Are we ready for RQF 2007? First RAE was a single school, eSpace/Fez was not ready at that stage so it was done with static html pages and ip rule security Dr Tittler was a member of the review panel for First RAE was a single school, eSpace/Fez was not ready at that stage so it was done with static html pages and ip rule security Dr Tittler was a member of the review panel for

24. UQ RAE 2006 Lessons learnt Best way to bug-test a system, throw users at it, fill it with data Data quality means everything Find bugs early Make sure the object creators understand what fields mean � communication and training DOI Will be adding admin user configurable javascript regex patterns and common patterns to apply to form controls, with configurable error messages Some data quality control features would be very useful A global search/replace regex on elements of XML � possibly XPATH/XQUERY based, would have been useful for DOI A global Version control interface for rollback of XML elements matching regexes, eg Controlled Vocab / RFCD bug Will be adding admin user configurable javascript regex patterns and common patterns to apply to form controls, with configurable error messages Some data quality control features would be very useful A global search/replace regex on elements of XML � possibly XPATH/XQUERY based, would have been useful for DOI A global Version control interface for rollback of XML elements matching regexes, eg Controlled Vocab / RFCD bug

25. What else can Fez be used for? Generic Institutional Repository (IR) Corporate Document Management System (DMS) Image collection manager Batch import directory full of files Migrate from ePrints/DSpace/(ADT) Batch import export.xml for ePrints Batch import METS objects for DSpace UQ ADT OAI->METS->Batch import for ADT

26. Important features in development Comment and annotation on objects Complex OAI provider based on FezACML security Object provenance and history logging to meet PREMIS audit recommendations

27. Grant based feature development MAMS mini-grant Federated authentication and authorization based on eduPerson attributes (using Shibboleth) APSR extra grant funding Predictive searching (like Google Suggest) Batch ingest desktop GUI client Macquarie Access Management System. Working with UQ central IT services We are doing the Shibboleth SP they are doing the Identity Provider The current Fez 1.2 release supports shibboleth authentication and authorization, Fez 1.3 will improve Shib support and provide Where are you from? (WAYF) functionality eduPerson attributes are an international standard for example, staff, students, school facultyMacquarie Access Management System. Working with UQ central IT services We are doing the Shibboleth SP they are doing the Identity Provider The current Fez 1.2 release supports shibboleth authentication and authorization, Fez 1.3 will improve Shib support and provide Where are you from? (WAYF) functionality eduPerson attributes are an international standard for example, staff, students, school faculty

28. Fez�s future Release of Fez 1.1 source code December 2005 http://sourceforge.net/projects/fez/ Fez 1.2 Released April 2006 FezACML security granularity down to datastreams Batch Processing done in background with process monitor Improved Reindexing Shibboleth authentication/authorisation option Much more� see release notes Fedora 2.2/3.0 support as they come out Fez 1.3 (due in July/August 06) Complex OAI Provider based on FezACML security Comment and annotation on objects Object provenance and history logging Fez Development APSR funded until end 2006, some possibilities afterwards Reindexing needs a little work and its likely we will be putting out a 1.2.1 release of Fez to address some indexing issuesReindexing needs a little work and its likely we will be putting out a 1.2.1 release of Fez to address some indexing issues

29. Fez technologies Free, open source + platform- independent PHP 5.0+ MySQL 5 A Server (Linux, Windows, Mac) Webserver software (Apache 1.3, others untested) Fedora 2.1.1 Java SDK, JRE 1.4+ Built-in Tomcat Server Fedora 2.0 also supported with a config change What you should care about here is all these technologies are free and open source and platform independent To run on Fedora 2.0 you need to make a slight change to the config files to point them to the old Fedora API locations Will not run on anything prior to Fedora 2.0 as it is built on the FOXML schemaWhat you should care about here is all these technologies are free and open source and platform independent To run on Fedora 2.0 you need to make a slight change to the config files to point them to the old Fedora API locations Will not run on anything prior to Fedora 2.0 as it is built on the FOXML schema

30. Production Fez sites UQ�s eSpace http://espace.library.uq.edu.au 95% is secured RAE content at this stage UNEDs site http://62.204.194.47/fez/ In Spanish, and shows how Fez Smarty template design allows for translation of Smarty web-apps into other languages