Standards- implementing standards, the reality? 1

1. Introduction and Background Edward Clarke – COSE Project , Staffordshire University COSE [Creation of Study Environments] is a VLE developed at Staffordshire University under Mark Stiles over the past c. 8 yrs, benefiting from JISC funding under JTAP, then 7/99 and now X4L programmes. Free to download. Plan to make it open source. Employed since Jan ’99 on COSE, with focus on interoperability aspects using IMS specifications. If you want to discuss any aspect of presentation in more detail or are a content developer interested in content transfer between systems, pls send email to e.a.clarke@staffs.ac.uk Standards- implementing standards, the reality? 1 About the title: not my title - problem selling specs as standards / key issue is managing expectations. Specs provide a framework for interoperability (only). Personal perspective, not necessarily shared by partners / CETIS ( though I don’t think there’s anything especially contentious here ). This is a big topic – each component specification has its own CETIS SIG / community / literature associated - which could be suited to different audiences. Can only present an overview here – not appropriate to present too much technical detail – only main conclusions / lessons learned.

2. Standards .. see http://ltsc.ieee.org/ there is only one – that for the LOM data model, but no binding for this (yet) – ? expected sometime later this year. work also progressing on runtime model / CMI ?? .. and Specifications see http://www.imsglobal.org/specifications.cfm Standards- implementing standards, the reality? 2 Why they are important Nothing new about the idea of standards, except maybe in context of e-learning (VLEs and MLEs). As any standards, helps to define the quality of product or service. E.g. In software industry J2EE app servers or servlet engines (‘agree on standards, compete on implementation’ - java slogan). Avoids lock-In problem – investing in a particular VLE constitutes a substantial commitment, but users don’t want content or other data locked in Facilitates interoperation with other systems (in a more dynamic way) For marketing purposes – selling point for VLEs

3. Scope of this presentation Based on experience of working with specs, through JTAP project and 7/99 project CO3** and now SURF X4L. Some references to how industry applies of the specs (or not). bias toward content related specs and identifying issues rather than describing our implementation. Standards- implementing standards, the reality? 3 a. what is / was CO3 b. challenges and solutions - metadata, content packaging, enterprise (LIP, SCORM) specs - to give flavour of the bleeding edge c. conclusions and main lessons learned d. references: e. perspective: that was a year ago. what’s happened since.

4. a. What is / was CO3? Attempt to interoperate between 3 different VLEs in use at the Universities of Staffordshire, Bangor and Huddersfield [ COSE, Colloquia and CoMentor] - but the main aim was experiential – not necessarily to interoperate but to report on the experience of attempting to do so. From c. 10/2000 – 02/2002. Interoperability projects need partners (!) Partners at Bangor and Staffordshire were amongst the first to attempt to adopt IMS specs, so had experience prior to CO3 – this was a big plus. Standards- implementing standards, the reality? 4 Staffs had already made progress in CP and metadata, so was well positioned. Bangor had 1 person on CP and 1 person (part time) on Enterprise / CoMantle. Diversity of systems. Bangor VLE was based on P2P – with no central server - this made implementation of Enterprise problematic. Huddersfield had java front end and (non-relational) strange db back-end. Project plan too ambitious and little progress made with SCORM or LIP (but give update on this later). For more on CO3 and related papers see http://www.staffs.ac.uk/COSE/ims_briefs.html

5. b. challenges and solutions General observations / recurring themes: can be made to work between partners (!) at least at the level of proving the concept but idea of universal interoperability is rather misleading – necessary to negotiate the profile of exchange / specs as a framework for interoperability (only). limitations of specs and associated documentation - esp. relating to conformance, use of extensions, multiplicity of elements and performance / processing / system load, unique identifiers, delays in release of updates, support for different versions of any particular spec, backward compatibility. spec development is iterative and takes a long time (years) - managing expectations Standards- implementing standards, the reality? 5 Metadata Specs Early specs: (v. 1.0 and v. 1.1) dtd’s superceded by (more powerful) schema; quality of spec docs improved; distinction between CORE and SEL elements removed. people (all kinds) need educating in use of metadata – not familiar with the labels used / jargon inc. classes/category of metadata, fixed vocabularies etc..

6. Metadata Specs (challenges and solutions) different vocabularies required in different communities ( cultural aspects ) Issues in practice persuading content authors (though should need no persuading) of benefits of metadata / promoting use of metadata. pros and cons in use of simple and complex metadata making metadata entry user friendly: if extensive metadata required, need to hide complexity, subject specialists / librarians required; Standards- implementing standards, the reality? 6 searching on / use of metadata *** at least until quite recently, actual use of the metadata / searching to locate resources rarely used. much easier to implement instance conformance than system conformance. there are now repositories which use IMS metadata Instance conformance v System conformance Instance conformance does not require single element descriptor (!!)

7. Content Packaging Specs (challenges and solutions) v. 1.0 and v. 1.1.x frequent point changes (during time of project) Issues around size and naming of packages / associated metadata* / submanifests / unique identifiers, conformance,.how to reference component files, how content is structured [ see next slide on navigation v organisation ]. Errata. Maintenance release for v 1.1.3 just published (? not approved), and still issues to be resolved Standards- implementing standards, the reality? 7 In practice (proprietary) packages come with all kinds of extensions which might, for example, describe the location relative to some root or installation directory, group structures, runtime parameters etc..ie they are supposed to be self contained and re-usable but often depend on native environment Big players have commercial interest in their own version of specifications. Licenses generate revenue for Bb, MS - partners build around their products. Problem with this policy on interoperability highlights the difference between academic and commercial interests.

8. Content Packaging Specs (challenges and solutions) “Learning Resource iNterchange (LRN) is Microsoft's implementation of the IMS Content Packaging Specification, version 1.1, IMS Metadata Specification, version 1.2 and the ADL SCORM specification, version 1.2. A LRN package holds the content that makes up a course or a reusable subset of a course. The LRN Toolpad is the point of access to the available LRN tools .. Open LRN Viewer: The LRN Viewer generates a Dynamic HTML (DHTML) view of LRN content, adding a syllabus and navigation features, and provides a framework that supports the SCORM run-time environment. The LRN Viewer uses the Microsoft XML Parser with Extensible Stylesheet Language (XSL) to generate the DHTML view.” LRN 3.0 toolkit not maintained / no updates available, generates it’s own content wrapper for resources loaded into Viewer, and support’s minimal SCORM interactions. Nonetheless most packages can be presented through viewer albeit that some errors may be generated, and good for MS products, so useful tool. Standards- implementing standards, the reality? 8 Issues (again) Confusion of ideas: Navigation v Organisation. Consider how a book is organised ( table of contents) and navigated (read). Packaging doesn’t do runtime behaviour, just provides simple view of content in terms of an organisation element. (we call this the heap of bits problem). Though this has been good enough in presenting some quality materials generated through NLN programme, this is not satisfactory. Also, re Aggregation / Disaggregation.- doesn’t handle submanifests or unique identifiers

9. Enterprise Specs (challenges and solutions) v. 1.01 problem with common interpretation of specification, and even of which version of dtd to use. (Only) 40 elements, but specification not well produced (ambiguities, ideas poorly expressed, omissions, errata). ?? no advantage over home grown systems. Of limited value without the use of extensions: extensions drive spec development but are ‘non-standard’. in practice, vendors (MIS / VLE) place premium on support for the spec in their own implementations. additional functionality desirable (see v. 1.1) but suggests costly requirements analysis particular to college or university to make the business case. Standards- implementing standards, the reality? 9 consider types of system – VLE to VLE transfers will in most cases result in loss of functionality / data. mapping roles is only approximate. Issues wrt unique identifiers / sourcedids, roles, groups relationship and group type. properties type. recstatus. extensions. conformance. data formats and convention for naming data files. validity, quality of data and handling of exceptions / errors. meaning of conformance (statements v. weak) Other issues wrt ‘data ownership’ – both in sense MIS v VLE ( should data be duplicated or how much / master slave relationship) and in terms of access (DMZs, read but not write permissions granted to VLE from MIS administrators) => Transfer protocols for updates and Security.

10. Enterprise Specs (challenges and solutions) v. 1.1 better - stronger conformance requirements*, more functional but does not appear to support xml schema as was supposed in draft release. (Advantages in use of schema as control documents v Dtd’s) negotiating profile of exchange – use of interoperability and profiling spreadsheets* Use of JDOM API ( for processing of XML docs in java aplications ) The base functionality of the Enterprise v. 1.01 specification was quite limited and the real benefits of an implementation were considered to be the added value that would result from proper requirements analysis of the business needs and processes of an institution, and use of associated extensions. Also, given the problems encountered with interpretation of the specification, it was fortunate that the partners could avail themselves of profiling tools produced through CETIS. In version 1.1, stronger conformance requirements have been introduced which use ideas drawn from such profiling tools, but which remain flexible and statements relating to disclosure are included which will aid transparency in assessing claims to conformance. Standards- implementing standards, the reality? 10 .

11. Enterprise Specs - what we did (not) do (and why) CO3 and SURF pilot were experiential (+). lay foundations for further work / actual implementation; working with 1.01 facilitate implementation of more mature specs. collaborative: insights from other partners. feedback to IMS (directly and through SIG) and UK community / dissemination. modified interface to allow import and export of enterprise xml / developed configurable XML generator to generate XML from csv files output from MIS. aims were to implement for batch enrolment and return of results. “results” depend on assessment sets. Standards- implementing standards, the reality? 11 MIS currently revising database according to Staffs University Modular Framework / COSE interoperability focused on content through X4L / future COSE version to be developed with relational database back-end / ? use of XML Schema transforms with generator

12. Learner Information Profile (LIP) Spec (challenges and solutions) Not much best practice / small but growing community to drive spec development Transcript ( good work ) and progress file ( difficult ) c. 160 elements in LIP ( a lot !) Standards- implementing standards, the reality? 12 Heterogeneity of PDP applications depending on different contexts. Schools/ colleges / university Subject specific / generic e.g. keyskills, careers service ‘ownership’ institution / tutor / student each ‘type’ could have it’s own vocabulary / xml binding Ref Centre for Recording Achievement and CETIS LIPSIG working in collaboration with other standards bodies

13. SCORM (challenges and solutions) About communications between learning object and system (VLE / LMS). specifies how a LO behaves at runtime (sequence) and allows user interaction, recording scores, tracking of sessions, personal profile and prefs. Debate round pedagogical model. May be subsumed by Learning Design in due course (18 months ? 3 years ? Longer ?) but here and now, this is all we have to play with in terms of runtime behaviour Implications for system architecture, structure of learning objects ? A number of (the bigger) vendors pay lip service only wrt / require premium for interoperable content ADL’s sample runtime environment (uses Tomcat) - ? reference implementation. US plugfests. Stuff that will run here ( Recombo outputs, Saba outputs, CAN Studios outputs ? Giunti labs ? Bb? hacked COSE content ) ( ? Find out more in Codebash 2 ***) Standards- implementing standards, the reality? 13 • ?? MS – specs reference MS style control docs as well as xsds and dtds. LRN toolkit 3.0 on metadata, CP and SCORM – good for MS products => Learnwise?, tekniCAL inter alia. Useful perhaps in propagating the idea of interoperability, but not faithful to spirit of standards development ??. . • *** Colleg Sigar report, Reload tool, Imminent release of new (conformant) versions of TekniCAL and Learnwise learning environments

14. c. Conclusions and main lessons learned What you want to know .. do they work / can they work? Standards- implementing standards, the reality? 14 they can, but I have to give you the long answer [excuse that a number of these are common sense observations] =>

15. c. Conclusions and main lessons learned • Paraphrasing from CO3 executive summary [ endorses S3 / JISC programmes report ] • There are a number of misconceptions regarding interoperability and the use of specifications .. the report is to help dispel such misconceptions by explaining with justification, what degree of interoperability might be expected and how. • Specifications are not standards, although the two terms are often mistakenly, and sometimes deliberately, used interchangeably. Specifications evolve and may eventually become standards, but this process takes years through the interactions of specification consortia, user communities, markets, test bed implementations and standards bodies. • Interteroperability applies in a particular context. Claims of interoperability are only meaningful in stating in what sense and with what particular application or system a given application or system is interoperable. [Systems do not (necessarily / usually) attempt to address all aspects of interoperability .] • Interoperability and Proprietary Solutions - the distinction should also be made between customized (perhaps proprietary) solutions to interoperability and an open standards approach. Use of specifications is not essential to achieve interoperability. In the “Building MLEs in Higher Education (HE) 7/99” programme, a number of projects within the programme took the view that achieving an integrated MLE is the principal goal and modifications to accommodate IMS specifications might be introduced at a later stage. • Interoperability, in its wider sense, means that a greater range of component systems from different vendors can be made to work together through open standards .. “A lack of open standards results in a fragmented market for education products, reducing choice and locking users into proprietary system”. Standards- implementing standards, the reality? 15

16. c. Conclusions and main lessons learned • In respect of managing expectations, the idea behind the first specifications was not that they should attempt to address all possible requirements but to focus on producing a specification that would be useful, if in a limited way, and according to relatively short timelines. Significant progress has been achieved in recent years .. Practice is catching up with the hype therefore, but many issues remain outstanding, and in that light, the IMS web site banner “making learning technology interoperable” is not wholly accurate.. • Given the status quo, what is the value of the latest specifications, which represent the maturation of the ‘first generation ’specifications? Some specifications proved to be more useful or easily usable than others and each should be considered on its own merits (amongst those tested, next point), but significant issues apply across all the specifications (below, relating to conformance, extensibility and backward compatibility). • It is helpful to draw a distinction between content-related specifications (Metadata and Content Packaging) and enterprise related specifications (Enterprise and LIP). Firstly, because the content related specifications use concepts that tend to be less ambiguous / they tend to be less problematic in terms of interpretation. Secondly, in considering the benefits that could be expected from implementing these specification sets. [In a recent presentation, a major player in the e-learning market indicated that hundreds of millions of pounds would be spent on content in the next few years, including an estimated 10 - 15 % of that sum directed at re-engineering content and testing. There are then powerful incentives for applying specifications to reduce those costs and apply resources more efficiently. The apparent gap in the specification set required for content interoperability relating to runtime behaviour has been some way resolved since IMS has worked closely with ADL in producing SCORM. Though it has been reported by a leading U.S. market analysis consultancy that similar sums would be spent on leveraging Internet information using new Enterprise Information Integration (EII) technology, by 2003, enterprise related specifications do not seem mature enough to impact in that sense]. Standards- implementing standards, the reality? 16

17. c. Conclusions and main lessons learned • Conformance: A feature of all the specifications is that conformance requirements are given, applying at different levels, but these can be written in very loose way and interpreted flexibly. Further, there is no conformance testing at present, nor would conformance guarantee interoperability. If there is one single area where the specifications should be upgraded, it is in the application of conformance and interoperability testing. Claims to conformance and interoperability need to be certificated in some way, and there are now plans are in progress to establish conformance authorities in a number of countries. In the meantime, conformance really means what can be agreed between partners and in communities of best practice: partners need to negotiate the profile of exchange of data or content transferred between systems and disseminate best practice • Extensibility is another feature built into all specifications, providing flexibility and enabling implementors to add features above the base level functionality. There is no requirement for a conformant system to process extensions, however. Extensions, whilst pushing the specification development process forwards, can, at the same time, compromise interoperability. • Backward Compatibility and hence durability, of versions of the individual specifications is also an issue. Whilst it may be generally true that point release versions of any particular specification do not change radically, there are significant costs in maintenance, and though the underlying data models may be maintained, there is no guarantee of this between the first and the second generation specifications that are likely to emerge in around two years time. The question then is - is it possible to stabilize on a usable set of the current specifications for a reasonable period of time (before the second generation emerge)? Despite the limitations of the specifications, there is an emerging body of opinion that this is the case. If this view were widely adopted across the UK HE/FE sector, current specifications would be of benefit even if they were later, possibly gradually, superceded. Standards- implementing standards, the reality? 17

18. c. Conclusions and main lessons learned • The experience gained from using the specifications should make the process of updating to newer versions easier and should also help in moving from first to second generation specifications. Implementation of the specifications and the resulting feedback to IMS is essential in establishing best practice and allows individual practitioners to input into the specification development process. In the context of early versions of certain specifications, associated costs in implementation and upgrading are appreciable, and these concerns have been expressed particularly from smaller players in the market. In respect of the CO3 Project, being wise with hindsight, it is clear that the project plan was too ambitious and underestimated the problems associated with implementation. • Without wishing to present an unduly critical view of the specifications, it is fair to say that they are not easy to work with and offer only a framework for interoperability. It has been suggested with some authority that the specifications could be considered as taking practitioners 70% of the way toward achieving interoperability. This might seem a modest achievement, but the real problem remains one of expectation management and the specifications are without doubt the best immediate prospect for achieving interoperability in its wider sense. • Specifications will continue to evolve, considering their expanding scope, harmonization, relation to other specifications, changes in technology, models of education, learner profiles and constituencies etc..Best practice can gradually become established through the efforts of the industry and a small but significant number of champions, and more tools are becoming available which will help this process. The main challenge lies not in technical implementation but in institutional change management in supporting these developments. Standards- implementing standards, the reality? 18

19. It’s a journey, but .. a. we’ll get there in the end b. it will be worth it c. staged approach is best d. gone too far to go back now e. there’s no alternative f. best option for interoperability g. I don’t want to go there yet h. I don’t want to go there Standards- implementing standards, the reality? 19 d. References C03 report itself, briefing papers and related references at http://www.staffs.ac.uk/COSE/ims_briefs.html The Managed Learning Environments Steering Group Reports [Mar. '02 Link to JISC Summary Report, Technical Report and individual pilot project reports] MAISIE Center Report - Making Sense of Learning Specifications & Standards [Mar. '02 S3 (from title) Guide from US with advice on what specifications mean in practice] Enterprise 1.1 [May 2002 From CETIS web site, describing new Enterprise specification with stronger conformance statements] Teach Yourself Scorm 1.2 online course @ http://www.tamucc.edu/~scorm/ Educause Extra [October 2001 From IMS web site, with an overview of IMS working groups and accomplishments from IMS CEO]

20. e. perspective : that was a year ago. what’s happened since CO3 project ended c Dec 2001 but wasn’t written up until 6 months later => that was a year ago. what’s happened since. Standards- implementing standards, the reality? 20 X4L and FAIR programmes. JISC 1/01 and SWANI. - Specs are starting to be adopted as standards IEEE LOM / SCORM (Harmonisation with IEEE binding for LOM. UK CMF). - Specs for Simple Sequencing, Learning Design, Digital Repositories Accessibility, maintenance release of CP v. 1.1.3. IMS stream-lined and new spec development process. Second generation specs. Collaboration with MIT in OKI. - Technology evolves. Web services. Specs previous focus was on data not behaviour / processes

21. e. perspective (continued) growing communities / constituencies. widespread adoption of VLEs including in schools now. user groups and best practise. portals open source tools RELOAD / COSE / ADL’s Sample RTE and repositories are being established *** ( see slides to follow ) challenging vendors ( ? to do more than pay lip service to standards) change management (?) still the main issue Standards- implementing standards, the reality? 21 - changes in further and higher education; students expectations are greater now they pay more in fees; are academic staff and administrators any more persuaded of advantages of managed learning environments; are these changes being enforced through competition? Oxford debate // VLE mailing list

22. SURF X4L Project - progress Most of the technical effort has been in investigating various tools and systems and in writing code to enable COSE content to be exported as standalone content. Content produced in the COSE VLE can now be saved to the hard drive as specification conformant content packages, and need only a javascript enabled browser for navigation and viewing. Such packages can be viewed in the RELOAD application, can be uploaded to the JORUM Intralibrary repository, and, as we have demonstrated using the ADL Sample RTE, with some modifications it should be possible to run the content in a SCORM conformant environment. Standards- implementing standards, the reality? 22 See the screenshots below [excuse quality of these], and / or please ask for a demo to learn more about our work. There is still much work to do, but the first phase of the project has been quite encouraging. Priorities now will be in addressing the problems in structuring the content for aggregation and disaggregation to facilitate re-use, to investigate changes to the system architecture to further progress with SCORM, and to do more work in exchanging content with the proprietary VLEs (Blackboard, Learnwise and TekniCAL) as used between project partners.

23. Figure showing standalone COSE content package loaded into RELOAD Standards- implementing standards, the reality? 23

24. Figure showing standalone COSE content package loaded in the RELOAD viewer Standards- implementing standards, the reality? 24

25. Figure showing standalone COSE content packages uploaded into Intralibrary Standards- implementing standards, the reality? 25

26. Figure showing standalone COSE content package in the Intralibrary viewer Standards- implementing standards, the reality? 26

27. Figure showing standalone COSE content package in the ADL SCORM RTE Standards- implementing standards, the reality? 27