200 likes | 388 Views
Implementing large scale and rapid growth assessment programmes Suzana Lopes Commercial Director. Presentation Outline. Pearson Company Overview Case Studies: Large Scale Assessment Programme DSA – Driving Standards Agency (UK) DVTA – Driver and Vehicle Testing Agency (Northern Ireland)
E N D
Implementing large scale and rapid growth assessment programmesSuzana LopesCommercial Director CAA Conference
Presentation Outline • Pearson Company Overview • Case Studies: • Large Scale Assessment Programme • DSA – Driving Standards Agency (UK) • DVTA – Driver and Vehicle Testing Agency (Northern Ireland) • Rapid Growth Assessment Programme • City and Guilds CAA Conference
Pearson Company Overview Pearson plc £4 billion annual sales World’s # 1 education business Over 30,000 employees Pearson VUE - The UK Market Leader in e-Assessment Acquired Promissor in January 2006 to expand assessment service Flexible and robust content development and delivery solutions Test centres in 145 countries and easy to deploy customer site testing A commitment to quality and outstanding service CAA Conference
Case Study: The Driving Theory Test • Contract awarded 31 October 2003 by Joint Authorities • Our contract is for delivery of theory tests • Nearly 1.8 million tests delivered annually • Learner tests for: • Car • Motorcycle • Truck • Buses • Driving instructor tests • Driving examiner tests CAA Conference
What needed to be achieved • Establish a contact centre to handle: • Telephone bookings • Mail and fax bookings • Build DDA compliant test centres at 151 locations • England • Scotland • Northern Ireland • Wales • Build and publish the tests • Develop bespoke booking and reporting systems • Real-time eligibility checks • Hire, train and transfer staff from previous vendor CAA Conference
What Needed To Be Achieved - The Timeframe CAA Conference
Test Centres • Requirement: 80% of the UK population to be no more than 20 miles from their nearest test centre • Difficult to find DDA-compliant (accessible) locations in some areas • Northern Scotland was particularly a problem • Decision was made to use coaches as mobile test centres • Eventually found locations for 151 test centres • Space that worked with the test centre plans • Safe and appropriate • Easily accessible by public transport • Flexible opening hours CAA Conference
Publishing The Tests • Requirement:Publish tests in English and Welsh - with voice • overs in 20 languages and available in British Sign Language • First understand the presentation and scoring requirements • Multiple choice and hazard perception components • Thousands of the items built into a variety of tests • Multiple forms of each test • Hundreds of video clips for HPT test with specialized scoring • Inventory the items and materials carefully • Identify missing components • Build the tests • Test, test and test CAA Conference
Test Centre Design and Test Delivery • Requirement: Design highly secure test centre with invigilator station • CCTV and audio monitoring and biometrics during check in process • Produced working prototype of test centre meeting all requirements • Opened on 1 March 2004 • Loaded Hazard Perception Test video clips on 30 stations to demonstrate smooth operation • Test registration and scheduling • Standardised procedures at the centres • Hired contractors to kit out 151 test centres • and mobile centres • Hired and trained staff CAA Conference
Example Test Centre Floor Plan CAA Conference
Critical Success Factors • Excellent Project Management • Working groups • Software • Testing centres • Contact centre • Test publishing • Security • Senior business group • Experienced test development and delivery staff • Members from Pearson VUE, DSA and DVTA • Weekly update meetings CAA Conference
Rapid Growth Assessment Case Study: City & Guilds • 125 year old not for profit company • Royal Charter • UK’s largest vocational awarding body • 1.2 million learners • 8500 centres • 600 awards • 102 Countries • Operating in 22 sectors CAA Conference
Overview • 2001 City & Guilds decided to launch assessment and learning portal • Plan to move centres to online registration and booking and develop computer based tests • Global search for test partner • Detailed specification • Reliability, scalability and flexibility ranked highly among the attributes required • 6 companies competed • Promissor awarded the contract in 2002 CAA Conference
Attributes Of The Market • Almost entirely paper based. • Candidate choice was limited but growing • Centres were cost focussed • Constant policy and funding changes • Backdrop of slow uptake of on line services • across industry CAA Conference
Test Development • Deployment team established • CBT item writers appointed by City and Guilds • Training in Promissor Builder • Easy to Use – non expert interface • Web based tool – collaborative • Track and view content changes • Store multiple classification schemes • Create tests based on balanced content criteria • Allows randomized tests, test pooled from an item pool and adaptive test CAA Conference
Test Delivery • Customers consulted about design of system • Regional training events held for customers • Online training provided to use registration system • Documentation provided for surveillance and invigilation at centre • Technical support line • Customer service support Promissor provide Centre Criteria Centre Approval Test Centre Installation Test Centre Demo Test Promissor make LIVE CAA Conference
BUILDER ItemBank Test Centre Set-up Test Published for Delivery Test Centre www.catglobal.com Booking (ENTERPRISE) SITE MANAGER Test Administrator Test Administrator Test Administrator Solution Overview Candidate Registration C&G System Registration www.catglobal.com Scheduling Results C&G System Candidates scheduled on-line Candidate Checks-in Candidate Takes Test CAA Conference
Rapid Growth • 2002: 200 tests delivered • 2003: Plan to move 80% of objective assessment to CBT • 65k tests delivered • 2004: 187k tests delivered • 2006 • 100k test per month • Over 250 computer based tests developed • 6000 registered centres CAA Conference
Critical Success Factors • Senior management support • Efficiency and value has to be proven internally as well as externally • On-screen must be mandatory • Technology right first time • Training must be easily available • Extensive support is needed CAA Conference