Leveraging the Internet in Exam Development and Delivery

1. Expect the Unexpected: Are We Clearly Prepared? Leveraging the Internet in Exam Development and Delivery Tony Zara, PhD Ho J. Mun Pearson VUE / Promissor Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

2. Agenda • Introductions • Exam Development • Requirements for virtual meetings • Guidelines for success • Disadvantages/advantages of virtual meetings • Regulatory client experiences • Examination Delivery • Credentialing Process • Discussion • The next big thing • Conclusions Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

3. Exam Development • How to coordinate geographically diverse panels committees, and/or employees? • High cost of travel • Time involved in travel • Multiple schedules to coordinate • Jet lag Sounds like an opportunity for virtual meetings Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

4. Requirements for Virtual Meetings • Need to provide an experience more rich than phone/email • Evaluate each meeting type for appropriateness (e.g., training, item review, standard setting, job analysis, item writing etc.) • Must evaluate capabilities of different software/systems • Bandwidth • Sharing applications • Messaging (public/private) • Plug-ins • Security • Asynchronous viewing • Firewall issues • Host/guest connectivity Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

5. Running Successful Virtual Meetings • Allow extra time • Decide in advance on the format for each part of the presentation (self-directed vs. host-controlled) • Participants login ahead of time • Have a wingman – multiple hosts allow you to deal with participants individually • Distribute necessary print materials (e.g., content outlines, checklists) prior to meeting • Review the different software options • Ensure support is readily available If this is your first time, conduct a dry run internally Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

6. Advantages of Virtual Meetings • Lower costs • Faster turnaround • Shorter, more focused meetings • Access to wider range of experts (allows for geographic diversity) • May allow small clients to perform work that they otherwise couldn’t afford • May allow side conversations (chat/IM) that are less disruptive to the whole group • Easier to include large groups for training Supplement – not replacement for – face to face (FTF) meetings; also allows for FTF meetings to be more productive Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

7. Disadvantages of Virtual Meetings • Technology issues • Lack of face time • No body language/ nonverbal cues • Participants “zoning-out”/multi-tasking • Travel perk for participants Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

8. Client Experiences • Benefiting Industries • Insurance departments • Real Estate commissions • Appraisers • Nurse Aide boards • Contractor boards • Certification clients Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

9. Examination Delivery • Types of computer-based testing • Web-based testing • Browser-based (connected) • Internet-based (disconnected) • Center-based testing • Third party sites • Private sites • Owned/operated sites • Independent testing Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

10. Web-based Testing • Browser-based • Live, connected testing across the Internet • Either center-based or independent • No standardization of testing stations • Internet performance issues • Internet-based • Internet used to distribute testing information • Center-based, disconnected • Client-server testing center model • Standardization of testing stations possible Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

11. Center-based Testing • Third party sites • Can deliver web-based or internet-based testing • Standardized procedures • Proctoring • Private sites • Corporate environments • Owned/operated sites • Most standardized environment • Optimized for testing • Professional motivated staffing Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

12. Independent Testing • Generally browser-based • Broad distribution of testing • Standardization issues • Security challenges • Unproctored delivery • Examinee identification • Testing intellectual property • Inexpensive assessment events Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

13. Appropriate Applications • Independent testing • Connected, browser-based • Low-stakes assessments • Diagnostic assessments • Practice testing • Center-based testing • Disconnected, client server • High-stakes examinations (licensure, certification, academic) • High-complexity examinations Use the tool appropriate for the assessment situation Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

14. Credentialing Process • Enhanced communications • Eligibility process • Data transfer • Results processing • State systems Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

15. Discussion The Next Big Thing? Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

16. Conclusions • As with all technological advances, the internet is here to stay in testing • The Internet is a useful tool for many contexts • Just because you have a hammer, not all problems are nails Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

17. Anthony R. Zara, Ph.D. Pearson VUE 952-681-3811 tony.zara@pearson.com www.pearsonvue.com Ho J. Mun Promissor 610-617-5148 ho.mun@pearson.com www.Promissor.com Contact Information Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia