170 likes | 278 Views
Expect the Unexpected: Are We Clearly Prepared?. Leveraging the Internet in Exam Development and Delivery. Tony Zara, PhD Ho J. Mun Pearson VUE / Promissor. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia. Agenda. Introductions Exam Development
E N D
Expect the Unexpected: Are We Clearly Prepared? Leveraging the Internet in Exam Development and Delivery Tony Zara, PhD Ho J. Mun Pearson VUE / Promissor Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Agenda • Introductions • Exam Development • Requirements for virtual meetings • Guidelines for success • Disadvantages/advantages of virtual meetings • Regulatory client experiences • Examination Delivery • Credentialing Process • Discussion • The next big thing • Conclusions Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Exam Development • How to coordinate geographically diverse panels committees, and/or employees? • High cost of travel • Time involved in travel • Multiple schedules to coordinate • Jet lag Sounds like an opportunity for virtual meetings Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Requirements for Virtual Meetings • Need to provide an experience more rich than phone/email • Evaluate each meeting type for appropriateness (e.g., training, item review, standard setting, job analysis, item writing etc.) • Must evaluate capabilities of different software/systems • Bandwidth • Sharing applications • Messaging (public/private) • Plug-ins • Security • Asynchronous viewing • Firewall issues • Host/guest connectivity Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Running Successful Virtual Meetings • Allow extra time • Decide in advance on the format for each part of the presentation (self-directed vs. host-controlled) • Participants login ahead of time • Have a wingman – multiple hosts allow you to deal with participants individually • Distribute necessary print materials (e.g., content outlines, checklists) prior to meeting • Review the different software options • Ensure support is readily available If this is your first time, conduct a dry run internally Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Advantages of Virtual Meetings • Lower costs • Faster turnaround • Shorter, more focused meetings • Access to wider range of experts (allows for geographic diversity) • May allow small clients to perform work that they otherwise couldn’t afford • May allow side conversations (chat/IM) that are less disruptive to the whole group • Easier to include large groups for training Supplement – not replacement for – face to face (FTF) meetings; also allows for FTF meetings to be more productive Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Disadvantages of Virtual Meetings • Technology issues • Lack of face time • No body language/ nonverbal cues • Participants “zoning-out”/multi-tasking • Travel perk for participants Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Client Experiences • Benefiting Industries • Insurance departments • Real Estate commissions • Appraisers • Nurse Aide boards • Contractor boards • Certification clients Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Examination Delivery • Types of computer-based testing • Web-based testing • Browser-based (connected) • Internet-based (disconnected) • Center-based testing • Third party sites • Private sites • Owned/operated sites • Independent testing Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Web-based Testing • Browser-based • Live, connected testing across the Internet • Either center-based or independent • No standardization of testing stations • Internet performance issues • Internet-based • Internet used to distribute testing information • Center-based, disconnected • Client-server testing center model • Standardization of testing stations possible Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Center-based Testing • Third party sites • Can deliver web-based or internet-based testing • Standardized procedures • Proctoring • Private sites • Corporate environments • Owned/operated sites • Most standardized environment • Optimized for testing • Professional motivated staffing Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Independent Testing • Generally browser-based • Broad distribution of testing • Standardization issues • Security challenges • Unproctored delivery • Examinee identification • Testing intellectual property • Inexpensive assessment events Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Appropriate Applications • Independent testing • Connected, browser-based • Low-stakes assessments • Diagnostic assessments • Practice testing • Center-based testing • Disconnected, client server • High-stakes examinations (licensure, certification, academic) • High-complexity examinations Use the tool appropriate for the assessment situation Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Credentialing Process • Enhanced communications • Eligibility process • Data transfer • Results processing • State systems Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Discussion The Next Big Thing? Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Conclusions • As with all technological advances, the internet is here to stay in testing • The Internet is a useful tool for many contexts • Just because you have a hammer, not all problems are nails Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia
Anthony R. Zara, Ph.D. Pearson VUE 952-681-3811 tony.zara@pearson.com www.pearsonvue.com Ho J. Mun Promissor 610-617-5148 ho.mun@pearson.com www.Promissor.com Contact Information Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia