23 rd Annual Conference September 11-13, 2003 Westin Harbour Castle Toronto, Canada

1. PUSHING THE BOUNDARIES OF REGULATION 23rd Annual Conference September 11-13, 2003 Westin Harbour Castle Toronto, Canada

2. Technology, Service and Delivery September 12th, 2003 1:30 – 3:00 p.m. Dan Luk CTO, Capstar

3. First Topic:“Point and Click – Using the Internet to Enhance your Testing Program”

4. Why the Internet? • Convenient • Fast • 24 X 7 • Efficient • Cost-Effective

5. Why Not the Internet? • Security • Control • Fairness

6. Web Usage and GrowthSource http://www.nielsen-netratings.com/new.jsp on 1/8/2003

7. When to use the Internet? Pre-Test Post-Test

8. Test Development • Web-based Test Development Tools • Collaborative Computing • Item Development and Review • Resolve Geographical Differences • Security • Authentication • Access Restrictions • Encryption

9. Communications • Types of Technology • Push Technology • e.g. e-mail • Pull Technology • e.g. Web Site

10. Push Technology • Main Advantages • Selective or Targeted • Customized Contents • Concise • Main Disadvantages • “Spam” • Frequent Address Changes

11. Pull Technology • Main Advantages • General Availability • Broad-based Information • Information Available On-Demand • Consistent Messaging • Interactive Contents • Main Disadvantages • Must be Easy to Navigate • Must Visit Frequently to View New Contents • Random Arrival

12. Types of Information • Bulletins • Application Procedures • Application Forms • Study Materials • Practice Exams • Etc.

13. Registration/Scheduling • Self-Service Vs. Call Center • Accessibility • Skills • Online Help Vs. Help Desk • High Availability • Easy to Review Information or to Make Changes • Implementation Methods • Real-time • Back-end Processed

14. Registration/Scheduling (Continued) • Business Rule Enforcement • Upper Vs. Lower Case • Data Consistency Vs. Data Cleansing • Browser Compatibility • ADA Accommodation such as Section 508 Compliancy • E-commerce (Electronic Payment Required) • Identity Theft

15. Test Delivery • High Stake Vs. Low Stake • Anyone Can Test Vs. Positive ID • Proctored Vs. Self-proctored • Connection Performance • Timed Exams • Test Driver Features • Test Security • Browser Compatibility

16. Data Management • On-Demand Vs. FTP • Dynamic Vs. PDF • Registration/Scheduling Records • Eligibility Processing • Test Reporting • Psychometric Analysis

17. Capstar Internet Usage • Exceeded Expectations • Anticipate Continued Growth • Continued Area of Focus

18. Second Topic:“Innovative Uses of Technology In Testing”

19. When to use Technology? Pre-Test Post-Test

20. When to use Technology? Lie! Cheat! Steal!

21. Lie • Whom Are You Really Testing?

22. Lie • Typical Preventive Measures • Policies and Procedures as Deterrent • Positive Identification Via Proctor

23. Traditional Identification Techniques • Government Issued Pictured Ids • ATT • Tamper Resistance Paper and Printing • Smart Cards • Holographic Images • Microprints • Bar Code • Data Strip

24. Biometrics ‘Recently, the term "Biometrics" has also been used to refer to the emerging field of technology devoted to identification of individuals using biological traits, such as those based on retinal or iris scanning, fingerprints, or face recognition.’ http://stat.tamu.edu/Biometrics/

25. Biometrics (Continued) • Access to Testing Area • Authentication Process • Re-entry to Testing Area (Same Day Vs. Future) • Re-authentication Process (Same Day Vs. Future) • Initial Testing Vs. Repeat Testing • Same Person Vs. Different Person

26. Cheat • Typical Preventive Measures • Policies and Procedures as Deterrent • Rules for Taking Breaks • Business, Eligibility or Re-take Rules • Proctored Exams • Clean Test Rooms

27. Traditional Monitoring Techniques • Active Proctoring • Proctor Window • Video and Audio Surveillance

28. Other Monitoring Techniques • Remote Proctoring • Remote Video and Audio Surveillance • Back-end Monitoring • Extreme Low Score • Extreme High Score • Sudden Score Fluctuation • Time-to-respond • Item Performance • Form/Test Performance • DB Cross-checking

29. Other Monitoring Techniques • Biometrics • Candidate’s Photo • Access to Testing Area • Authentication Process • Re-entry to testing area • Re-Authentication Process

30. Steal • Reasons for Prevention to Include: • Intellectual Property Protection • Exam Integrity • Public Safety

31. Steal • Typical Preventive Measures • Policies and Procedures • Business, Eligibility or Re-take Rules • Rules for Taking Breaks • Active Proctoring • Proctor Window • Video and Audio Surveillance

32. Other Techniques • Remote Proctoring • Remote Video and Audio Surveillance • Back-end Monitoring • Extreme Low Score • Extreme High Score • Sudden Score Fluctuation • Time-to-respond • Item Performance • Form/Test Performance • DB Cross-checking

33. Other Techniques (Continued) • Initial Testing Vs. Repeat Testing • Same Person Vs. Different Person • New Items • New Forms • Continuous Testing Vs. Windowed Testing • Software Algorithms • Randomization • Computer Adaptive • Item Algorithms

34. Trade-off Security Cost

35. Other Technologies Used in Testing • Improved Paper-and-pencil Process • Dynamic Scanning and Scoring • Fax-based Scoring • Simulation-based Testing • Innovative Item Types • Computer-based Practical Testing • 3-D Display

36. Other Technologies Used in Testing (Continued) • Standards • SCORM • IMS QTI • Other Testing Devices • PDAs • Tablet PCs • IVR/Cell Phones • Alternate Devices • Wireless Technologies

37. Q & A

38. Presentation Follow-up • Please pick up a handout from this presentation –AND/OR- • Please give me your business card to receive an e-mail of the presentation materials –OR- • Presentation materials will be posted on CLEAR’s website

39. Technology, Service and Delivery September 12th, 2003 1:30 – 3:00 p.m. Dan Luk CTO, Capstar