Creating and using online assessments to improve training effectiveness

1. Creating and using online assessments to improve training effectiveness Tom Blodgett and Allen Woods TBA Resources, Inc. Created by Martin Belton martin.belton@qmark.co.uk June 2001

2. Introduction • Computers have revolutionised training • More people are being trained • More subjects to train • More methods of delivering training • Internet/intranet have prevented distance becoming the problem

3. Introduction ‘...industry is spending billions and billions on training programs and doing no evaluation of the results’ - Prof. Gary Cheeriness, Rutgers

4. Introduction • Too much training is ‘throw it against the wall and hope some of it sticks’ • Ineffective • Costly • CEOs won’t be impressed by the ‘let’s try it again’ argument for ever • Assessment validates the effectiveness of training, to standardize best practices

5. Why you need training assessment • Assessment enables us to discover: • Whether training is needed • What training is needed • What an individual has mastered • What has been absorbed by a group • Whether the investment has payback • What can be done to improve the training

6. Pre course Assessment LMS Database Learning object 1 Learning object 2 Post course Assessment 2 Post course Assessment 1 Learning object 3 Learning object 4 Improving training efficiency

7. Today’s training methods • Training or instructing by computer is no different from any other style of training • Formal process to help employees learn • A coordinated set of activities leading to change in employee behavior • Goals remain the same: • Instruction results in employee learning • Employees use new knowledge to address challenges

8. Perception Software • Creates questions, tests, using Windows applications (Question Manager and Session Manager) • Runs assessments online, in a secure environment (Security Manager) • Creates reports to analyze participant’s data (Enterprise Reporter)

9. Questionmark PerceptionThe Big Picture 4. Report on the results. Reports include coaching reports, item analysis, gap analysis, and others. 1. Create questions and group into quizzes, tests, exams and surveys using a web browser or Windows PC Deliver assessment using a standard, or secure, browser, or using a Windows PC 2. Specify security parameters, schedule assessments and potentially link to other systems

10. Applications in the Big Picture Windows Based Windows Based Enterprise Manager Browser Based Browser Based Secure Browser Enterprise Manager Browser Based Browser Based PC Based

11. Databases/TablesUnder the Hood Report Formats Questions Results AssessmentDefinitions Self-Registration Names & Passwords Credit Card Schedules

12. ParticipantResults ItemResults Inet InternetIntranetInternal Network Questions Questions AssessmentDefinitions AssessmentDefinitions Perception System Diagram Publish BBA WBA Reporting on Results Participants Answering Assessments Authoring Assessments

13. Fits the MeadWestvaco enterprise • Perception is AICC compliant • offers integration with leading Learning Management Systems (LMS) • Aggregates assessment results into a single database to demonstrate employee progress • Linked/AICC systems include: • WBT Manager, Docent, Registrar, Tecnikal, SABA, Web CT, Thinq, Fretwell Downing Le

14. Fits the enterprise (cont’d) • Perception questions are IMS compatible • IMS is a global coalition of around 250 technology vendors/institutions • XML-based standards enabling learning providers to re-use training resources among different online learning systems • Questions are exportable and “future-proof” • Opportunity to incorporate additional assessments into your learning solutions