1 / 12

Formative Evaluation of IAT Services Computing Sites Consultant Training Materials

Formative Evaluation of IAT Services Computing Sites Consultant Training Materials. Hsing-Liang Lin & Pei-Ju Liu March 11, 2003. Introduction.

kolina
Download Presentation

Formative Evaluation of IAT Services Computing Sites Consultant Training Materials

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Formative Evaluation of IAT Services Computing Sites Consultant Training Materials Hsing-Liang Lin & Pei-Ju Liu March 11, 2003

  2. Introduction • The goal of IATS computing sites consultant training is to offer consultants to provide excellent customer service, assisting customers with a wide variety of questions. • The purpose of the orientation is to help the consultants to obtain the necessary knowledge and skills to perform in his or her job. The orientation is taken for six hours totally. • The instructional material of this training includes seven parts: Site Operations, PC, MAC, SGI, Peripherals, Adaptive, and WebCT.

  3. Overview of the instructional materials • 1. Objective • The learner will be able to know the IAT service work team and his role. • The learner will understand all rules and policies he should follow. • The learner will be able to fulfill his job expectations and guidelines. • The learner will be able to login to CSIS and manage his work shift. • The learner will have the appropriate knowledge and skills to use most hardware and software in computing sites. • The learner will be able to help customers to solve problems or look for other solutions.

  4. 2. Audience • the IAT service consultants • 3. Status • The material is now using and the latest version was revised on Jan 10, 2003. • 4. Media Format • print based • 5. URL • http://iatsites.missouri.edu

  5. Formative Evaluation Framework • Method: • Interview • Questionnaires • Debriefing section • Observation • Interviewees • One Expert • Two Users • Data Analysis • Recommendations • Self Evaluation

  6. Expert Review • A subject sophisticates expert • Mr. Thitinun Boonseng, a site support specialist in IATS • Comments: • The material is poor in index. • There are still few errors in it. • Need more graphics and information to enhance explanations.

  7. User S Evaluation • A medium-level user. • A student in university of Missouri. • Motivation: • He is very interesting in the consultant job in IATS. • Comments: • The length of lines is too long to read. • Only few graphics were used. • No more sub-title is available for index.

  8. User H Evaluation • A low-level user. • A student in university of Missouri. • Motivation: • She wants to have a part time job in IATS next semester. • Comments: • Need a table for the whole items listed. • Need more graphics to support the content.

  9. Recommended Revisions • Add an index or content table to indicate more specific topics within the material. • Use more graphics or screenshots to enhance text explanations. • Revise the content formatting • Update the latest contents. • Some content need more sufficient explanations • Use consistent phraseology throughout in the material.

  10. Revisions Arguments • The material structure based on the frequency?? • Expert: the content should base on how often it is used • Our opinion: Both frequency and difficult levels should be considered. • Our Recommendation: Use “Top Questions Index” indicating where consultants can directly and quickly • Web or hard copy?? • Expert: hard copy • Both Users: Web-based System • Our recommendation: hard copy, because the procedures of using a web system might be more complex than opening hard copy.

  11. Self Evaluation-- Strengths • Different evaluation forms between expert and users. • For expert, we don’t want to restrict his thoughts and we would like to obtain more practical and detail comments rather than the rating scales. • For users, the form is more like a guideline helping them to check those contents and be able to deeply think and judge the material. • (format and content please check attachments) • User representative • The users we selected both have great motivation and similar background with the potential target population so that their opinions and thoughts are much closer target audiences.

  12. Self Evaluation-- Improvements • Some user debriefing questions are inappropriate • Q5. What is least valuable about this material? • Q6. What is most valuable about this material? • Time & Material representative • To reduce this bias, using more users to evaluate different lessons will be helpful. If we use this method, not only we are able to gather data through all contents and don’t miss observation and interaction information but also users don’t need to spend too much time in completing the evaluation.

More Related