Training and Instructional Design

1. Training and Instructional Design Unit 2: Needs Analysis Lecture b This material (Comp20_Unit2b) was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC000003. This material was updated by Columbia University under Award Number 90WT0004. This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/.

2. Needs Analysis Learning Objectives • Objective 1: Identify an instructional problem • Objective 2: Plan and implement an instructional needs assessment • Objective 3: Analyze learner, task, and situational characteristics • Objective 4: Compare the methods used to collect data in a needs assessment • Objective 5: Select appropriate data collection methods to meet different needs analysis situations

3. ADDIE 2.15 Figure (Hall, M.V. and Zimmerman, J., 2012).

4. Data collection • The needs analysis stage of your training is likely to be successful if quality methods are used to collect data • Different workplace and training situations require different data collection methods

5. Strategy 2.16 Figure (Hall, M.V. and Zimmerman, J., 2012).

6. Example of identifying the purpose and audience • Organizational analysis: • Need information about organizational goals, available resources, constraints, and support • By assessing the attitudes and perceptions of staff and clinicians, and reviewing company documents • Audience to target: • Senior clinicians who can speak accurately about the organization’s goals • All employees? A cross – section of the workforce to get a holistic view

7. Data collection methods • Survey • Interview • Focus / Discussion group • Observation • Document review

8. Description and purpose of surveys • A way to get lots of information quickly and / or easily from a group of people individually • Can be in – person, written, or by telephone

9. Advantages and disadvantages of surveys • Advantages • Can reach large numbers of people • Questions can focus responses as either very specific or open – ended • Can establish relationships among indicators • Can complete anonymously • Inexpensive to administer • Easy to compare and analyze • Many sample questionnaires already exist • Disadvantages • No in – depth responses on issues • Does not allow respondents to exchange ideas • Limited to specific questions • May not reach people who are hard to reach • Wording can bias clients’ responses • Are impersonal • May not get full story

10. Survey writing tips • Use simple language • Make the survey short • Avoid double negatives • Avoid open – ended questions • Do not write leading questions • Avoid double – barreled questions • Offer an “out” for questions that do not apply

11. Description and purpose of interviews • Formal or informal discussion with an individual to gather in – depth information on specific topics • When wanting to fully understand someone’s impressions or experiences, or learn more about their answers to questionnaires

12. Advantages and disadvantages of interviews • Advantages • Can get in – depth information • Questions can be flexible and be tailored to each person • Can get spontaneous responses • Can get honest opinions, individual perspective • Can get to questions that people are uncomfortable answering in a group • Can get ideas or opinions that never occurred to you • Can be done in a setting familiar to participant • Disadvantages • Time consuming to administer • Interviewers needed and can be costly • Interviewer can bias clients’ responses • Not efficient for reaching a large number of people • Not useful if trying to get answers to specific questions from people or get consistent answers

13. Interview guide 2.1 Table (Hall, M.V. and Zimmerman, J., 2012).

14. Description and purpose of focus groups • Small group discussions around a set of questions guided by a moderator • To explore a new topic, get in – depth knowledge, ask questions about “how and why,” understand a gap in understanding between groups of people, or prepare for a survey

15. Advantages and disadvantages of focus groups • Advantages • Participants can build on each other’s comments • Can be quicker than a survey • Can be less costly • Can reach more people than interviews • Can hear a variety of ideas and opinions • Participants can take advantage of the shared experience • Disadvantages • Need a good facilitator for safety and closure • Difficult to schedule six to eight people together • A group or an individual can dominate the session • Information gathered may not represent the larger community • Smaller numbers reached • Can be hard to analyze responses

16. Description and purposes of observation • Observer records an expert performing job in natural or structured setting • To gather accurate information about how a program actually operates, particularly about processes

17. Advantages and disadvantages of observation • Advantages • Can view operations of a program as they are actually occurring • Can adapt to events as they occur • Disadvantages • Can be difficult to train observers • Can be difficult to interpret seen behaviors • Can be complex to categorize observations • Can influence behaviors of program participants • Can be expensive

18. Description and purpose of document review • When want impression of how program operates without interrupting the program • Analyze budget documents, quality control documents, goal statements, evaluation reports, scheduling and staffing reports, or other documents for existing problems, such as testimonials from other organizations

19. Advantages and disadvantages of document review • Advantages • Can get comprehensive and historical information • Does not interrupt program or client’s routine in program • Information already exists • Disadvantages • Often takes a lot of time • Information may be incomplete • Need to be clear about what looking for • Not a flexible means to get data • Data restricted to what already exists

20. Pilot test your instrument • Pilot test if possible • Identify and eliminate problems that might occur • Keep practice exercises realistic • Try out survey or interview tool on a few persons • Pilot test results • Make changes prior to actual data collection • If changes are drastic, pilot test again and / or re – train data collectors

21. Guidelines for implementation • Keep survey instruments short, one or two pages, if possible • Provide directions on how to return the survey and prepare online surveys if possible • Always let people know how you will use the data • Always find a way to report the information back to the participants • Offer incentives, such as lunch for focus groups and interviews

22. Implement your instrument • Confidentiality should always be foremost in your mind • Inform respondents about your confidentiality guidelines when you begin data collection • Make surveys anonymous • Assign fictitious names to respondents or provide answers in categories rather than by respondent

23. Case example • Meaningful Use clinical quality measure: • The percentage of patients 18 years of age and older who were current smokers or tobacco users, who were seen by a practitioner during the measurement year and who received advice to quit smoking or tobacco use or whose practitioner recommend or discussed smoking or tobacco use cessation medications, methods or strategies

24. Unit 2: Needs Analysis, Summary – Lecture b • Choose the method that is most appropriate for your audience and the type of information to be collected. Ultimately, the decision of data collection method will take into account the validity of the responses collected, balanced with budget and efficiency concerns.

25. Needs Analysis References – Lecture b References: Carkhuff RR, Fisher SG. Instructional systems design: volumes I & II. Amherst, MA: Human Resource Development Press; 1984. CarlinerS. Training Design. Danvers, MA: American Society for Training and Development; 2003. Clark DR. Bloom’s Taxonomy [monograph on the Internet]. Big Dog & Little Dog’s Performance Juxtaposition; c2004 [cited 2010 Jun 21]. Available from: http://www.nwlink.com/~donclark/hrd/sat.html. Clark DR. Instructional System Design (ISD). Big Dog & Little Dog’s Performance Juxtaposition; c2004 [cited 2010 Jun 21]. Gagne RM, Wager WW, & Golas K. Principles of Instructional Design (5th ed.). California: Wadsworth Publishing, 2004. Reigeluth CM. Instructional-design Theories and Models: A new paradigm of instructional theory. New Jersey: Lawrence Erlbaum Associates, Inc, 1999.

26. Needs Analysis References – Lecture b (Cont’d – 1) Charts, Tables and Figures: 2.17 Figure: Hall, M.V. and Zimmerman, J., 2012 2.18 Figure: Hall, M.V. and Zimmerman, J., 2012 2.1 Table: Zimmerman, J. (2010). Interview guidelines. Department of Biomedical Informatics, Columbia University Medical Center. New York, NY

27. Unit 2: Needs Analysis, Lecture b This material (Comp 20 Unit 2b) was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number IU24OC000013. This material was updated in 2016 by Columbia University under Award Number 90WT0005.