Bureau of Medicine & Surgery Phases of Curriculum Design: Implementation

1. Bureau of Medicine & SurgeryPhases of Curriculum Design: Implementation Click to continue.

2. IN THIS MODULE: 1. Steps in the implementation phase. 2. Overview of developing a validation plan. 3. Overview of developing a teaching schedule. 4. Elements to include in staff orientation. 5. Potential problem areas on first implementation. 6. Outputs of the implementation phase.

3. IMPLEMENTATION Analysis In the implementation phase, you conduct the course and carry out a validation plan to evaluate how well your methods and materials work in practice. As in previous phases, you may find it necessary to go back and modify the output from the development phase, based on the results of your validation study. Design Development Implementation Evaluation / Maintenance

4. IMPLEMENTATION Steps in the implementation phase include the following: 1. Develop an implementation plan, including a validation plan and a staff orientation plan. 2. Develop the teaching schedule. 3. Obtain equipment and supplies (not covered in this module). 4. Arrange materials to be issued to students (not covered in this module). 5. Conduct the course. 6. Carry out the validation plan.

5. Expectation Reality Validation Plan In the validation plan, you specify how you will determine whether or not the course and its components worked. The validation plan should address both the course as a whole and the individual lessons and units within the course. The point of the validation study is to compare the expectations for the course and the reality from teaching it. NOTE: A sample validation plan is shown on page 5-5 of the Curriculum Development Guide: Technical Education and Training.

6. Expectation Reality Validation Plan • The validation plan should address: • Class achievement. • Validity and reliability of evaluation instruments. • Student feedback. • Instructor feedback. • For each of these: • 1. Define expectations: What will be acceptable? Unacceptable? • 2. Specify data collection methods; develop collection instruments. • 3. Specify data analysis methods. • 4. Specify actions for unacceptable elements.

7. Expectation Reality CLASS ACHIEVEMENT • For each unit in the course, • consider the following questions: • How well has the class done? • Did a majority of the students successfully complete each part of the unit? • Did parts of the unit pose particular problems for the class or a group of students within the class? • Did students who did not perform well on the unit have common traits or backgrounds?

8. Expectation Reality VALIDITY AND RELIABILITY Validity and reliability are two measures of the effectiveness of evaluation instruments. Reliability is a measure of consistency. Does the instrument give the same results over time? Across classes? For example, a scale is reliable if it always tells you that the same 5 pound weight weighs 10 pounds. It might not be accurate, but it is reliable! Validity is a measure of appropriateness. Does the instrument measure what you intended it to measure? An instrument cannot be valid if it is not reliable (but a reliable instrument is not always valid!). For example, if you intend to measure a person’s ability to perform a procedure, a written test (measuring the student’s knowledge of the procedure) is not a valid measure of that ability.

9. Expectation Reality STUDENT FEEDBACK • When presenting a course, students are your • immediate customers. It makes sense to find • out what they think of the course. This will be • true every time the course is presented, but • student feedback is even more important for • the first iteration of a course. • Questionnaires for student feedback focus on the students’ perception of the content and conduct of the course: • Were the learning objectives clear? • Was there sufficient time to meet the learning objectives? • Did the students understand the purpose of the • lesson/unit/module? • Were study aids sufficient? Useful? • Was the material too hard? Too easy? • Were audiovisual materials useful? • Were the instructors knowledgeable? Helpful? Clear?

10. Expectation Reality INSTRUCTOR FEEDBACK Instructors can provide valuable information on the structure of the course in operation. They will be quick to recognize areas that need more or less time and where students run into difficulty. They can also provide information on how students reacted to various pieces of the course. Instructor feedback is usually gathered informally, frequently in staff meetings. For the first iteration of a course, it might be useful to use a more formal process, with questionnaires or structured interviews. See Chapter 5 in the Curriculum Development Guide: Technical Education and Training for additional information on student and instructor feedback.

11. STAFF ORIENTATION PLAN How will you train the trainers? Each instructor needs to be familiar with all parts of the course and understand how each part fits with the sections he or she will be teaching. Instructors must know how to use performance and product evaluations and all instructors must interpret the standards for performance in the same way. Each instructor must be thoroughly familiar with the sections of the course he or she will be teaching. Staff orientation must include time for instructors to personalize lesson topic guides.

12. TEACHING SCHEDULE • The teaching schedule • lays out the time and space • for every activity in the course. • In the teaching schedule: • Account for every period in the course. • Include location and instructor(s) for each period. • Include training aids and equipment required for each period.

13. OTHER CONSIDERATIONS Your implementation plan should also include a time line or POA&M (Plan of Action and Milestones) for getting ready for the first class of students. Make sure you allow enough lead time for ordering supplies and equipment, printing or duplicating materials for students, conducting the instructor orientation, and completing personalization of the lesson topic guides.

14. CONDUCT COURSE • The day has arrived! The long • process of analysis, design, and • development has finally come • together as the course comes on line. • Because we in the Navy Medical Department rarely have the chance to pilot a new course, the first convening class has to serve as our pilot. Be aware that there may be reefs ahead! In particular, the first class may uncover: • Weaknesses in scheduling. • Problems with test reliability and/or validity. • Ambiguous materials. • Conflicting information among lessons or units. • Make sure everyone is ready to deal with emergencies!

15. Validation CARRY OUT THE VALIDATION PLAN As stated on the last slide, the first convening class for most Navy Medical Department courses is the pilot implementation for a new course. Make sure that everyone involved in implementing the course thinks VALIDATION and follows the validation plan. Secondly, be alert for additional information about how the course is working that was not initially included in the validation plan. Amend the plan as needed.

16. OUTPUTS FROM THE IMPLEMENTATION PHASE • Outputs from this phase include: • Graduates • Validation Report • Summary of student achievement. • Summary of student input. • Summary of instructor input. • Reliability data, summary analysis for test instruments and items. • Summary of adjustments made during implementation. • List of recommended changes from validation process.

17. SUMMARY Although the slides in this module address the implementation phase for a new course, most of the steps are also needed for implementing subsequent iterations of the course. The validation plan can be modified and used to monitor the effectiveness of instruction. You may not need as much detail, but you will continue to watch student and instructor reactions to the materials and methods, reliability and validity of testing instruments, and general class achievement (or its negative side: attrition rate). The staff orientation plan becomes a plan for orientation of new instructors. The teaching schedule becomes the model for later schedules.

18. If you have any comments or suggestions on this or other Curriculum Design and Development staff development modules, please forward them to: • Dr. Anne L. Ballard (BUMED-531) • Bureau of Medicine and Surgery • 2300 E Street NW • Washington, DC 20372-5300 • Phone: 202-762-3829 or DSN 762-3829 • Email: alballard@us.med.navy.mil • Your input is always welcome!