FACULTY DEVELOPMENT MODULE ON ASSESSMENT

1. FACULTY DEVELOPMENTMODULE ON ASSESSMENT Harford Community College Nursing Faculty Orientation 2007 - 2008

2. INTRODUCTION Evidence-based teaching

3. If learning is essential, using effective teaching strategies is a key skill needed by educators • Evidence-based teaching is the conscientious, explicit, and judicious use of current best evidence in making decisions about the education of professional nurses. • A useful reference for faculty interested in teaching with an evidence-based approach is: Stevens, K. & Cassidy, V. (1999). Evidence-based teaching. Sudbury MA: National League for Nursing.

4. Evidence-based teaching practice (EBTP) Should involve systematic study of: teaching strategies; learning processes; learner characteristics; learner outcomes; academic success; effects of technology; organization of curriculum; theory verification; measurement and methodological issues; educational program standards; and administrative effectiveness. (NLN, Shaping the Future, spring 2004)

5. INTRODUCTION Value of the scholarship of teaching

6. The Scholarship of Teaching and Learning (SOTL) • We invite you to read more about SOTL to explore its value to your practice as an educator. Read and listen to the audio comments on the following SOTL web page by copying the following into your web browser address bar…….. http://www.issotl.org/tutorial/sotltutorial/u1a/u1a4.html You will be invited to discuss your own goals for scholarship of teaching with Director Putland and Dean Wrobel as you continue your orientation. Now please continue this module by continuing to the next slide.

7. INTRODUCTION Transforming Nursing Education (NLN, 2005) Innovation in Nursing Education: A Call to Reform (NLN, 2003)

8. “Learning and innovation go hand in hand. The arrogance of success is to think that what you did yesterday will be sufficient for tomorrow” William Pollard “Innovation is the ability to see change as an opportunity – not a threat.” Steve Jobs Innovation

9. “When complexity and ambiguity make progress difficult, transformational leaders seek novel solutions to problems using creativity and intuition. They facilitate the work of others and empower their success.” Joel, L. in Andersen, C., 1999. Transformation

10. The National League for Nursing Board of Governors, after input from nurse educators in all types of programs, issued the following two position statements to lay a framework for national debate and to guide curriculum development initiatives in the early 21st century. Please read the two position statements found at the web addresses below and then complete Activity 1 included in your workbook of this module presentation. • http://www.nln.org/aboutnln/PositionStatements/transforming052005.pdf • http://www.nln.org/aboutnln/PositionStatements/innovation082203.pdf

11. INTRODUCTION Challenges Facing Nurse Educators Today

12. Most masters programs in nursing have focused on preparation for advanced clinical practice rather than for advanced practice as a nurse educator. • So many new nurse educators, while they are excellent clinicians, have had little if any preparation in education theory, curriculum development, teaching and learning, and assessment strategies. • These issues combined with a growing faculty shortage in Maryland means that faculty development initiatives are needed to help new educators prepare for their academic role.

13. Purpose of this Module on Assessment Therefore the purpose of this orientation module is to provide a beginning framework for self-assessment of one’s knowledge base related to effective teaching and learning.

14. The Purpose of Assessment • Competency assessment is always outcome oriented; the goal is to evaluate performance for the effective application of knowledge and skill in the practice setting. Competency assessment techniques address psychomotor, cognitive, and affective domains. Competencies can be generic to clinical skills and leadership skills needed in practice, specific to a clinical specialty, at either basic or advanced levels (Benner, 1982; Gurvis & Grey, 1995).

15. Principles of Good Assessment identifies readiness to learn and unique learning needs utilizes resources and empowers success for life-long learning uses appropriate assessments

16. Unique Learning Needs • Adult Learners- bring experience, benefit from active participation, expect a high degree of influence on what they are to be educated for, need to see applications for new learning, expect their responses to be acted on when asked for feedback on the progress of the program • More Diversity- need for students to gain a culturally competent education, need for diversity in faculty role models, and need for all students to have access to resources that can empower their success • Multiple Life Stressors- the majority of nursing students today work and have significant family responsibilities while attending nursing school • Individual Learning Styles- all students do not learn the same way, so use of only one or two teaching methodologies can result in ineffective learning for many students

17. Resources for Faculty via web resources to promote student success on HCC website at: http://www.harford.edu/Advising/advinfo.asp?FA=StudentLife Resources for Students and Faculty available regarding pre-nursing requirements for admission at: http://www.harford.edu/Advising/prenursing.asp?FA=Employee Resources Available at HCC • Adult Literacy and ESL resources at HCC at: • http://www.harford.edu/cet/literacy/default.asp?FA=Employee

18. FERPA Family Educational Rights and Privacy Acts Go to Harford Community College website for this important orientation at: http://www.harford.edu/registration/FERPA/FERPA_Faculty/FERPA_Faculty_files/frame.htm

19. Types of Assessment • Diagnostic:a set of tools and interventions to assess the teachers readiness to teach and the students readiness to learn in ways that promote learning based on the complexity of the content, the level of competency required, the individual learning styles of the student, and the facilitators and barriers to successful assimilation and application of knowledge • Formative:will provide information to faculty about the process toward student success, quality of experience, and success of innovation in the curriculum; Data which reflects program competencies that can be used for summative evaluation should be collected • Summative:will help in determining how the innovation facilitates student achievement of course and program goals and outcomes. These data will allow benchmarking outcomes across similar programs given similar sample populations

20. Teaching & Learning for Different Levels of Thinking Skills Cognitive…... Psychomotor….. Affective Adapted from University of Hawaii, Honolulu Community College (HCC) new faculty training courses

21. Cognitive Domain of Learning:Knowledge • The cognitive domain has three practical instructional levels including fact, understanding, and application. The fact level is a single concept and uses verbs like define, identify, and list. The understanding level puts two or more concepts together. Typical verbs for this level include describe, compare and contrast. The application level puts two or more concepts together to form something new. Typical verbs at this level include explain, apply, and analyze. Delivery in this domain is typically a lecture or creative teaching presentation and the evaluation will be subjective and objective test items.

22. Psychomotor Domain of Learning: Skills • In the psychomotor domain the student will produce a product. The three practical instructional levels include imitation, practice, and habit. The first level, imitation, will simply be a return of the demonstration under the watchful eye of the instructor. The practice level may occur without direct oversight of the instructor. The habit level occurs when the student can perform the skill accurately, but in twice the time or less that it takes the instructor or an expert to perform. The evaluation will be a performance or skill test since content to do the skill is cognitive.

23. Affective Domain of Learning:Perceptions or Values • The affective domain is based upon behavioral aspects and may be labeled as beliefs. The three levels in the domain are awareness, distinction, and integration. The verbs for this domain are generally limited to words like display, exhibit, and accept and these apply at all levels. The first two levels are really cognitive; integration is behavioral and requires the learner to evaluate and synthesize. The content in this domain will usually involve discussions. The testing in the first two levels will be cognitive, whereas the third level will require an affective checklist.

24. Assessing and Evaluating Domains of Learning Cognitive…... Psychomotor….. Affective Adapted from NLN educational program, Innovative Teaching Strategies, by Mahoney, P. & Ortelli, T. NSNA Pre-Convention Faculty Workshop in Anaheim, CA, April 11, 2007.

25. Assessing and Evaluating the Cognitive Domain • group work • directed paraphrase • concept/mind mapping • writing • one minute papers • subjective test items • objective test items

26. Assessing and Evaluating the Psychomotor Domain • return demonstration • video tape • peer critique • peer coaching • simulations • pick out of a hat

27. Assessing and Evaluating the Affective Domain • self-assessment • journaling • attitude surveys • individual and group assessment

28. Major Categories in the Taxonomy of Educational Objectives(Bloom 1956) • Adapted from http://faculty.washington.edu/krumme/guides/bloom.html

29. Hierarchy of cognitive/thinking skills determine objective terminology It is important for new faculty to understand the taxonomy behind objective levels used in education. The most widely used taxonomy of cognitive domains of learning was developed in the mid-1950’s by Benjamin Bloom.

30. Knowledge level objectives: Knowledge of terminology; specific facts; ways and means of dealing with specifics (conventions, trends and sequences, classifications and categories, criteria, methodology); universals and abstractions in a field (principles and generalizations, theories and structures):Knowledge is (here) defined as the remembering (recalling) of appropriate, previously learned information. Outcome verbs in knowledge level objectives: define; describe; enumerate; identify; label; list; match; name; read; record; reproduce; select; state; or view.

31. Comprehension level objectives: Comprehension of grasping (understanding) the meaning of informational materials. Outcome verbs in comprehension level objectives: classify; cite; convert; describe; discuss; estimate; explain; generalize; give examples; make sense out of; paraphrase; restate (in own words); summarize; trace; or understand.

32. Application level objectives: Applicationof previously learned information in new and concrete situations to solve problems that have single or best answers. Outcome verbs in application level objectives: act; administer; articulate; assess; chart; collect; compute; construct; contribute; control; determine; develop; discover; establish; extend; implement; include; inform; instruct; operationalize; participate; predict; prepare; preserve; produce; project; provide; relate; report; show; solve; teach; transfer; use; or utilize.

33. Analysis level objectives: Analysis is the breaking down of informational materials into their component parts, examining (and trying to understand the organizational structure of) such information to develop divergent conclusions by identifying motives or causes, making inferences, and/or finding evidence to support generalizations. Outcome verbs in analysis level objectives: break down; correlate; diagram; differentiate; discriminate; distinguish; focus; illustrate; infer; limit; outline; point out; prioritize; recognize; separate; or subdivide.

34. Synthesis level objectives: Synthesis is creatively or divergently applying prior knowledge and skills to produce a new or original whole. Outcome verbs in synthesis level objectives: adapt; anticipate; categorize; collaborate; combine; communicate; compare; compile; compose; contrast; create; design; devise; express; facilitate; formulate; generate; incorporate; individualize; initiate; integrate; intervene; model; modify; negotiate; plan; progress; rearrange; reconstruct; reinforce; reorganize; revise; structure; substitute; or validate.

35. Evaluation level objectives: Evaluation is judging the value of material based on personal values/opinions, resulting in an end product, with a given purpose, without real right or wrong answers. Outcome verbs in evaluation level objectives: appraise; compare and contrast; conclude; criticize; critique; decide; defend; interpret; judge; justify; reframe; or support.

36. Hierarchy of objective levels (Bloom, 1956) • As the level of cognitive complexity required increases, the objective level for Bloom’s taxonomy also increases. • Level I are the lower levels of knowledge, comprehension and application objectives; • Level II are the higher levels of knowledge, application, analysis, synthesis, and evaluation objectives • Test blueprints for courses should reflect progressive complexity of objectives, with test questions and evaluation tools reflecting that progression of complexity across the program.

37. Bloom’s Taxonomy as a Basis for Writing Objectives and Assessments in HCC Courses Activity Two and Three

38. Activity Two • Request a copy from Director Putland of the current course objectives for the HCC nursing class you have been hired to teach • Identify in writing, on the copy, the level of objective (Bloom’s Taxonomy) that corresponds to each objectives in your course • Identify in writing, on the copy, the domain of learning that currently is guiding the teaching and learning in the class based on the course objectives

39. Activity Three • Write a proposed new list of course objectives for the class you have been hired to teach that reflect the appropriate domain of learning (cognitive, psychomotor, or affective) and are listed in an order that progresses from least complex to most complex across the course ===================================== • Set and appointment to review and discuss with Director Putland your work on Activity Two and Activity Three

40. HCC Curriculum Review Process Curriculum content is guided by requirements of accrediting bodies (NLNAC and Middle States Association of Colleges and Schools) and the Maryland State Board of Nursing. To review the HCC Curriculum Review Process go to: http://docushare.harford.edu/dsweb/Get/Document-224674/Educational%20Procedures%20and%20Practices.doc

41. Important Additional Aspects of Assessment to Consider • Authenticity- course objectives are clear, are shared with each student at the beginning of the course, and are consistently applied in ways that are culturally competent and meet student’s individual learning needs. Faculty provide assessment of each student’s progress toward achieving course objectives at frequent and regular intervals, and offer resources and remediation as needed to empower student success at Harford Community College.

42. Important Additional Aspects of Assessment to Consider • Validity: • Content: degree to which items in assessment instruments adequately represent the content • Criterion-related (concurrent or predictive): degree to which scores in an instrument are correlated with an external criterion • Construct- degree to which an instrument measures the construct under investigation Evidence of validity comes from a triangulated view of goals/outcomes that confirms quality outcomes for students, faculty, and agency or staff employees.

43. Important Additional Aspects of Assessment to Consider • Reliability • These measures assure the stability, consistency, and equivalence of course outcomes • Statistical evaluations vary based on the type of data collected. Item analysis, internal consistency measures, and other descriptive statistics with comparison of means can assist faculty to measure whether change has occurred related to benchmark outcomes

44. Other Considerations and Synthesis and Application of Content Thank you for participating in this Faculty Development Module on Assessment. While we cannot give you a graduate education in nursing education in one brief module, our hope is that we have provided a review and a baseline framework of terminology to help as you assess the course objectives in your class to guide your teaching, and prepare to assess and evaluate learning, as you begin your career at HCC. We’re glad you’re here! The nursing faculty and administration at Harford Community College in Bel Air, MD