Good Afternoon

1. Good Afternoon International Assessment and Retention Conference 2007 Assessment as a Cross-Campus Strategy for Improving Student Learning and Retention Jon Persavich, Ph.D.Dean and Professor of EducationSchool of Health Managementjpersavich@atsu.edu

2. Mission Driven Context Based Curriculum Design; Creating an Online Doctorate Program in Health Education Incorporating Embedded Authentic Assessment Mission Driven Context Based Curriculum Design Creating an Online Doctorate Program in Health Education Incorporating Embedded Authentic Assessment 2

3. Agenda Definition of Terms Today’s Students Assessment Methods Model for Embedded Authentic Assessment -- Theoretical -- Applied 3

4. Definition of Terms Assessment of student learning = Test (no one best way). There is a wealth of literature providing evidence that test format is not important. However, sometimes beliefs are so strong that stakeholders cannot be convinced otherwise. My goal is not to convince or cajole; but to provide information and a model for implementing embedded authentic assessment within curriculum design. 4

5. Definition of Terms Assessment, determining how, what, when, where and under what conditions students are learning, is an essential part of teaching. Assessment, is no longer seen exclusively as a psychometric measurement problem, but as a an educational design issue. Accountability, informing parents and the public about how well a school is educating its students and about the quality of the social and learning environment. 5

6. Definition of Terms Mission Driven Context Based Curriculum Design -- Mission statement drives curriculum design. -- Curriculum offerings incorporate student context (what they bring to the classroom). -- Assignments and assessments require the students and facilitator to create and maintain an active learning environment. Assessment Principle outcomes objectives assignments assessments Accountable to both the learner and the public. 6

7. Today’s Students The Average 21 year old has spent -- 10,000 hours playing video games -- 20,000 hours watching TV -- Seen 500,000 commercials -- Sent 250,000 eMails -- Spent less than 5,000 hours reading Prensky, 2004 7

8. Today’s Students Characteristics of Information-Age Student • * Computers are not technology – have not known life without them. • * Internet better than TV – socialization, interactivity. • * Doing is more important than knowing – result orientation. • * Multi-tasking is a way of life. • * Typing is preferred to handwriting. • * Zero tolerance for delays. • * Consumer and creator are blurring – file sharing, cut and paste; • if it is in digital format it is everyone’s property. Frand, 2006 8

9. Today’s Students Growth in Online Education -- 18% annual growth in enrollment. -- Growth rate more than 10 times that of general post-secondary. -- 56% of institutions identify online education as critical to their long-term growth strategy. Sloan Consortium, 2005, 2006 9

10. Assessment Methods Nine Principles of Good Practice for Assessing Student Learning -- Begins with educational values. -- Reflects an understanding of learning as multidimensional, integrated and revealed in performance over time. -- Programs it seeks to improve have clear, explicitly stated purposes. -- Outcomes are equally as important as experiences leading to them. -- Ongoing and not episodic. Astin, et.al. 10

11. Assessment Methods Nine Principles of Good Practice for Assessing Student Learning -- Fosters wider improvement when representatives from across the educational community are involved. -- Begins with issues of use and illuminates the questions that people really care about. -- Leads to improvement when it is part of a larger set of conditions that promote change. -- Meets the responsibilities to students and the public. Astin, et.al. 11

12. Assessment Methods Essay Questions -- Should be able to test high-level cognitive skills; synthesize. -- Candidates interpret questions differently, fair marking difficult, differential examiner differential marking. -- Amount of time taken to answer a question has a negative impact on the reliability of the test. 12

13. Assessment Methods Short Answer Questions -- Response time 5 – 10 minutes. -- Answers not normally more than a page long. -- Not easy to produce items which test cognitive levels higher than simple recall. 13

14. Assessment Methods Multiple Choice Questions -- An item stem with 4 or 5 choices. -- Allow quick coverage of broad topics. -- Hard to write items to test higher cognitive levels. -- True/False not possible for some disciplines. 14

15. Assessment Methods Embedded Assessment -- The assessment process is integrated into the work of both faculty and students. -- Clear link between what is taught and what is assessed. -- Used to evaluate the stages of student learning; rather than simply being summative. -- Enables faculty to consider which skills or knowledge might best be introduced at which levels or in what sequence. 15

16. Assessment Methods – Embedded -- Students are simply fulfilling the normal requirements of the course(s); do not know their work is being used for assessment proposes, thereby eliminating issues related to motivation. -- Help with academic honesty as individual students must apply what they are learning to their own context. -- Results can be shared with students as a group, allowing them to understand better the criteria that faculty expect them to meet and helping them to evaluate their own strengths and weaknesses. 16

17. Assessment Methods Authentic Assessment -- No single agreed upon definition. -- Assessment is authentic when we directly examine performance on worthy intellectual tasks. -- Tasks are either replicas of or analogous to the kinds of problems faced by citizens, consumers or professionals in the field. (Wiggins, 1993) 17

18. Assessment Methods -- Authentic -- Assessments call upon the examinee to demonstrate specific skills and competencies, that is, to apply, the skills and knowledge acquired. (Stiggins, 1987) -- Authentic Assessment drives the curriculum. Faculty first determines the tasks that students will perform, to demonstrate their mastery, and then curriculum is developed that will enable students to perform those tasks. This as been referred to as planning backwards. (McDonald, 1992) -- Authentic Assessment values the thinking behind work, the process as much as the finished product. (Pearson, Valencia, 1987, Wiggins, 1989, Wolf 1989) 18

19. Assessment Methods -- Authentic Benefits of Authentic Assessment -- Students assume active role in the assessment process, may result in reduced test anxiety and enhanced self-esteem. -- Can be used successfully with students of varying cultural backgrounds, learning styles and academic ability. -- Tasks are more interesting and reflective of students daily lives. -- Promotes more student-centered approach to teaching. -- More positive attitude toward school and learning. Hart, Authentic Assessment: A Handbook for Educators 19

20. Assessment Methods -- Authentic -- Faculty assume a larger role in the assessment process than through traditional testing programs; assuring the process reflects course goals and objectives. -- Provides valuable information to the faculty on student progress as well as the success of instruction. -- Constituents (and students) more readily understand authentic assessment than the abstract percentiles, grade equivalents, and other measures of standardized tests. Hart, Authentic Assessment: A Handbook for Educators 20

21. Assessment Methods -- Authentic -- Authentic assessments present the student with a full array of tasks that mirror the priorities and challenges found in the best instructional activities: conducting research, writing, revising and discussion, and collaborating with others. Conventional tests are generally limited in these areas. -- Authentic assessment helps to achieve validity and reliability by emphasizing and standardizing the appropriate criteria for scoring such as (varied) products; traditional testing standardizes the objective “items” and, hence, (one) right answer to each item. 21

22. Model for Embedded Authentic Assessment -- Theoretical • Authentic Assessment utilizes learning activities that encourage students to use higher-order thinking skills. These include: • Performance Assessment, challenges students to use skills in a variety of authentic contexts. • -- writing, revising and presenting reports to peers • -- conducting week long experiments and analyzing results • -- working with a team to prepare for a debate 22

23. Model for Embedded Authentic Assessment -- Theoretical • Short Investigations • -- evaluating obtainment of basic concepts and skills • --- students asked to interpret, describe, predict, calculate or explain • Open-Response Questions • --- brief written or oral answer • --- mathematical solution • --- drawings, diagrams, maps 23

24. Model for Embedded Authentic Assessment -- Theoretical • Portfolios, document learning over time vs episodic, provides a long-term account of student’s reflections, documents self-assessment, ability to take in new accounts particularly effective when combined with the peer and group review process. • --- written • --- electronic • 5. Self-Assessment, documents (written or oral) the students participation, in the active learning process. • -- Most difficult part of a project • -- Next steps • -- What should be done differently • -- What did you learn from a particular activity 24

25. Model for Embedded Authentic Assessment – Applied D.H.Ed. 916 Course Syllabus This is the first 10 week course in the D.H.Ed. Program of study. It outlines the entire program and contains the following deliverables: -- Committee nominations and approval -- Problem Statement first through final draft -- Dissertation Proposal first draft -- Literature Review preliminary 25

26. Model for Embedded Authentic Assessment – Applied D.H.Ed. Completion Matrix This single handout contains the entire D.H.Ed. Program of study. It gives each student an overview of what to expect during the program of study. 26

27. Model for Embedded Authentic Assessment – Applied D.H.Ed. Dissertation Completion Timeline This file contains the schedule for the completion of each of embedded deliverables for this program of study. The program is designed to eliminate the ABD status and thus have a positive impact on the 50-60% failure to complete doctoral programs of study. In the United States the students in high school and doctoral programs have the same chance of finishing. 27

28. Closing Thoughts Fairness does not exist when an assessment is uniform, standardized, impersonal and absolute. Rather, it exists when an assessment is suitable; in other words, when it is personalized, natural and flexible. The best assessments always teach/show students and faculty alike the kind of work that matters most; they are enabling, thought provoking, and forward-looking, not just reflective of prior teaching. Thank you! 28