The Role of Assessment in the EdD – The USC Approach

1. The Role of Assessment in the EdD – The USC Approach

2. To Whom is the EdD Accountable? • What are the “checks and balances?” • For the PhD degree, one of the major “quality control” mechanisms is the dissemination of work for eternal peer review • For the EdD, if the primary role is not to produce theoretical, generalizable knowledge for academic outlets, what is the equivalent?

3. The USC Context • Large - 150 plus students per year • One EdD - school wide • Common mission – improve urban education • Common themes that define core (learning, accountability, diversity, leadership • All faculty are EdD faculty • Thematic dissertations - 1 faculty & team of 8-9 students

4. How We Are Approaching Assessment At USC, the mission of the School is “to strengthen urban education locally, nationally, and globally. We define urban education broadly, conceptualizing it as something that takes place within many contexts including pre-kindergarten through high school, in human services, higher education, and workplace settings. We seek to transform urban education by: • Preparing and developing educational leaders who are change agents with a commitment to focusing on urban education, developing the competencies to address the complex educational and social issues facing urban communities. • Leading the search for innovative, efficacious, and just solutions to the challenges in urban education by engaging in collaborative action research on educational practices and policy. • Creating mutually beneficial partnerships to rethink curriculum, improve educational environments, and develop sound policy.

5. External Evaluation Our “new” program is now 10 years old – we are conducting an external evaluation (West Ed) with the following questions: • How have the programs been implemented? • To what extent have the programs achieved their student learning, program quality, and educator placement objectives? • What implementation and student factors are predictive of student outcomes in the programs and beyond in the students’ professional employment?

6. Approach An adapted version of the Kirkpatrick Four Level Evaluation model – measures are organized by type of outcome: • Reactions to the Program, • Classroom learning, • Transfer/implementation of learning, and • Program outcomes. Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs (3rd ed.). San Francisco: Berrett-Koehler Publishers.

7. A sampling of instruments and collection methods - Formative Reactions(Ensures that students are motivated and interested in learning.) • Student feedback forms to measure emotions, motivation, and self-efficacy • Student end-of-course evaluation • Faculty end-of-course evaluations • Administrators and student advisor feedback of students’ reactions Learning(Measures knowledge acquired, skills learned and improved, and/or attitudes changed) • Student assignments and grading rubrics • Student pre-assessment of knowledge and skills at the beginning of a course compared to achievement of course goals measured by course grading rubrics (pre- and post-assessments)

8. Formative Assessment Continued… Transfer/Implementation/Behavior(Measures learning transfer and the extent to which behavior changes as a result of learning) • Student performance in subsequent courses • Early warning systems • Student self-report of using new knowledge and skills in their professional work • Faculty perceptions of students’ applying knowledge and skills learned in previous classes to learning opportunities in current classes. • Capstone alignment with program goal, quality indicators, and completion rates • Employer reports about student job performance throughout the program

9. A sampling of instruments and collection methods - Summative Program Results (Measures the extent to which the program goal has been met or exceeded.) • Graduation rates • Placement statistics • Graduate attainment of career goals • Graduate perceptions of knowledge, experience, and self-efficacy in performing the program goal • Faculty perceptions of the implementation of the program • Employer perceptions of graduate performance • Third-party and University evaluations

10. Establishing a Process The results of this external evaluation will be used by the Dean and the Governance Committee to create a long term strategy/structure/process for ongoing assessment