Increasing Assessment Effectiveness in a Time of Decreasing Budgets Ken Stanton

1. Increasing Assessment Effectiveness in a Time of Decreasing BudgetsKen Stanton

2. PhD in Engineering Education at Virginia Tech • First cohort began Fall 2008 • 16 total students • Coursework in engineering, education, and engineering education • Teaching and research assistantships available • Graduate Recruitment Days every March and October– travel grants available! • www.enge.vt.edu

3. Introduction – The Problem • ABET EC2000 standards are a lot of work for engineering departments • Purposes: assessment and accountability • Problems: some faculty less familiar with assessment, requiring more and more of programs • Solutions: faculty development, assessment experts • Economic downturn affects education budgets • Problem: difficult to implement the solutions above • Enrollment, and therefore class size, increases • Time and resources limited • Any solutions: must be efficient, use what we have, be simple and easy to implement

4. Introduction – Proposed Solution • Address the need to increase assessment effectiveness from three angles • Utilize lean assessment techniques • Implement a process that maximizes intrinsic motivation • Take advantage of efficiency of assessment methodology

5. Innovative Solution – Lean Assessment • Goal of assessment Typical process for an assignment Teaching toward learning objectives Assignment created for student Student completes assignment Grade and feedback given to student

6. Innovative Solution – Lean Assessment Typical assessment process Teaching toward learning objectives Assessment created for student Student completes assessment Feedback given to student, may be graded Use to improve teaching and assessments Collect and analyze evidence of learning

7. Innovative Solution – Lean Assessment • Definition • simple, efficient, and effective assessment techniques • based in the principle of lean manufacturing • productive and simple methods • still attain useful data • Methods • Some existing • Angelo and Cross (1993) • Some modified • Some novel

8. Innovative Solution – Lean Assessment • Use to assess teaching and learning

9. Innovative Solution – Lean Assessment

10. Innovative Solution – Implementation • These assessment methods alone do not solve the problem • Consider motivation theory in implementation • Self-Determination Theory (SDT) by Deci & Ryan (2000) has 3 needs • Competence – confidence, self-efficacy • Autonomy – choice that aligns with identity / values • Relatedness – connection to others identify with • Satisfaction of these leads to internalization of motivation – move from extrinsic to intrinsic

11. Innovative Solution – Implementation • Competence • Lacking due to inexperience with assessment • Lean assessment techniques – simple, effective • Faculty helping each other • Autonomy • Challenged by requirements being handed down • Helped somewhat by flexibility of ABET criteria • Use assessment to address faculty’s teaching concerns • Relatedness • Can take opportunity to connect with students through feedback

12. Innovative Solution – Implementation • Review program educational objectives (PEOs), program learning outcomes, and course outcomes established for the program and course(s) of interest. • Interview faculty for input on assessment and what they would like to improve in their teaching and students’ performance. Review current assignments and assessments being used in their course(s) of interest. • Identify lean assessment techniques that • are good for measuring the outcomes from #1 • meet goals that faculty name as important to them • can complement/replace existing assignments and activities in the course(s).

13. Innovative Solution – Implementation • Briefly illustrate how the lean assessment techniques match the faculty member’s goals and can be used alongside or in place of existing course materials. • Assist with familiarization and use of the assessments, if necessary, and encourage sharing the expectations and motives with students. • Likely to be effective seeing as studies found faculty generally hold positive views of assessment

14. Innovative Solution – Assessment Methodology • Many times we develop courses by deciding material then assessments • Less efficient and less productive • Wiggins & McTighe (2005) coined “Backward Design” • Work from learning objectives to ways to measure them (assessments) to how to teach it • Ensures efforts are goal-oriented and assessments can be used for accreditation • Using this process can increase efficiency of teaching and assessment efforts

15. Introduction – Proposed Solution • Address the need to increase assessment effectiveness from three angles • Utilize lean assessment techniques • Implement a process that maximizes intrinsic motivation** • Take advantage of efficiency of assessment methodology

16. Future work • Dissertation study • From a motivational standpoint, how do engineering faculty members engage in assessment of student learning? • Also look at ways to integrate assessment into the culture of teaching and learning

17. Thank youQuestions?