220 likes | 374 Views
Assessment Processes. Tim Wilson Computer and Software Engineering 22 August 2006. Overview. History Assessment Process Objectives ABET Outcomes Indicator Courses Annual Assessment Meeting. History I. Computing Curriculum Assessment Plan (AY 1999 – 2000) Identified learning outcomes
E N D
Assessment Processes Tim Wilson Computer and Software Engineering 22 August 2006
Overview • History • Assessment Process Objectives • ABET Outcomes • Indicator Courses • Annual Assessment Meeting
History I • Computing Curriculum Assessment Plan (AY 1999 – 2000) • Identified learning outcomes • Proposed way to assess them • Developed by teamwork, consensus building • Nick Brixius • Tom Hilburn • Massood Towhid nejad • Farahzad Behi • Andrew Kornecki
History II • AY 2000 – 2001 • Plan modified to better incorporate ABET Engineering Criteria 2000 framework • Distinction between Program Educational Objectives and Program Outcomes • Plan modified to fit five-column format of ERAU SACS accreditation • Stretch to limit to three to five outcomes
History III • Fall 2002: ABET accreditation visit for BS Computer Engineering • Fall 2003: BSCE accredited • Fall 2004: ABET accreditation visit for BS Software Engineering • Fall 2005: BSSE accredited
Assessment Process Objectives • Measurement: • Are program outcomes being achieved? • Feedback: • If not, what needs to be done • Satisfy ABET requirements • Satisfy University / SACS needs
Assessment Process Attributes • Implementable • Has to be able to get going • Accurate and Reliable • Measurements need to be valid • Useful • Need to be able to determine appropriate changes • Sustainable • Has to be able to be performed on a recurring (annual) basis
Program Outcomes I (Computer Engineering) • An ability to apply knowledge of mathematics, science, and engineering; • An ability to design and conduct experiments; • An ability to analyze and interpret data; • An ability to design and implement a computer system or component to meet desired needs; • An ability to design and implement a software system or computational process to meet desired needs; • An ability to function on multi-disciplinary teams; • An ability to identify, formulate, and solve engineering problems; • An understanding of professional and ethical responsibility; • An ability to communicate effectively; • An understanding of the impact of engineering solutions in a global and societal context; • A recognition of the need for, and an ability to engage in life-long learning; • An understanding of contemporary issues in computer engineering; • An ability to use the techniques, skills, and modern engineering tools necessary to engineering practice; and • An understanding of real-time, safety-critical, embedded computer systems.
The Process • Recurring collection of specified data • An annual assessment meeting by department faculty • Review collected data • Determine whether outcomes met • Propose changes • Program revision once every three years • Process revision once every three years
The Data • Currently • Performance in indicator courses • Feedback from graduating seniors • Also (previously) • Feedback from recent graduates • Feedback from employers • Feedback from Industry Advisory Board • Collected anecdotal information
Indicator Courses I • Subset of the curriculum • Satisfies criteria making the particularly useful for outcomes assessment
Indicator Criteria • Course contributes to achievement of several outcomes • Collection of indicator courses covers the collection of outcomes • Distributed across lower- and upper-division courses • Each outcome assessed by multiple courses • Prerequisite to several other courses • Terminal course of multi-semester sequence • Capstone design course
Senior Survey • Graduating students asked to assess their ability to perform each program outcome • e.g., Apply knowledge of math, science, and engineering • Rating from “Poor” = 1 to “Excellent” = 5
Assessment Meeting I • Annual assessment meeting held at end of spring semester • Morning: Review individual course assessments of indicator courses in Computer Engineering, Software Engineering • About 10 minutes per course • Instructor describes assessment to rest • Sometimes vigorous discussion • Much information exchange
Course Assessment Form I • Indicate degree of achievement numerically, as per the table below • Indicate the confidence in your assessment numerically, as per the table below • Measure achievement relative to where students should be in the program, not where students completing the program should be
Assessment Meeting II • Afternoon: Review assessment of all data for program • Indicator courses • Senior survey • Look at each outcome one by one • Determine whether outcome was achieved
Assessment Write Up • Assessment meeting results used to prepare annual assessment summary for each program • Tabular data • Summary of discussion • Action items, proposed changes • Assessment summary used to enter data into ERAU Assessment / Planning system
Summary • Outcomes assessment process • Indicator courses for sustainability • Annual assessment meeting • Useful information exchange • Useful for both ABET and SACS