Assessment Processes

1. Assessment Processes Tim Wilson Computer and Software Engineering 22 August 2006

2. Overview • History • Assessment Process Objectives • ABET Outcomes • Indicator Courses • Annual Assessment Meeting

3. History I • Computing Curriculum Assessment Plan (AY 1999 – 2000) • Identified learning outcomes • Proposed way to assess them • Developed by teamwork, consensus building • Nick Brixius • Tom Hilburn • Massood Towhid nejad • Farahzad Behi • Andrew Kornecki

4. History II • AY 2000 – 2001 • Plan modified to better incorporate ABET Engineering Criteria 2000 framework • Distinction between Program Educational Objectives and Program Outcomes • Plan modified to fit five-column format of ERAU SACS accreditation • Stretch to limit to three to five outcomes

5. History III • Fall 2002: ABET accreditation visit for BS Computer Engineering • Fall 2003: BSCE accredited • Fall 2004: ABET accreditation visit for BS Software Engineering • Fall 2005: BSSE accredited

6. Assessment Process Objectives • Measurement: • Are program outcomes being achieved? • Feedback: • If not, what needs to be done • Satisfy ABET requirements • Satisfy University / SACS needs

7. Assessment Process Attributes • Implementable • Has to be able to get going • Accurate and Reliable • Measurements need to be valid • Useful • Need to be able to determine appropriate changes • Sustainable • Has to be able to be performed on a recurring (annual) basis

8. Program Outcomes I (Computer Engineering) • An ability to apply knowledge of mathematics, science, and engineering; • An ability to design and conduct experiments; • An ability to analyze and interpret data; • An ability to design and implement a computer system or component to meet desired needs; • An ability to design and implement a software system or computational process to meet desired needs; • An ability to function on multi-disciplinary teams; • An ability to identify, formulate, and solve engineering problems; • An understanding of professional and ethical responsibility; • An ability to communicate effectively; • An understanding of the impact of engineering solutions in a global and societal context; • A recognition of the need for, and an ability to engage in life-long learning; • An understanding of contemporary issues in computer engineering; • An ability to use the techniques, skills, and modern engineering tools necessary to engineering practice; and • An understanding of real-time, safety-critical, embedded computer systems.

9. Curriculum and Outcomes

10. The Process • Recurring collection of specified data • An annual assessment meeting by department faculty • Review collected data • Determine whether outcomes met • Propose changes • Program revision once every three years • Process revision once every three years

11. The Data • Currently • Performance in indicator courses • Feedback from graduating seniors • Also (previously) • Feedback from recent graduates • Feedback from employers • Feedback from Industry Advisory Board • Collected anecdotal information

12. Indicator Courses I • Subset of the curriculum • Satisfies criteria making the particularly useful for outcomes assessment

13. Indicator Courses II

14. Indicator Criteria • Course contributes to achievement of several outcomes • Collection of indicator courses covers the collection of outcomes • Distributed across lower- and upper-division courses • Each outcome assessed by multiple courses • Prerequisite to several other courses • Terminal course of multi-semester sequence • Capstone design course

15. Senior Survey • Graduating students asked to assess their ability to perform each program outcome • e.g., Apply knowledge of math, science, and engineering • Rating from “Poor” = 1 to “Excellent” = 5

16. Assessment Meeting I • Annual assessment meeting held at end of spring semester • Morning: Review individual course assessments of indicator courses in Computer Engineering, Software Engineering • About 10 minutes per course • Instructor describes assessment to rest • Sometimes vigorous discussion • Much information exchange

17. Course Assessment Form I • Indicate degree of achievement numerically, as per the table below • Indicate the confidence in your assessment numerically, as per the table below • Measure achievement relative to where students should be in the program, not where students completing the program should be

18. Course Assessment Form II

19. Assessment Meeting II • Afternoon: Review assessment of all data for program • Indicator courses • Senior survey • Look at each outcome one by one • Determine whether outcome was achieved

20. Assessment Meeting III

21. Assessment Write Up • Assessment meeting results used to prepare annual assessment summary for each program • Tabular data • Summary of discussion • Action items, proposed changes • Assessment summary used to enter data into ERAU Assessment / Planning system

22. Summary • Outcomes assessment process • Indicator courses for sustainability • Annual assessment meeting • Useful information exchange • Useful for both ABET and SACS