Assessing Student Learning in EPICS

1. Assessing Student Learning in EPICS

2. Assessing Student Learning: Outline • What to assess • Learning objectives and expectations • Artifacts – data to assess • Grading • Sr. Design Example

3. What to Assess • Students are given academic credit for mastering course content, • Not for the service they provide for the community • Students are therefore assessed on their demonstrated mastery of course content

4. Defining Learning Objectives • Started with ECE Senior Design: A student who successfully fulfills the course will demonstrate: • an ability to apply technical material from their discipline to the design of engineering products; • an understanding of design as a start-to-finish process; • an ability to identify and acquire new knowledge as a part of the problem-solving/design process; • an awareness of the customer in engineering design; • an ability to function on multidisciplinary teams and an appreciation for the contributions from individuals from other disciplines; • an ability to communicate effectively with both technical and non-technical audiences ; • demonstrates an awareness of engineering ethics and professional responsibility; • an appreciation of the role that engineering can play in social contexts . • Common set of outcomes across disciplines • Engineering  your discipline

5. Multidisciplinary Assessments • EPICS often requires multidisciplinary approaches • Assessing students from different areas requires their own learning objectives in their “own language” • Freshman vs senior • One vs two credits • Engineer vs Liberal Arts • Important to be specific about expectations and outcomes

6. Evaluation Rubric

7. Setting Expectations • Teams set goals for the semester through a project plan • Faculty advisor approves plan • Students set individual personal goals for each semester • Goals in week 4, updates weeks 8 and 12 • Faculty advisor approves goals • Self assessments weeks 4, 8, 12 and 16

8. Artifacts: Data to Assess • Students produce artifacts that can be assessed during their EPICS experience • Design Notebooks • Reflections • Self-assessments • Presentations • Reports • Project documentation • Delivered projects • Manuals or other documentations with project

9. Design Notebooks & Documentation • Build a record (portfolio) of an individual’s work • Document individual progress • Capture reflection/metacognitive activities • Academic and service experience • Guidelines provided and evaluate during the semester • % of points on format and content • Discipline useful for later as professionals • Project documentation • Documents successes and achievements • Digital archive for projects

10. Assessing Team and Individual Work • Teams are assessed • Project plan • Customer/Partner feedback • Presentations and team reports • Individual artifacts assessed • Peer assessments • Summary of accomplishments • Individual Notebooks • Reflections • Design records - authored • Observations

11. “Dry Run” Grading = Calibration • Midsemester grading is a “dry run” • If the semester were to end today, you would receive a _____ • All resources and artifacts evaluated • Self assessments evaluated • Students provided with a team and individual grade or range and comments • What would they have to do to improve? • Calibrates students and faculty • Problems can be identified early • Need for documentation reinforced

12. Final Grading • Repeat process for dry run grades • Final self-assessment • Students receive comments and their grades • Use dry run evaluations as a basis • Students addressed concerns over the last half of the semester? • Emphasis on documentation • Do the artifacts represent their level of work?

13. ABET, Sr. Design and EPICS • EPICS projects are well-matched to the ABET criteria. • Customer-driven service learning means that each team has a different project and that each student may have a different role on the team. • This variability requires procedures for assessment, tracking, and documentation of projects and of student outcomes.

14. Senior Design and EPICS • Sr. Design option for • ECE, IDE and CS students • Two semesters of EPICS • Outcomes matrix • Demonstrate each has been met • Approvals, TA, Advisor, EPICS • Project Description • Approvals, Advisor, EPICS, ECE • Documents saved on Sharepoint Server (revision control)

15. EPICS and ABET • Fulfills capstone design requirement for Purdue BSEE, BSCmpE and BSIDE degrees • 3 credits over 2 senior semesters • 8 outcomes related to the ProgramObjectives & Outcomes • 2 approval processes: • Significant design experience on a suitable project • Satisfaction of course outcomes by each student: “A student who successfully fulfills the course requirements associated with at least 3 credits of EPICS taken over 2 or more semesters will …”

16. Project Approval • Project Description Form: • Team & project name • Project members, majors, expertise • Project & customer summary • How builds on earlier ECE courses • New technical knowledge acquired • Multidisciplinary nature • How project involves professionalcomponent (criterion 4) constraints • One form per project w/ seniordesign students per semester • Reviewed/approved by team advisor, EPICS administrators, ECE Senior Design committee

17. Outcomes Certification Documenting Outcomes: • Deliverables • Design notebook • Design reviews • Reports • Presentations • Weekly reports • Customer feedback • Peer evaluation • Self assessment

18. Outcomes Certification • Outcomes record maintained by students • Reviewed twice per semester by TAs and team advisor: incremental approval • Semester-end and year-end review byEPICS administration • Reviews: • Approval ofoutcomes anddocumentation • Redirect student’sefforts