Program Assessment in Technology Instruction

2. Program Assessment inTechnology Instruction Marvin I. Sarapin, Ph.D.

3. Purpose of Presentation • Present Assessment Model • Developed for NAIT Accreditation • Discuss Assessment Activities M. Sarapin Kean University

4. NAIT Standard 6.16 Assessment Plan and Integration An assessment plan shall be comprised of, but not limited to, the following for each program: M. Sarapin Kean University

5. NAIT Standard 6.16 Assessment Plan and Integration • program mission statement • the desired program outcomes/student competencies • evidence that the program incorporates these outcomes/student competencies • the assessment measures used to evaluate student mastery of the student competencies stated • compilation of the results of the assessment measures, and • evidence that these results are used to improve the program. (NAIT, 2000) M. Sarapin Kean University

6. Phase I Review Program Goals and Objectives Phase II Identify Student Outcomes Phase III Validate Student Outcomes Phase IV Administer Assessment Instruments Phase V Revise Program/Courses Revise Methods Assessment Model

7. Phase IReview Program Goals and Objectives • Review Program Definitions • Review Program Mission Statement • Develop short-term goals • Develop long-term goals M. Sarapin Kean University

8. Program Definitions • BS in Telecommunications and Information Technology • BS in Industrial Technology with option in • Electronics Technology • BS in Computer Integrated Design and Manufacturing Technology M. Sarapin Kean University

9. MISSION The mission of the Department of Technology, at Kean University, is to promote the transmission, expansion, and application of technical knowledge through teaching, research, and community service. The programs offered by the department prepare technically oriented professionals for leadership, management, and service positions in industry, business, education, and government. Our primary purpose is to provide technology related undergraduate, graduate and continuing professional education programs to support the career goals of individuals entering and advancing in technical and managerial positions at all levels of business, industry, and education. M. Sarapin Kean University

10. Goals • Prepare for NAIT accreditation • Develop new degree programs • Revise general education curriculum • Restructure advisory committee • Maintain educational technical environment M. Sarapin Kean University

11. Phase IIIdentify Student Outcomes • Develop documentation by COURSE! • Use course outline objectives • IT Program Core student outcomes • CIDM-Option area of specialization student outcomes • Electronics-Option area of specialization student outcomes • Telecom Program area of specialization student outcome M. Sarapin Kean University

12. Phase IIIdentify Student Outcomes • State objectives in behavioral terms. • As a result of this course the student should be able to: M. Sarapin Kean University

13. Sample IT Core Outcome Statements • As a result of taking Tech 1200, Introduction to Computer Aided Drafting (CAD), the Industrial Technology student should be able to: • 1.1. Hand letter. • 1.2. Sketch using the isometric and orthographic systems. • 1.3 Accurately measure using the Engineer’s and Architect’s scale. • 1.4. Use AutoCAD for two dimensional drafting. • 1.5. Understand and be able to use absolute, relative and polar coordinate systems. • 1.6. Use geometric construction, descriptive geometry, and auxiliary view techniques. • 1.7. Create working drawings for a hypothetical design problem. • 1.8. Identify a simple design problem and apply the various steps involved in the design process to reach a successful solution to the problem. M. Sarapin Kean University

14. Sample Specialization Outcomes M. Sarapin Kean University

15. Phase IIIValidate Student Outcomes • Compile student outcome statements • Technology Core • Areas of specialization • Rate student outcome statements • Revise outcome statements M. Sarapin Kean University

16. Advisory Committee By-Laws Purposes • To provide program validation including mission and goals, course sequencing, instructional content, technical support, industrial applications, assessment, NAIT Accreditation, and Kean University Program Review. • To assist in program publicity, student and faculty recruitment, special project identification and funding. • To provide feedback and other assistance to the Technology Program’s students, faculty, administration, and Alumni. M. Sarapin Kean University

17. Advisory Committee Make Up • The Department of Technology advisory committee shall include (3) three individual from each Option or Major Program offered by the Department of Technology (Industrial Technology Program). The three committee members from each Option shall be represented as follows: • One individual representing industry from the technical area • One distinguished Alumni from the technical area • One current student from the technical area M. Sarapin Kean University

18. Student Outcome Assessment ScaleAdministered to Advisory Committee • 5 This student outcome is essential • 4 This student outcome is extremely important • 3 This student outcome is important • 2 This student outcome is less important than most • 1 This student outcome should be revised • 0 This student outcome should be eliminated M. Sarapin Kean University

19. Outcome Core Ratings a b c d e f g h I j k l Total 11 Raters T1200 1.1 2 5 3 2 1 0 2 3 3 3 4 28 2.55 1.2 3 5 5 3 4 5 3 5 4 5 4 46 4.18 1.3 3 4 4 5 2 5 3 5 3 5 4 43 3.91 1.4 5 5 5 5 1 5 4 5 2 5 4 46 4.18 1.5 4 5 5 4 1 5 4 5 4 5 3 45 4.09 1.6 4 4 5 5 3 5 4 5 3 5 3 46 4.18 1.7 4 4 5 4 3 5 5 5 4 5 4 48 4.36 Mean 1.8 5 4 5 5 2 5 5 5 4 5 3 48 4.36 31.82 3.98 Sample Advisory Committee Student Outcome Rating Data M. Sarapin Kean University

20. Sample IT Program Core Student Outcome Rating Visual Goal 1.1 was deleted! M. Sarapin Kean University

21. Phase IVAdminister Assessment Instruments • Current student survey • Alumni survey • Employer survey • Alumni outcome assessment • Other M. Sarapin Kean University

22. Current student survey • Survey develop and revised by faculty • Administered to current students bi-yearly • Collect demographics • Sample student satisfaction M. Sarapin Kean University

23. Sample current student information M. Sarapin Kean University

24. Alumni survey • Survey developed and revised by faculty • Administered to graduates for last 6-years every third-year • Collect demographics, employment history, satisfaction, etc • Obtain permission to survey employer M. Sarapin Kean University

25. Alumni follow-up information M. Sarapin Kean University

26. Sample follow-up information M. Sarapin Kean University

27. Sample follow-up information M. Sarapin Kean University

28. Employer survey • Rate graduate’s performance • Compare to graduates of other programs • Collect programmatic observations and perceptions M. Sarapin Kean University

29. Sample employer information M. Sarapin Kean University

30. Alumni outcome assessment survey • Administer outcome assessment instrument • Alumni rate importance of outcome statements • Alumni rate their attainment of outcomes M. Sarapin Kean University

31. Sample Outcome Rating M. Sarapin Kean University

32. Other Assessment measures • Course imbedded instruments • Rubrics • Portfolios • Certifications M. Sarapin Kean University

33. Phase VRevise Program/Revise Courses Revise Assessment Methods Closing the loop! • Revise stated student outcome statements • Revise course outlines • Revise instruction • Revise assessment process M. Sarapin Kean University

34. Closing remarks • Assessment is time consuming but can be made practical • Assessment causes the faculty, advisors, student, alumni, and employers to examine the program M. Sarapin Kean University

35. For more information contact: msarapin@kean.edu M. Sarapin Kean University