Conducting a Visit Using the New Criteria

1. Conducting a Visit Using the New Criteria 1

2. New Criteria Visits Outline • Introduction • Criteria • Self Study • Visit Preparation and Conduct • Exit Meeting • Visit Forms • Draft Statement • Final Statement • Wrap Up

3. General Criteria

4. ABET Definitions (1) • Program Educational Objectives – Program educational objectives are broad statements that describe the career and professional accomplishments that the program is preparing graduates to achieve. • Program Outcomes – Program outcomes are narrower statements that describe what students are expected to know and be able to do by the time of graduation. These relate to the skills, knowledge, and behaviors that students acquire in their matriculation through the program.

5. ABET Definitions (2) • Assessment – Assessment is one or more processes that identify, collect, and prepare data to evaluate the achievement of program outcomes and program educational objectives. • Evaluation – Evaluation is one or more processes for interpreting the data and evidence accumulated through assessment practices. Evaluation determines the extent to which program outcomes or program educational objectives are being achieved, and results in decisions and actions to improve the program.

6. Notes on definitions • These are very important: you (and your PEVs) should know them well. • There is no requirement that the institution use these terms as defined by ABET in the self study. It is important to determine what terms the institution uses and the correspondence with the ABET definitions.

7. Criterion 1 – Students Students can complete the program in a reasonable amount of time. They have ample opportunity to interact with their instructors. Students are offered timely advising, by qualified individuals, about the program’s requirements and their career alternatives. Students who graduate from the program meet all program requirements. Comment: Similar to previous criteria.

8. Criterion 2 – Program Educational Objectives The program has documented, measurable educational objectives, based on the needs of the program’s constituencies. Comment: Appropriate constituencies (e.g., faculty, students, alumni, employers) should be used in determining the constituency needs.

9. Criterion 3 – Program Outcomes The program has documented, measurable outcomes, based on the needs of the program’s constituencies. Comment: If there is broad constituency input to objectives and outcomes are derived from objectives, then broad constituency input to outcomes development may not be needed. (The faculty is the primary constituency involved in development of outcomes.)

10. Criterion 3 – (continued) The program enables students to achieve, by the time of graduation: • An ability to apply knowledge of computing and mathematics appropriate to the discipline • An ability to analyze a problem, and identify and define the computing requirements appropriate to its solution • An ability to design, implement and evaluate a computer-based system, process, component, or program to meet desired needs

11. Criterion 3 – (cont.) • An ability to function effectively on teams to accomplish a common goal • An understanding of professional, ethical, legal, security and social issues and responsibilities • An ability to communicate effectively with a range of audiences • An ability to analyze the local and global impact of computing on individuals, organizations and society • Recognition of the need for, and an ability to engage in, continuing professional development • An ability to use current techniques, skills, and tools necessary for computing practice.

12. Notes on Criterion 3 – (1) • Institution is expected to define its own outcomes. • Additional elements will appear in program criteria.

13. Notes on Criterion 3 – (2) • Institution will need to show how a-i are enabled. • How can this be done? • What about partial compliance? • Is it sufficient for the institution to have an effective assessment process that covers a-i? • Notice that it says enables students to achieve; doesn’t demand that all students have achieved them. • How can this be evaluated?

14. Notes on Criterion 3 – (3) • Criterion 3 (c) should be interpreted as appropriate for the nature and objectives of the program. • 3(c): An ability to design, implement and evaluate a computer-based system, process, component, or program to meet desired needs • Normally not all of “system, process, component, or program” are required. • I.e., the “or” is not inclusive • One could be enough

15. Criterion 4 – Continuous Improvement The program uses a documented process incorporating relevant data to regularly assess its educational objectives and outcomes, and to evaluate the extent to which they are being met. The results of the evaluation are documented and used to effect continuous improvement of the program through a documented plan.

16. Notes on Criterion 4 (1) • Must assess the extent to which outcomes are being achieved by students. • Effective assessment usually implies: • Measures based on actual student performance for outcomes • A priori establishment of performance goals • No requirement that program improvements must have been made. • What is satisfactory evidence?

17. Notes on Criterion 4 (2) • Must assess the extent to which objectives are being achieved by graduates. • Effective assessment for objectives usually implies: • Collection of data relative to the achievement of each objective • Surveys are normally used. • A priori establishment of performance goals • No requirement that program improvements must have been made. • But assessment and evaluation results must be documented, including decisions as to whether improvement might be needed.

18. Notes on Criterion 4 (3) • Note that the use of data/ideas from sources other than the assessment of objectives and outcomes achievement can be used to improve the program. • This should not be discouraged, but it is not by itself evidence of program improvement referred to in the 2008-09 CAC criteria. • Proposed harmonized criteria recognize additional sources to suggest improvements.

19. Criterion 5 - Curriculum The program’s requirements are consistent with its educational objectives and are designed in such a way that each of the program outcomes can be achieved. The curriculum combines technical and professional requirements with general education requirements and electives to prepare students for a professional career and further study in the computing discipline associated with the program, and for functioning in modern society. The technical and professional requirements include at least one year of up-to-date coverage of fundamental and advanced topics in the computing discipline associated with the program. In addition, the program includes mathematics appropriate to the discipline beyond the pre-calculus level. For each course in the major required of all students, its content, expected performance criteria, and place in the overall program of study are published.

20. Notes on Criterion 5 • Program criteria provide further specificity. • Note that the curriculum must be designed so that its outcomes can be achieved. • No mention of enabling a-i. • When might a-i enter into the picture relative to the program’s curriculum? • What does “… performance criteria … must be published” mean?

21. Criterion 6 – Faculty A. Faculty Qualifications Faculty members teaching in the program are current and active in the associated computing discipline. They each have the educational backgrounds or expertise consistent with their expected contributions to the program. Each has a level of competence that normally would be obtained through graduate work in the discipline, relevant experience, or relevant scholarship. Collectively, they have the technical breadth and depth necessary to support the program. Comment: Similar to present criteria.

22. Criterion 6 – (cont.) B. Faculty Size and Workload There are enough full time faculty members to provide continuity, oversight and stability, to cover the curriculum reasonably, and to allow an appropriate mix of teaching, professional development, scholarly activities, and service for each faculty member. The faculty assigned to the program has appropriate authority for the creation, delivery, evaluation and modification of the program, and responsibility for the consistency and quality of its courses. Comment: Similar to present criteria.

23. Criterion 7 – Facilities Institutional facilities including the library, other electronic information retrieval systems, computer networks, classes, and offices are adequate to support the educational objectives and outcomes of the program. Computing resources are available, accessible, systematically maintained and upgraded, and otherwise adequately supported to enable students to achieve the program’s outcomes and to support faculty teaching needs and scholarly activities. Students and faculty members receive appropriate guidance regarding the computing resources and laboratories available to the program. Comment: Similar to previous criteria.

24. Criterion 8 –Support The institution’s support for the program and the financial resources available to the program are sufficient to attract and retain qualified faculty members, administer the program effectively, acquire and maintain computing resources and laboratories, and otherwise provide an environment in which the program can achieve its educational objectives and outcomes. Support and resources are sufficient to provide assurance that the program will retain its strength throughout the period of accreditation. Comment: Similar to existing criteria.

25. Criterion 9 – Program Criteria Each program must satisfy applicable Program Criteria (if any). Program Criteria provide the specificity needed for interpretation of the General Criteria as applicable to a given discipline. If a program, by virtue of its title, becomes subject to two or more sets of Program Criteria, then that program must satisfy each set of Program Criteria; however, overlapping requirements need to be satisfied only once. Note that this criterion will disappear (as a criterion: its requirements will remain) if the proposed harmonized criteria are adopted.

26. (End of General Criteria) • Questions/comments?

27. Existing CAC Program Criteria • Computer Science • Information Systems • Information Technology

28. Program Criteria Notes (1) • Applicable program criteria are determined by the title (name) of the program being evaluated. • Determined by HQ based on the program title listed on the RFE. • Be alert to a possible mismatch between expectations of program and HQ program criteria assignment. • Check program requirements for consistency with assigned criteria. • Do this well BEFORE the visit. • Graceful recovery from a problem is almost impossible after the visit.

29. Program Criteria Notes (2) • If a program title does not imply one or more existing sets of CAC program criteria, it is evaluated under the General Criteria only. • Examples: Information Science, Digital Forensics, Computational Science

30. Computer Science Program Criteria Additions Additional criteria added to General Criteria: • Capabilities that must be enabled. • Curriculum requirements. • Faculty qualification requirement.

31. CS Program Criteria (1) 3. Program Outcomes The program enables students to achieve, by the time of graduation: j) an ability to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices; k) an ability to apply design and development principles in the construction of software systems of varying complexity.

32. CS Program Criteria (2) 5. Curriculum Students have the following amounts of course work or equivalent educational experience: a. Computer science: One and one-third years that includes: • coverage of the fundamentals of algorithms, data structures, software design, concepts of programming languages, and computer organization and architecture. • An exposure to a variety of programming languages and systems. • proficiency in at least one higher-level language. • advanced course work that builds on the fundamental course work to provide depth.

33. CS Program Criteria (3) 5. Curriculum (cont’d) b. One year of science and mathematics: • Mathematics: At least one-half year that must include discrete mathematics. The additional mathematics might consist of courses in areas such as calculus, linear algebra, numerical methods, probability, statistics, number theory, geometry or symbolic logic. • Science: A science component that develops an understanding of the scientific method and provides students with an opportunity to experience this mode of inquiry in courses for science or engineering majors that provide some exposure to laboratory work.

34. Notes on CS Curriculum • No probability and statistics requirement. • No specific number of credits of science. • No comments about communication skills, ethics, design, etc. since these are subsumed by the abilities in the program outcomes criterion.

35. CS Program Criteria (4) 6. Faculty Qualifications Some full time faculty members have a Ph.D. in computer science.

36. Visit Forms 36

37. Visit Forms to be Discussed • Program Evaluation Worksheet (PEW) • PEV Visit Report (PER) • Program Audit Form (PAF) • Program Evaluation Tracking Form (PET) • Short Form (SF)

38. Program Evaluation Worksheet (PEW) • Completed by each PEV. • Columns for before visit, after day 0 and day 1, and at the exit meeting. • Updated by PEV as the visit progresses. • Used by the TC to track the evaluation status and as consistency check for the Draft Statement. • Collect a copy before the visit and at the end of the visit. • (View form.)

39. PEV Visit Report (PER) • Replaces the previous VRF. • Much shorter than before. • Includes curriculum and transcript evaluation that is completed before the visit. • Includes specific information about OO&A. • Principles for completion are the same as before. • Includes the draft of the PEV’s section(s) of the Draft Statement (exit meeting statement). • TC combines all PERs into a single file for submission to HQ. • (View form.)

40. Program Audit Form (PAF) • Similar to previous form. • TC works with PEVs to produce one form for each program. • Consistency with exit meeting statement/draft statement is essential. • TC leaves forms with dean after exit meeting. • TC combines forms for all programs into a single file for submission to HQ.

41. Program Evaluation Tracking (PET) Form • Similar to the previous PEA. • Created by TC for each program from the PEWs. • Use the template. • Eliminate Program Criteria sections not used. • Combine all PETs into a single file, in alphabetical order of program name. • Used to track the evaluation status from the Exit Meeting to the Final Statement, including editing. • (View form.)

42. Short Form (SF) • One form for all programs at an institution. • Essentially the same as what is used by other commissions. • TC prepares SF after the visit. • (View form.)

43. Deliverables • From PEVs to TC (at end of visit) • Program Evaluator Worksheet (PEW) • Program Evaluator Report (PER) • Program Audit Form (PAF)

44. Deliverables (cont.) • From TC to ABET HQ (cac@abet.org) • Within 2 days after visit: Initial SF • Within 30 days after visit (5 files): • Short Form (SF) • Draft Statement (DS) • PET for each program (one file) • PAF for each program (one file) • PER from each PEV (one file)

45. Draft Statement 45

46. Draft Statement (2) • There can be at most one each of deficiency, weakness, and concern per criterion. • A shortcoming may have multiple factors that contribute to the shortcoming. • See example statements. • Note that shortcomings in program criteria are listed under Criterion 9 (Program Criteria), not merged into other criteria.

47. Draft Statement (3) • Do not use “category”. • Use “criterion” instead. • Applies to old (as well as new) criteria.

48. Draft Statement (5) Section for each program: • General description (faculty, students, etc.) • Strengths • Shortcomings • Observations

49. Draft Statement (6) Strength statements: • What was observed. • What makes it stand out above the norm. • What positive impact it has on the program.

50. Draft Statements (7) Shortcomings • Criterion citation • What was observed that causes a shortcoming • Negative impact as a result of the observed characteristics