McCoy College of Business Administration (MBA Program)

1. McCoy College of Business Administration (MBA Program) • Why Assessment? • We give grades don’t we? • What is an Assessment Program? • What can we do? • How do we do it? • Special thanks to Dr. Doug Eder and Dr. Kathryn Martell for much of the content in this presentation.

2. The First and Only Goal: To teach for long-term retention of information and application to new situations --after Halpern & Hakel

3. Deep (expert) learning is a curriculum function. It is not a course function. Universities are systems

4. Direct vs. Indirect Assessment • Direct assessment acquires evidence about student learning and the learning environment: Exams, projects, logs, portfolios, observations.... • Indirect assessment acquires evidence about how students feelabout learning and their learning environment: Surveys, questionnaires, interviews, focus groups, reflective essays....

5. Whatever Assessment Is... ...It’s Multiple Measures Over Time.

6. Why Assessment Won’t Go Away • Limits on money • Shifts in public view of Higher Education • State laws and education standards • Accountability to accrediting agencies to maintain operating credentials * * * * * Result: We must demonstrate quality of outcome and return on investment. Through assessment educators meet responsibilities to themselves, to students, and to the public.

7. What is “Program-Level” assessment as specified by AACSB? • The college or school must specify program-level learning goals for each separate degree program. Generally, such goals are anticipated for each degree, not for separate majors or concentrations within a degree. For example, the McCoy College offers a Bachelor of Business Administration (BBA) degree with defined majors in accounting, CIS, economics, finance, management, and marketing. A set of program-level learning goals for the BBA degree must be provided and assessed; goals for each major will not be required for AACSB program review accreditation. The MBA program has program-level learning goals reflecting graduate-level learning outcomes.

8. Strongest Links:The Scholarship of Teaching • Clear Goals • Adequate Preparation • Appropriate Methods • Significant Results • Effective Presentation • Reflective Critique

9. Assessment vs. Evaluation • Assessment focuses on the student and the learning environment. • Evaluation focuses on the professor and the teaching performance.

10. We Give Grades, Don’t We? Aren’t grades (by themselves) enough? NO!

11. When used alone, grades are no longer regarded as adequate indicators of student learning. • Professor autonomy: Grades in one course or section may be recorded using a different standard than grades in another. • General accountability: Much of the current public annoyance with higher education comes from a lack of skill and content mastery by students who have received good grades.

12. Grades may reflect many things besides student mastery of course objectives: Verbal ability Participation Cooperation Extra credit Attendance Effort Criterion Performance vs. Value Added

13. Good Assessment Asks Good Questions • Do we have a curriculum or a set of common courses? Which of these do our students experience? • What do we want our students “to be” or “to have” when they have completed the program? What are our curricular goals? • What do our students “do” to demonstrate accomplishment of curricular goals (i.e. course objectives)? How much and what kinds of writing? What kinds of math? Oral Presentations? Analysis and Reports? Are these relevant to our curricular goals? • How do we demonstrate to ourselves that students are achieving or have achieved curricular goals?

14. Graduate Common Core Courses • 1. MGT 5314 Organizational Behavior and Theory • 2. ECO 5316 Managerial Economics • 3. MKT 5321 Marketing Management • 4. QMST 5334 Advanced Statistical Methods for Business • 5. ACC 5361 Financial and Managerial Reporting and Analysis • 6. FIN 5387 Managerial Finance • 7. CIS 5354 Decision Support Models in Management • 8. MGT. 5313 Administrative Policy

15. GRADUATE LEARNING GOALS (MBA)Upon completion of the Master of Business Administration (MBA), a student will be able to: • 1. Integrate knowledge of fundamental business disciplines to effectively manage domestic and global organizations in a dynamic environment. • 2. Integrate appropriate information technologies for managing business data for decision making, enhancing productivity, and communicating with others. • 3. Evaluate analytical skills and critical thinking processes used in business decision making. • 4. Evaluate the issues associated with ethical leadership and conducting business in an ethical, legal, and socially responsible manner. • 5. Demonstrate the ability to communicate effectively, both orally and in writing, in new and unfamiliar circumstances. • 6. Apply the skills needed to effectively lead and contribute to dynamic workgroups.

16. An Assessment Question: How Do You Know... ...that students walk out your door looking like you want them to? What behaviors have they exhibited or products have they produced? What are the indicators for your goals?

17. Goals and Objectivesfor Students • Goals express what we want our students to be. • Objectives describe what we want our students todo. ..... • Objectives are indicators of goals.

18. Graduate Faculty Teaching Common Core Courses Should: • Talk to each other (if multiple faculty teach the course) . . . • Agree on specific objectives for the course . . . • Make sure course objectives are relevant to Program-Level goals . .

19. To be assessment friendly, objectives should: • Focus on students • Make the learning goals visible (serve as indicators) • Describe behaviors or products (doing, making) that can be captured by assignments.

20. Bloom’s Taxonomy LEVEL SOME COGNITIVE BEHAVIORS • Evaluation Appraisal of an Analysis or Synthesis • Synthesis Assembly of Application • Analysis Disassembly of Application • Application Use of Understanding • Understanding Management of Knowledge • Knowledge Memorization of facts, language, concepts, principles, theories

21. The Next Step . . . • Graduate Faculty Should Agree on How Best to Measure Student Performance on each Objective Contributing to Program-Level Goals.

22. Assessment days or centers Case studies Classroom assessments Completion and retention studies Content analyses Debates Direct observations Focus groups Graduate success Internships and service learning Interviews (including videotapes) Exams for certification and licensure Matrices Performances Portfolios of several kinds Projects (Primary Trait Analysis) Questionnaires and surveys (Direct and telephone; employer, alumni, and student attitude and satisfaction) Reflective essays Study and activity logs Tests (Locally-developed and standardized) Transcript analyses Some Assessment Ways and Means

23. Another Step . . . • Depending on the method (ways and means) used to assess a course objective, develop a rubric or a method of measurement. • Evaluate student performance on the specific objective on three levels: Exceeds Expectation; Meets Expectation; Does Not Meet Expectation. • Report your findings. • Discuss the result(s), determine any actions to take (if necessary).

24. Example Rubric for MGT. 4335

25. Responsibilities of Core Course Coordinators • 1. completing the Course Alignment Matrix which specifies which program-level learning goals are covered in a specific core course, • 2. developing course learning objectives that lead to fulfillment of program-level learning goals, • 3. collecting data from faculty teaching course sections to support or measure student learning of program-level learning goals (including a representative sample of measurement devices/techniques/assignments to be maintained in the office of the core course coordinator), • 4. reporting collective results on program-level learning goals to the McCoy College Assessment Committee, and • 5. discussing and acting on specific recommendations made by the college assessment and curriculum committees.

28. What documentation must be retained by the Core Course Coordinators? • Core course coordinators should retain copies of instruments used for direct assessment of student learning, such as assignments, written products, test questions, rubrics, scoring grids, etc. The course coordinators also must retain sample student products relating to the instruments used. Aggregate results of direct assessments for various program-level learning goals gathered and/or submitted to the assessment committee should be retained by the course coordinator by semester or year for review. Minutes of core course faculty meetings should be retained with special emphasis given to items concerning improvements in instruments used to assess student learning and acknowledgement and review of data.

29. Must all students be assessed? • For the purpose of meeting AACSB-International standards relating to Assurance of Learning (16, 18 and 20), sampling may be utilized as long as it is representative. Faculty teaching college core courses for various programs, as scholars in their respective fields and under the leadership of the Core Course Coordinator, have the responsibility for determining appropriate representative sample sizes. Graduate core course coordinators may use census data from graduate core courses for assessment purposes.

30. Should college core course faculty and coordinators gather data and submit results every year? • Program-level learning goals for undergraduate (BBA) and graduate (MBA) programs are reviewed on a scheduled rotational basis. At least two program goals are reviewed each year, and college core courses covering that particular program goal are required to submit the results of course-embedded assessment to the assessment committee during the scheduled review year. Since the McCoy College views assessment as a continuous process, core course coordinators and core course faculty are encouraged, but not required, to gather data each semester for the reporting of assessment results.

31. Example Undergraduate Goal Rotation • Program-Level Goal Rotation Sequence: • Academic Year 2005-2006: • Goal 1 – Conceptualize a complex issue into a coherent, persuasive written or oral statement. • Goal 5 – Understand the importance of group dynamics in achieving organizational goals and use the skills needed for effective teamwork. • Academic Year 2006-2007: • Goal 2 – Use critical thinking skills to evaluate information, solve problems, and make sound decisions. • Goal 3 – Use information technology skills in decision-making. • Academic Year 2007-2008: • Goal 4 – Apply general concepts of ethical behavior in dealing with stakeholders. • Goal 6 – Understand the importance of culture and diversity.

32. We have never been asked... ...to assess everything all at once.

33. Who sets performance standards? How should Core Course Coordinators report student performance on various college-level learning goals? • For each program-level learning goal assessed in a college core course, the faculty will determine their minimum expectation or standard. For reporting data on any specific program-level learning goal, core course coordinators should aggregate data and report the results by three levels: Exceeds Expectations; Meets Expectations; or Does Not Meet Expectations. In order to preserve the integrity of the assessment process, core course coordinators should release aggregate data by course (all sections) and not individual results by the professor teaching the core course. Program Assessment in the McCoy College is not the evaluation of faculty performance in the classroom; it concerns the aggregate level of student learning of various program-level goals.

34. Example Report on Program Goal

35. An Important Lesson from the Farm A pig doesn’t get any fatter merely by weighing it.

36. How many students must meet or exceed the performance standard on a college-level program goal? There is no prescribed percentage of students that must meet the standard articulated by faculty. AACSB and other constituent groups will look at whether and how these data are used for continuous improvement purposes. Thus, a poor showing of student mastery of a given program-level learning goal would only be a concern if the curriculum was not subsequently modified to improve student skills/performance of that learning goal. A second emphasis in the Assurance of Learning Standards is that program-level learning goals are used to communicate competencies of graduates to students, employers and other constituents. Thus, faculty have the responsibility to develop and modify curricula to insure that program-level goals are achieved by most graduates and not just a portion.

37. What Assessment Asks of You! • One day of initial development time to determine course objectives and agreeing on a grading rubric for specific course objectives. • Two to three hours per semester (depending on the goal rotation) to extract student performance on course objectives leading to the fulfillment of program-level goals. • Writing a short report (1 page) concerning student performance. • Maintaining examples of student work, grading rubrics or other examples demonstrating student achievement of course objectives. • Talking to other faculty about continuously improving classroom processes and products to improve student learning of program level goals over time.

38. Closing Thought The enemy of the good is the perfect.