1 / 29

2004 Southern Business Administration Association Meeting, Myrtle Beach, South Carolina

2004 Southern Business Administration Association Meeting, Myrtle Beach, South Carolina. The UCF College of Business Approach to Assessing Student Learning Taylor Ellis, Ph.D. Patrick Moskal, Ph.D. Associate Dean Director, Testing & Evaluation University of Central Florida.

neila
Download Presentation

2004 Southern Business Administration Association Meeting, Myrtle Beach, South Carolina

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 2004 Southern Business Administration Association Meeting, Myrtle Beach, South Carolina The UCF College of Business Approach to Assessing Student Learning Taylor Ellis, Ph.D. Patrick Moskal, Ph.D. Associate Dean Director, Testing & Evaluation University of Central Florida

  2. The UCF College of Business Approach to Assessing Student Learning Presentation Outline • UCF background. • Overview of the assessment process in the College of Business Administration. • Development of student learning outcomes. • Description and role of our Testing Lab in the assessment process.

  3. The UCF College of Business Approach to Assessing Student Learning UCF Background • Large metropolitan university with over 44,000 students. • College of Business Administration (CBA) offers 7 undergraduate, 10 Masters, and 5 Doctoral degree programs. • ~140 full-time and 60 part-time faculty in CBA. • ~8,000 students enrolled in CBA each year.

  4. The UCF College of Business Approach to Assessing Student Learning Issues Faced in UCF CBA • Number of students. • Required courses in core. • Pre-requisites in majors. • Computer Testing Lab.

  5. The UCF College of BusinessApproach to Assessing Student Learning CBA Assessment Process

  6. The UCF College of BusinessApproach to Assessing Student Learning CBA Assessment Process (continued)

  7. The UCF College of Business Approach to Assessing Student Learning Assessment Process CBA Departmental Review Committee (DRC) • DRC consists of CBA Department Chairs. • DRC Chair participates on the UAC and reports CBA assessment results. • Supported by Director of Testing and Evaluation.

  8. The UCF College of Business Approach to Assessing Student Learning Assessment Process CBA Teaching and Learning Committees • Selected for each degree program area. • Establish student learning outcomes (SLOs) and ensure data collection. • Participate in an annual review of assessment results, and plans for upcoming year.

  9. The UCF College of Business Approach to Assessing Student Learning Assessment Process • Each degree program will develop 4-10 general learning objectives (expectations). • Each objective will be evaluated by multiple outcome measures (exam questions or assessment types). • Summative scales will be developed to measure most learning outcomes. • Allows for evaluation in multiple classes. • Allows for pinpointing specific weak areas.

  10. The UCF College of BusinessApproach to Assessing Student Learning Assessment Process

  11. The UCF College of Business Approach to Assessing Student Learning Assessment Goals • Establish student learning outcomes for each degree program, based on the CBA Mission. • Acquire data that provide direction for program improvement and meet both AACSB and regional accreditation requirements. • Develop comprehensive data collection and reporting system. • Collect multiple measures over time (data triangulation). • Utilize existing data where possible. • Cause minimal intrusion on normal faculty workload.

  12. The UCF College of Business Approach to Assessing Student Learning Why Do We Need Student Learning Outcomes? • Provide feedback to students about their progress. • Communicate instructional goals to stakeholders (e.g., trustees, community, university administration, faculty, accrediting agencies, parents). • Provide the means for assessing student learning.

  13. The UCF College of BusinessApproach to Assessing Student Learning Learning Objectives Begin by developing a list of goals and objectives, such as the following “essential competencies” for all UCF Business graduates. BE2010 Competencies: • Teamwork • Communication • Creative Thinking • Adapting to change • Diversity • Ethics

  14. The UCF College of BusinessApproach to Assessing Student Learning Learning Objectives From our goals, we develop specific learning objectives. An objective is defined as a quantifiable and/or observable achievement accomplished under specifiable conditions. • How do we develop objectives? Start by answering basic questions like: • What do we want our students to know when they finish our course or program? • What should our students be able to do when they finish our course or program?

  15. The UCF College of BusinessApproach to Assessing Student Learning Objectives Vs. Outcomes • In general, an objective is what we hope that we will see, and an outcome is what we did see. • Learning objectives define the outcomes of instruction in clear terms. • Outcomes must be both observable and measurable. • All objectives/outcomes should include three primary components: the action or behavior to be measured, conditions of learning, and standards of performance.

  16. The UCF College of BusinessApproach to Assessing Student Learning You Want SMART Objectives • Specific (Clear and definable terms describing the knowledge, skills, and attitudes) • Measurable (Reliable data that can be assessed in multiple ways) • Attainable (Feasible to be accurate; need not be perfect) • Realistic (Expected standards must be practical) • Time-bound (When will results will be observed?)

  17. The UCF College of BusinessApproach to Assessing Student Learning Types and levels of student learning outcomes • Cognitive: intellectual skills. • Knowledge, comprehension, application, analysis, synthesis, evaluation (from Bloom, 1956). • Affective: what a student believes (values). • Behavioral: psychomotor skills.

  18. The UCF College of BusinessApproach to Assessing Student Learning • General Knowledge and Skills Examples • Communication • Organizational • Quantitative • Critical Thinking • Interpersonal Skills and Awareness • Group Dynamics • Motivational and Leadership • Ability to Learn and Train Others • Ability to Manage Information Technology • Global Awareness • Understanding Issues of Diversity • Ethical Awareness

  19. The UCF College of Business Approach to Assessing Student Learning CBA Assessment Types • Course embedded assessments of individual and group knowledge and skills (e.g., exams, projects). • Program level exams (e.g., doctoral candidacy exams). • Certification and standardized exams. • Direct observation. • Teamwork. • Student portfolios. • Internships and service learning. • Student records. • College, AACSB, and employer surveys. • Placement/employer data.

  20. CBA Accreditation Database Development Plan

  21. The UCF College of BusinessApproach to Assessing Student Learning CBA Assessment Database • Maintain results for each student. • Prefer automated input from electronic testing. • Allow input from additional sources. • Allow data analyses. • Create reports specific to each degree program and for diverse stakeholders.

  22. The UCF College of BusinessApproach to Assessing Student Learning CBA Testing Laboratory • 140 networked PCs • Positioned to maintain exam security. • Netsupport School Pro software to manage PCs (e.g., monitor, block unapproved access). • WebCT and InQsit1 packages available for exams. • Online student exam scheduling. 1 Developed at Ball State University.

  23. The UCF College of BusinessApproach to Assessing Student Learning CBA Testing Laboratory

  24. The UCF College of BusinessApproach to Assessing Student Learning CBA Testing Laboratory • Over 7,500 exams given in fall 2003. • Over 12,500 exams given in spring 2004. • Goal is to automatically extract specified program evaluation and accreditation data from course exams.

  25. The UCF College of BusinessApproach to Assessing Student Learning Testing Software Features • InQsit: Easy to use and generate tests (Word, text files, Publisher’s test banks). • WebCT: Complete course management system. • Test Question Types: T/F, multiple choice, multiple response, short answer, matching, identification of objects on chart/photo (InQsit), limited essay grading (InQsit), ungraded essay, survey.

  26. The UCF College of BusinessApproach to Assessing Student Learning Testing Software Features (continued) • Random selection of questions from a question pool. • Randomization of questions and answer choices. • Adaptive scoring. • Use of audio and video in tests. • Start and stop times may be set by date and time. • Analysis of Results: automatic student grading and item analysis.

  27. The UCF College of BusinessApproach to Assessing Student Learning Testing Lab Components Under Development • Lab entrance will have student ID reader. Students swipe card or enter PID for computer verification. • Proctor software will allow the faculty/proctor PC to display: • Student name and photo. • Test the student is authorized to take. • Computer assigned for exam. • Seat reservation duration.

  28. The UCF College of BusinessApproach to Assessing Student Learning Questions?

  29. The UCF College of BusinessApproach to Assessing Student Learning For more information, contact: Patrick J. Moskal, Ph.D. University of Central Florida College of Business Administration Undergraduate Student Services, BA-II, Room 101G Orlando, Florida 32816 407-823-6059 pmoskal@bus.ucf.edu

More Related