1 / 30

Use of Research Data to Assess and Improve Educational Quality and Institutional Effectiveness

Use of Research Data to Assess and Improve Educational Quality and Institutional Effectiveness. Presentation at California Community Colleges Chief Instructional Officers Workshop by Dorte Kristoffersen Associate Vice President, ACCJC. Presentation Outline. Some Definitions

dalila
Download Presentation

Use of Research Data to Assess and Improve Educational Quality and Institutional Effectiveness

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Use of Research Datato Assess and Improve Educational Quality and Institutional Effectiveness Presentation at California Community Colleges Chief Instructional Officers Workshop by Dorte Kristoffersen Associate Vice President, ACCJC

  2. Presentation Outline • Some Definitions • Characteristics of Good Data and Data Sources • Data-driven Decision-making • Purposes of Accreditation • Types of Data and Data Requirements • Examples of Effective Use of Data • Concluding Remarks

  3. Definitions of Evidence and Data General definitions: • Evidence includes everything that is used to determine or demonstrate an assertion • Data refers to groups of information that represent the qualitative or quantitative attributes of a variable or set of variables.

  4. Use of the Terms Evidence and Data in Accreditation • Evidence is all information included in the accreditation process, i.e. Self Evaluation Reports of Educational Quality and Institutional Effectiveness, information supporting the claims and descriptions in the Self Evaluation Report, interviews during the site visit, and previous External Evaluation Reports. • Data is information deliberately collected by the institution to provide it with clear information about the quality of its education and services.

  5. Characteristics of Good Data • For data to be a useful and reliable source of information and for decision-making, it needs to have the following characteristics. It should be: • Accurate and tested for validity and significance • Up-to-date and complete in terms of aspects of analysis it is meant to cover • Consistently used • From reliable sources • Longitudinal and disaggregated as appropriate I

  6. Data Sources • There are several sources of data than an institution can draw information from. Data sources can be internal, external or both. Examples of data sources are: • Institutional data, such as statistics and survey results • Demographics at the local/district, regional or federal level • Data reported to State Government and/or relevant associations, such as Cal-PASS • Other sources relevant for the institution

  7. Data-driven decision-making • It is a sound principle when making judgments to act on what we know. Judgments should not be based on assumptions and feelings. • Data is a necessary knowledge-base for decision-making • The information that an institution collects, analyzes, and reflects upon should be designed to answer questions the institution has raised against its mission and institutional objectives, and the information will thus determine what action the institution takes to support the improvement of educational quality and support student success

  8. Data-driven decision-making continued • ‘No News; is normally ‘Good News’ But ‘No data’ does not mean ‘good news’. It means no sense of bearings and no direction for the future.

  9. Purposes of Accreditation • To provide assurance to the public that education provided by institutions meets acceptable levels of quality. • To promote continuous institutional improvement. • To maintain the high quality of higher education institutions in the region/nation.

  10. Types of Data • The principle of data-driven decision-making should apply to an institution’s reflections on its achievement of its mission and institutional objectives and its compliance with the Accreditation Standards • The Commission requires various kinds of specific data • Most of the data is required to support institutions’ claims of compliance with the Accreditation Standards and some of the data relate to the U.S.D.E requirements • The Commission also expects that institutions collect data which is relevant for the institutions’ internal continuous quality improvement activities

  11. Commission Data Requirements • Data about incoming students, i.e.: • Student demographics • Student enrolment data across instructional programs • Student educational goals • Student development needs

  12. Commission Data Requirements Continued • Data about incoming students should include information about students studying in Distance Education or Correspondence Education mode, such as: • Account of programs, courses, certificates where 50% or more is offered in distance education or correspondence education mode • Annual growth in headcount enrollment

  13. Commission Data Requirements Continued • Data about Program Review, such as: • Number of enrolled students • Student learning outcomes at program and course level • Assessment results

  14. Commission Data Requirements Continued • Data on student grievances and complaints, including follow-up • Data on Internationalization, i.e.: • Numbers of international campuses • Numbers of programs for non-U.S. students • Numbers of non-U.S. students enrolled

  15. Commission Data Requirements Continued • Two types of data are particularly important as they relate directly to student success, i.e. • Student Achievement Data • Student Learning Outcomes Data • These two types of data will be explored in more detail as examples

  16. Student Achievement Data • Data on student achievement has been long required by the ACCJC. Institutions already use some common measures to examine student achievement, and some of these measures derive from federal regulatory language.

  17. Student Achievement Data Continued • Required Student Achievement Data: • Retention rates from term to term • Student progression to the next course/level of course, including from pre-collegiate to collegiate level •  Graduation rates (certificate or degree) • Job placement rates • Licensure pass rates • Transfer rates to four-year institutions

  18. Value of Student Achievement Data • Student Achievement Data is end-point data providing an institution with key information about the achievement of its institutional mission and in terms of student success • Such data will: • If collected longitudinally and analyzed continuously provide the institution with indications as to whether improvements in pedagogy or services may be required to improve student progression and completion • Keep institutions informed about fluctuations and serve as a warning if rates decrease and trends need to be reversed

  19. Value of Student Achievement Data Continued • Student achievement data will also: • Provide information to the institutions about barriers to completion and transfer • Present trends that identify the need for institutions to explore the barriers through the collection of additional data and/or conduct separate surveys • If collected in disaggregated form inform institutions as to whether all students progress and complete their studies or if attention needs to be given to special groups

  20. New Commission Requirement • As of Fall 2012, Self Evaluation Reports of Educational Quality and Institutional Effectiveness should include data on key student achievement measures in disaggregated form by age, gender, race/ethnicity, socio-economic status and other measures that the institution considers relevant for its population

  21. An Example of Student Achievement Data • Research study by Institute for Higher Education Leadership and Policy (IHELP). Divided we Fail: Improving Completion and Closing Racial Gaps in California’s Community Colleges, October 2010 • IHELP followed 250.000 degree seeking Californian community college students over the course of six years • The goal of the study was to use it to identify solutions to improve completion rates for all students and closing the disparities across racial/ethnic groups • The study is conducted at a generic level, but the findings and how the can be used can serve as an example for individual colleges

  22. An Example Continued • Too few students reach the milestones: racial/ethnic disparities abound: • Too many students fail to complete • Six years after enrolling 70% of degree-seeking students had not completed a certificate or degree and had not transferred to a university (most had dropped out) • Failure rates among black students (75%) and Latinos (80%) are higher

  23. An Example Continued • Critical milestone is missed: • Only 40% of degree-seeking students had earned at least 30 college level credits at a CCC. • 35% of Latino students and 28% of black students reached this milestone • Fewer Latinos who reach the above milestone complete a certificate, degree or transfer (47%) as compared to white (60%), Asian-Pacific Islander (58%) and black (53%) students

  24. Student Learning Outcomes Data • The Commission also requires institutions to demonstrate educational effectiveness by collecting and analyzing student learning outcomes data • Student Learning Outcomes data: • Is end-point data • Consists of learning outcomes at course, program and degree level • Is derived from summative assessment

  25. Value of Student Learning Outcomes Data • Student Learning Outcomes data: • Provides information about how well students are able to achieve institutional and programmatic missions and objectives, and • Is therefore a key source of data for Program Review • May help institutions identify barriers to student learning and student success and how these barriers can be addressed • May identify whether improvements in curriculum and teaching and learning methods are warranted to increase the achievement of learning outcomes

  26. Key Questions to Ask about Data Collection and Data Use • What is the mission of the institution (Institutional, Programmatic)? • What are the institutional objectives? • What is the institution trying to achieve in terms of its institutional missions and educational quality, including possible goals and targets? • What data is relevant to collect to gain information on institutional achievements?

  27. Key Questions to Ask continued • What are the data sources, i.e. internal and/or external? • Who should be involved in the data analysis? • How should the outcomes of the analysis be reported? • Who is responsible for taking action and ensuring that improvements are made?

  28. Concluding Remarks • Data is collected intentionally and its meaning and relevance has been considered • It is purposeful and designed to provide information on institutional achievements • It is collected systematically and continuously • Data is interpreted, reflected and acted upon • It is used for improvement

  29. ‘Learning without thought is labor lost; thought without learning is perilous.’ (Confucius 551-470 BC)

  30. Thank you ACCJC/WASC 10 Commercial Blvd, Suite 204 Novato, CA 94949 Tel: 415-506-0234 Fax: 415-506-0238 Website: www.accjc.org Email: accjc@accjc.org

More Related