College Ranking Metrics

1. College Ranking Metrics Where Does UMKC Fit in the Rankings?

2. Why pay attention to rankings? • Although rankings are terribly imprecise, some of the metrics used are important • Prospective students pay attention and compare institutions on a number of indicators • UMKC can use metrics to benchmark against peer institutions and identify areas for improvement

3. Select National University and College Rankings • Undergraduate Education • U.S. News & World Report • America’s Best Colleges 2009 • http://colleges.usnews.rankingsandreviews.com/college • Forbes.com (The Center for College Affordability and Productivity) • America’s Best Colleges 2008 • http://www.forbes.com/2008/08/13/college-university-rankings-oped-college08-cx_rv_mn_0813intro.html • Research Productivity and Graduate Education • The Center for Measuring University Performance • The Top American Research Universities • http://mup.asu.edu/research.html • National Research Council (New rankings due Spring 2009) • Assessment of Research-Doctorate Programs (1995) • http://sites.nationalacademies.org/pga/Resdoc/index.htm

4. US News and World Report Rankings • Schools are categorized by mission derived from Carnegie classification • National Universities • Offer full range of undergraduate majors, plus master’s and Ph.D. programs, and emphasize faculty research • 262 institutions, including UMKC • Gather data from each college for up to 15 indicators of academic excellence • Each factor is assigned a weight that reflects how much it matters • Colleges are ranked against their peers based on their composite weighted score

5. US News Tier Structure Top Schools Institutions ranked 1 to 130 Tier 3 Institutions ranked 134 to 188 UMKC: Ranked 165th Tier 4 Institutions ranked 197 to 260

6. 2009 U.S. News MethodologyMeasures and Their Weights

7. U.S. News Measures by Cluster Selectivity Graduation & Retention • SAT/ACT • Top 10% • Acceptance • Freshman Persistence • Graduation • Grad Rate Performance Peer Assessment 15% 25% 25% Financial Resources Faculty Resources 10% • Class Size • Salaries • Faculty • Student/Faculty Alumni Giving 5% 20%

8. Issues with U.S. News Metrics • Peer assessment: reputational only, heaviest weight • Graduation/retention rates: insensitive to different populations; reported only for limited subset of students (first-time, full-time freshmen) • Faculty resources: class sizes used not necessarily ideal; model biased towards small colleges • Student selectivity: ACT/SAT scores at entrance are input variables, not a reflection of institutional performance • Financial resources: educational expenditures per student are difficult to compare across institutions • Alumni Giving: a proxy for student satisfaction with their alma mater; only counts undergraduate giving over the past two years, and only counts former students who received a degree • Best for small, homogeneous liberal arts colleges (i.e., Ivy League Universities and Private Liberal Arts Colleges)

9. Where Does UMKC Rank?

10. UMKC’s Operational Peers

11. UMKC’s Aspirational Peers

12. UMKC’s Peer Institutions(as identified by UM System) • Cleveland State University* • East Tennessee State University • George Mason University • Georgia State University* • Indiana Univ. - Purdue Univ./Indy* • SUNY at Buffalo • Temple University* • University of Akron • University of Alabama - Birmingham* • University of Arkansas at Little Rock • University of Cincinnati* • University of Houston - University Park* • University of Illinois – Chicago* • University of Louisville • University of Memphis* • University of Nevada - Las Vegas • University of Pittsburgh* • University of South Alabama • University of South Florida • University of Toledo* • University of Wisconsin - Milwaukee* • Virginia Commonwealth University* • Wayne State University* • Wright State University * Members of the Urban 21

13. Where Can Improvement be Made? Measures driven by single year data • Peer Assessment • SAT/ACT Scores • Top 10% Freshmen • Class Size • Full-time Faculty • Faculty with Terminal Degrees • Acceptance Rate • Student/Faculty Ratio • Graduation Rate Performance Measures driven by multiple-year data • Expenditures per Student (2 yr avg.) • Faculty Compensation (2 yr avg.) • Alumni Giving (2 yr avg.) • Freshman Persistence (4 yr avg.) • Graduation Rate (4 yr avg.)

14. Forbes.com Rating Factors

15. Issues with Forbes.com Ratings • 569 schools ranked; every different type of four year institution is ranked together (apples and oranges) • RateMyProfessors.com as a measure of student learning? • Who’s Who in America used as a dubious measure of the achievement of graduates (and as a proxy for academic quality) • Ranking schools based on accumulated debt – also a proxy for the net worth of the students attending the institution • Only one school in the top 50 has an enrollment over 10,000 (average N of top 50=2,285) • “Forbes methodology … reveals a profound bias in favor of wealth and class and against institutions that have distinctive missions – especially those that serve low income students and significant populations of minority students.” (Patricia McGuire, Inside Higher Education, August 28, 2008)

16. UMKC’s Operational Peers

17. UMKC’s Aspirational Peers

18. The Center for Measuring University Performance Rankings • Determines the Top American Research Universities by their rank on nine different measures: • Total Research Expenditures • Federal Research Expenditures • Endowment Assets • Annual Giving • National Academy Membership • National Academy of Sciences, National Academy of Engineering, and Institute of Medicine membership directories • Faculty Awards • Fulbrights, Guggenheims, National endowment for the Humanities Awards, NSF CAREER Awards, Sloan Fellowships, etc. • Doctorates Granted • Postdoctoral Appointees • SAT/ACT range

19. Issues with The Center Metrics • Focused on research productivity and prestige, designed to assess research excellence, not educational excellence, so metrics focused on research productivity of faculty, not educational programs • Funded research, especially federally funded research, seen as most prestigious; other types of scholarship and creative activity not as valued • Some emphasis on faculty awards compensates for heavy emphasis on funded research • Overall rankings reflect numbers of faculty and types of programs – Larger Universities with Medical Schools, Engineering Schools, and Agricultural Schools do best (i.e., large land grant Universities) • These rankings not concerned with Universities with other missions (e.g., urban serving institutions)

20. UMKC’s Operational Peers

21. UMKC’s Aspirational Peers

22. UM System Accountability Metrics • UM System metrics found within various rankings: • US News Rankings • Freshmen persistence • Rate six-year graduation rate • Percent of tenured and tenure track faculty with terminal degrees • Number of entering students in the top 10% of high school graduation class • Ranked faculty salaries compared to peers and market • Faculty and staff benefits compared to peers and market • Freshman applicant acceptance rate • Educational expenditures per FTE student • Alumni giving percent • Forbes.com Rankings • Number of student awards • Number of faculty awards • Average undergraduate debt burden upon graduation • Four-year graduation rate • The Center Rankings • Total externally sponsored research expenditures • Total federally funded federal research expenditures • Annual fundraising • Number of faculty awards, fellowships and memberships • Total degrees awarded by level (including doctorate)