Efficiency as a Measure of Knowledge Production of Research Universities

1. Efficiency as a Measure of Knowledge Production of Research Universities Amy W. Apon* Linh B. Ngo* Michael E. Payne* Paul W. Wilson+ School of Computing* and Department of Economics+ Clemson University

2. Content • Motivation • Methodology • Data Description • Case Studies • Conclusion

3. Motivation • Recent economic and social events motivate universities and federal agencies to seek more measures from which to gain insights on return on investment

4. Motivation • Traditional measures of productivity: • Expenditures, counts of publications, citations, student enrollment, retention, graduation … • These may not be adequate for strategic decision making • Traditional Measures of Institutions’ Research Productivity: • Are primarily parametric-based • Often ignore the scale of operation

5. Research Question • What makes this institution more efficient in producing research? • What makes this group of institutions more efficient in producing research? • How do we show statistically that one group of institutions is more efficient than the other group

6. Efficiency as a Measure • Using efficiency as a measure of knowledge production of universities • Extends traditional metrics • Utilizes non-parametric statistical methods • Non-parametric estimations of relative efficiency of production units • No endogeneity: we are not estimating conditional mean function because we are not working in a regression framework • Scale of operations is taken into consideration • Rigorous hypothesis testing

7. Background • We define as the set of feasible combinations of p inputs and q outputs, also called the production set. • There exists a maximum level of output on a given input (the concept of efficiency) • The efficiency score is an estimation with regard to the true efficiency frontier • Range: [0,1] Output Infeasible set Feasible set Input

8. Hypothesis Testing Procedure

9. Convexity Output Output Output Infeasible set Infeasible set Infeasible set Feasible set Feasible set Feasible set Input Input Input • Test for Convexity • Null hypothesis: The production set is convex • Alternative: The production set is not convex

10. Constant Returns to Scale Output Output Infeasible set Infeasible set Feasible set Feasible set Input Input • Test for Constant Returns to Scale • Null hypothesis: The production set has constant returns to scale • Alternative: The production set has variable returns to scale

11. Group Distribution Comparison • Test for Equivalent Means (EM) • Null hypothesis: • Alternative: • Test for First Order Stochastic Dominance (SD) between the two efficiency distributions: • Null hypothesis: distribution 1 does not dominate distribution 2 • Alternative: distribution 1 dominates distribution 2

12. Case Studies • University Level • Departmental Level • Grouping Categories • EPSCoR vs. NonEPSCoR • Public vs. Private • Very High Research vs. High Research • “Has HPC” versus “Does not have HPC”

13. Hypotheses Institutions from states with more federal funding (NonEPSCoR) will be more efficient than institutions from states with less federal funding (EPSCoR) Private institutions will be more efficient than public institutions Institutions with very high research activities will be more efficient than institutions with high research activities

14. University: Data Description • NCSES Academic Institution Profiles • NSF WebCASPAR • Web of Science • Aggregated data from 2003-2009 • Input: Faculty Count, Federal Expenditures • Output: PhD Graduates, Publication Counts

15. University • Test of Convexity: • p = 0.4951: Fail to reject the null hypothesis of convexity • Test of Constant Returns to Scale: • p = 0.9244: Fail to reject the null hypothesis of constant return to scale

16. University: EPSCoR vs NonEPSCoR • While the first set of EM/SD tests indicates that the distribution of efficiency scores for EPSCoR institutions does not dominate the distribution of efficiency scores for NonEPSCoR institutions, • The second set of EM/SD tests also rejects the notion that the distribution of efficiency scores for NonEPSCoR institutions is greater than the distribution of efficiency scores for EPSCoR institutions. • This implies that NonEPSCoR institutions are at least as efficient as EPSCoR institutions

17. University: Public vs. Private • The first set of EM/SD tests indicates that the distribution of efficiency scores for public institutions dominates the distribution of efficiency scores for private institutions, • The second set of EM/SD tests also supports this result by rejects the notion that the distribution of efficiency scores for public institutions is greater than the distribution of efficiency scores for private institutions. • This result shows strong evidence that public institutions are more efficient than private institutions

18. University: VHR vs. HR • This result shows strong evidence that institutions with very high research activities are more efficient than institutions with only high research activities

19. Department: Data Description • National Research Council: Data-Based Assessment of Research-Doctorate Programs in the U.S. for 2005-2006 • Input: Faculty Count, Average GRE Scores • Output: PhD Graduates, Publication Counts • 8 academic fields have sufficient data: • Biology • Chemistry • Computer Science • Electrical and Computer Engineering • English • History • Math • Physics

20. Department

21. Department: EPSCoR vs NonEPSCoR

22. Department: Public vs. Private

23. Department: VHR vs. HR

24. Implication • Efficiency estimations, together with hypothesis testing, provide insights for strategic decision making, particularly at departmental level. • Lower efficiency estimate does not mean a program is not doing well. • Issues: • Lack of data and integration/curation of data

25. Questions