Social Justice in HE Evaluation: Opportunities and Dangers

1. Social Justice in HE Evaluation: Opportunities and Dangers Mala Singh Centre for Higher Education Research and Information, The Open University, UK m.singh@open.ac.uk

2. Simple Story Line • Contemporary higher education is enveloped in accountability-increased reporting to range of external stakeholders • Evaluation is a policy instrument of accountability-demonstrating compliance • Quality is a code for performance, responsiveness and success in competition MSingh NYU 07May2009

3. Feel-good or public good proposal? • If accountability demands/evaluation is here to stay, should Social Justice be included in evaluation systems? • USA and South Africa as examples of systems where this approach has been tried • Extending the scope of evaluation to include SJ-opportunities, limits and dangers MSingh NYU 07May2009

4. Contested Issues:What is at stake? • What is the measure of excellence in higher education • Who is entitled to define it and judge its achievement-by what means • Who is able to compel its demonstrability MSingh NYU 07May2009

5. Social Justice in HE Evaluation • Including the historically excluded- disjunctures between legislation, policies, institutional cultures and practices • Social justice-politics of Recognition? Distribution? Transformation? • Premise-that higher education can contribute to addressing social injustice both directly and indirectly MSingh NYU 07May2009

6. Setting the Scene • Far-reaching changes to higher education as learning, research and work setting • Impressive massification -globally 51 m students in PSE in 1980 to 140 m in 2006 (Teichler) • Continuing inequality and exclusion-globally 80% participation rates in HICs and 7% in LICs; nationally US 41% white, 32% black and 24% Hispanic(Altbach); SA 61 % white, 16 % black, black completion rates after 5 yrs 30 % with 56 % dropping out(Scott) MSingh NYU 07May2009

7. Three Takes on Evaluation • Evaluation as central part of HE-key to ‘guild authority’, credentialling, reputational hierarchies, resource allocation and rewards (Henkel)-peer review • Evaluation for social reform and monitoring social progress-US in 1960s • Evaluation as a policy instrument of the ‘evaluative state’ to ensure social accountability and stakeholder responsiveness of HE MSingh NYU 07May2009

8. Evaluation and Social Reform • Evaluation as a professional field in service of social reform in the Johnson era (USA 1960s) • Great Society programmes aimed at racial injustice and poverty-evaluations of interventions in education(school curricula, health, welfare) • Approaches in evaluation studies about social justice, participatory evaluations, potential for deliberative democracy-House, Howe, Patton. MSingh NYU 07May2009

9. The ‘Evaluative State’ • Trends in UK/European HE in mid eighties-shift from state control to state steering • State use of intermediary agencies to monitor performance and provide consumer information; self-regulation • Increase in private and privatised provision • Policy isomorphism in reform discourse • 200 members of INQAAHE worldwide MSingh NYU 07May2009

10. Key Issues • The rise of the ‘evaluative state’- juridification and contractualism(Neave) • Shifting power balances in the Clark triangle of state, market and academe • Systems theory-from inputs and processes to outputs in judging HE • Heightened accountability, increased power of evaluation, continuing inequalities, social justice underserved by states and markets. MSingh NYU 07May2009

11. USA and SA Evaluation Systems • USA-institutional and subject accreditation –somewhat voluntary system owned by HEIs, used for improvement and to demonstrate quality levels to be eligible for federal funds. Inclusion of diversity considerations in some systems • SA –institutional audit and programme accreditation-mandatory national system used for improvement and eligibility for programme funding. Inclusion of social justice/social transformation MSingh NYU 07May2009

12. Diversity in US Evaluation • Most evaluation systems look at institutional mission but not in a directive way iro SJ • WASC Statement on Diversity and Criteria-response to changing demography of the state and its reflection in HE(1980s) • Demographic diversity-students, faculty, governing boards-from affirmative action to diversity • Curriculum, pedagogy, campus culture and experience MSingh NYU 07May2009

13. Diversity in US Evaluation • Diversity is a measure of quality education-cosmopolitanism, tolerance, engagement with other cultural perspectives, preparation for world of work and civic participation-tapping human potential of all citizens • Accusations of punitiveness and social re-engineering; diversity focus about neither education nor quality MSingh NYU 07May2009

14. Diversity in US Evaluation • Proposition 209(1996)-no direct focus on race, ethnicity and gender in public HE • Shift from access to retention and graduation rates MSingh NYU 07May2009

15. Social Justice in SA Evaluation • Post 1994 reconstruction of SA society and HE-social justice and transformation • Reflected in evaluation system(2004) –criteria for institutional and programme evaluation • Demographic representivity(race, class, gender, disability) as well as social transformation-institutional culture, curriculum change, new pedagogies, new research themes and new community partnerships MSingh NYU 07May2009

16. USA and SA Evaluation Systems • Different systems-different languages to designate a similar social concern about education, quality and inclusion • Diversity/SJ focus in criteria, training of evaluators, in the panel engagement with staff and students, in self-review and agency report and recommendations MSingh NYU 07May2009

17. USA and SA Evaluation Systems • Easier to look at input measures and measurable achievements • How to measure SJ competencies-no standards and indicators set for them-may help to avoid SJ fundamentalism in evaluation • SJ defined in relation to dominant contextual imperatives-no Platonic absolutes-part of democratic debate and engagement MSingh NYU 07May2009

18. In search of conclusions • Opportunities-exploring the interface of excellence and inequality in HE • Inserting dialogic engagement on social justice/transformation issues pertinent to context into evaluation • Counterposing search for market competencies with broader social competencies and impact MSingh NYU 07May2009

19. In search of conclusions • Limits-what educational outcomes can be measured in evaluation • Easier iro numbers-demographic diversity in governance/ student body/faculty and administrators, curriculum change • More difficult to measure iro value and worldview outcomes MSingh NYU 07May2009

20. In search of conclusions • Dangers-bringing state/party political agendas back into academe with no guarantees as to how evaluation findings will be used • Increasing onerousness of evaluation and burden of evidence on academics • Using SJ to withhold accreditation MSingh NYU 07May2009

21. Unresolved Issues • Is SJ a constitutive component of quality and excellence or only a framing condition? • Who can best organise HE and make it work in the public interest-states, markets or academe? • Evaluation as positivist science or hermeneutics? Will strong accountability demands be satisfied with interpretative rather than quantitative findings? MSingh NYU 07May2009

22. Unresolved Issues • SJ in transnational contexts-whose SJ? • Expecting too much of HE? • Populist proposal-save the world outside of evaluation and HE(Fish)? • Bringing together the three takes on evaluation-guild authority, social reform and excellence in the public interest MSingh NYU 07May2009