1 / 17

A case study of one institution’s approach to institutional research

A case study of one institution’s approach to institutional research. Penny Jones Elizabeth Maddison University of Brighton. Preliminaries: definition and purpose.

bela
Download Presentation

A case study of one institution’s approach to institutional research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A case study of one institution’s approach to institutional research Penny Jones Elizabeth Maddison University of Brighton

  2. Preliminaries: definition and purpose • ‘Self-study is about collective reflective practice carried out by a university with the intention of understanding better and improving its own progress towards its objectives, enhancing its institutional effectiveness, and both responding to and influencing positively the contact in which it is operating. As such, self-study is intimately linked to university strategy, culture and decision-making – with an emphasis on each of the collective, reflective and practical components of this definition’ From ‘Managing Institutional Self-Study’ by David Watson and Elizabeth Maddison, 2005

  3. University of Brighton • >21,000 students; >2,000 staff; >£135m turnover • >5,500 awards 2007 • submitted 287 staff in 16 RAE units of assessment • highly distributed (five sites; UCH; four partner colleges) • joint medical school (first graduates July 2008) • major funding from HEFCE, TDA and NHS

  4. University context • national debate on and requirements for accountability • HEFCE, TDA, NHS, PSBs etc • Better Regulation • ‘single conversation’ • CUC Pls guidance • the Accountable Institution Project (HEFCE-funded; 3 universities)

  5. University context • 1999 no real analytic capacity • problematic HESES return • 2000 first data analyst appointed on fixed term contract • 2008 two permanent data analyst posts plus one part-time survey post about to be filled • continuous improvement in data quality • 2008 clean data audit from HEFCE

  6. University context • 2007 ‘basket of indicators’ approved by Board of Governors as basis for their own monitoring of institutional performance against Corporate Plan and reporting for HEFCE • significant time series including student retention; surveys of student finance; why chose Brighton / decliners • targets for Faculties (e.g. research grants bid and won; research student completions; commercial income)

  7. Critical success factors in IR at Brighton • senior management commitment; SU involvement • data quality improvement and sustained effort • real examples where data is informing practice and decision-making, and / or identifying questions to be addressed • feeding in at key moments (e.g. ‘what we know about what students think’) • expectation that Heads know the ‘facts’ about their Schools; will investigate / challenge / respond / change practice

  8. Using a data framework in an effective wayFrom ‘Managing Institutional Self-Study’ by David Watson and Elizabeth Maddison, 2005. • integrate the data cycle with the committee cycle, including Board of Governors • focus on Brighton’s objectives and practices • focus on performance indicators identified in corporate plan and assessing them in appropriate ways • keep it well organised and managed to fulfil internal and external requirements • ensure it supports risk management • The data framework at the University of Brighton

  9. Challenges • timeliness of analysis • data quality – and understanding when/where data does not have to be perfect • balancing analysis for information only with analysis to support and/or challenge decision making • to improve the quality of analysis over time, and with changing requirements • data literacy – communicating analysis using different modes to provide appropriate access to different users

  10. 1. The Retention Report – an example of analysis well integrated into university cycles Student Cycle HESA return 07/08 HESA return 06/07 Analysis Cycle Registration Committee Cycle Board of Governors A S J O HESA Performance indicators Senior Management Team J N M D HESES Return 07/08 Academic Standards Committee A J M F Withdrawals survey Student Retention Review Group RETENTION REPORT – Student cohort 06/07 • Addressing data literacy • Report on the web • Hard copy of the report sent out to key customers • Lunch time seminar tailored to attendees • An offer of one to one sessions with analyst Budget agreed for retention issues

  11. 2. The National Students Survey – using incomplete data and other challenges • results published at JACS subject level do not map to internal schools and faculties. • data only published at ‘department‘ level if threshold of 10 or more met. • an example of the complexity…

  12. The complexity JACS Level 3 SCHOOL A -‘departments’ (with number of respondents) Sociology (116) SCHOOL B BA Hons Social Science (30) BA Hons Criminology and Sociology (47) BA Hons Criminology and Social Policy (18) BA Hons Health and Social Care (13) BA Hons Sociology and Social Policy (11) BA Hons Criminology and Applied Psychology (77) BA Hons Applied Psychology and Sociology (36) BA Hons English and Sociology (22) Social Policy (192) SCHOOL C Others in Subjects Allied to Medicine (74) SCHOOL D Psychology (113) SCHOOL E Unidentified Respondents from departments > 10 respondents English Studies (54) SCHOOL F

  13. The NSS – the challenge continued… • difficult to ask academics to be accountable for data where we are unsure who the respondents making up the data are • why it matters… Unistats website • resolution this year – NSS willing to provide JACS mapping to make unpicking the results easier. • increase response rates – more data at a lower level • good example of difficulty in balancing analysis for info only and for challenge

  14. 3. The ‘dashboard’ – improving analysis over time • new corporate plan 2007-2012 • opportunity to improve high level analysis provided to senior management and Board of Governors • undertook comparator group analysis and researched dashboard techniques • resulting UoB Dashboard • the challenges

  15. Tensions From ‘Managing Institutional Self-Study’ by David Watson and Elizabeth Maddison, 2005

  16. Still to do • herd the plethora of people involved in data analysis and evaluation (practitioners and academics; quantitative and qualitative) • bring together data to give complete perspective on each School (e.g. NSS; clearing %; Retention; student and staff data; student complaints / appeals) • clearer processes and timetable (revisiting data cycle and framework) • reduce reinvention • review external frameworks (e.g. CSR) • align/dialogue between ‘IR’ and academic HE research interests • improve level of analysis (school; course; subject)

  17. Still to do • agree definitions (research; ‘third stream’) • continuous attention to data quality and for collecting, using and reporting on data • inter-institutional comparisons • contribute to national debate (e.g. metrics for community engagement) • technical capacity • market intelligence • is ‘good enough’ ‘good enough’? • continuous attention to ‘so what’? • avoid spurious veracity

More Related