1 / 23

2014-15 Budget Measure

Quality Indicators for Learning and Teaching 2014 Survey Managers’ Information Forum 18 July 2014 Dr Andrew Taylor Branch Manager, Higher Education Data and Analysis. 2014-15 Budget Measure. 2014-15 Budget measure Upholding Quality – Quality Indicators for Learning and Teaching (QILT)

istas
Download Presentation

2014-15 Budget Measure

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quality Indicators for Learning and Teaching2014 Survey Managers’ Information Forum18 July 2014Dr Andrew TaylorBranch Manager, Higher Education Data and Analysis

  2. 2014-15 Budget Measure • 2014-15 Budget measure Upholding Quality – Quality Indicators for Learning and Teaching (QILT) • QILT – suite of surveys over the student life cycle, from commencement to employment • New, purpose built website to publish results and inform student choice

  3. Development of Indicators • Indicator development for performance funding • AQHE Reference Group, chaired by Professor Ian O’Connor • Consultation process: • Discussion papers • Submissions • Roundtables

  4. Key issues • Survey burden on students and institutions • Centralised administration • Employer surveys ad hoc and not well done • Need for robust measures of teaching performance

  5. AQHE Report Key Reference Group recommendations: • Centralised administration of the suite of performance measurement instruments under a single third-party provider • The suite of instruments to initially comprise the University Experience Survey (UES) and a new Graduate Outcomes Survey (GOS) • Scoping study on a new instrument to assess employer needs and satisfaction with graduates Not recommended: • Composite Teaching Quality Indicator • Collegiate Learning Assessment

  6. MyUniversity website • Live in April 2012 • Clear, meaningful and transparent information • Confidence intervals • Subject by institution level data

  7. Data sharing • Amendments made to the Higher Education Support Act 2003 (HESA) in November 2013 • Personal information can now be provided to a range of bodies, including • TEQSA • higher education providers and some peak bodies • state governments • Safeguards remain • Limited uses • No on-sharing • Provider consent • Universities Australia data sharing agreement allows access for member institutions

  8. QILT Components • Three surveys: • University Experience Survey (UES) • Graduate Outcomes Survey (GOS) • Employer Satisfaction Survey (ESS) • Plus new purpose built QILT website to publish results from the surveys • QILT contractor selected through tender to administer all elements

  9. Timing

  10. University Experience Survey • 2014 UES administered by GCA/SRC consortium • 2015 UES and 2016 UES comprise part of QILT • Methodology in line with previous administrations

  11. National results, key items, 2011-2013

  12. National results, scales, 2012-2013

  13. International benchmarking

  14. Graduate Outcomes Survey • GOS will provide timely labour market information • Review existing AGS instrument to develop new GOS instrument and methodology • Reporting methods need review • Survey administered in two rounds October and April (following year), though alternative proposals considered • 2015 AGS administered by GCA • GOSto replace AGS from the second half of 2015 • 2016 GOS and 2017 GOS comprise part of QILT

  15. Employer Satisfaction Survey • ESS measures employer satisfaction with generic skills, technical skills and work readiness of graduates • Recent graduates provide contact details of direct supervisor • Pilot ESS in 2013-14 across 4 universities and 5 fields of education • Pilot ESS report available from: www.education.gov.au/employer-satisfaction-survey • Further development of ESS in October-December 2014 • instrument and survey methodology to be reviewed and tested on larger scale • universities participate opt-in basis • 2014 ESS trial, 2015 ESS and 2016 ESS comprise part of QILT

  16. ESS – key results Overall 92 per cent of employers stated they would be confident in recommending someone with the same qualification from the same university for a similar position to the graduate’s role Overall satisfaction by key factors (skill clusters) • Foundation skills — 95 per cent • Adaptive skills — 89 per cent • Teamwork and interpersonal skills — 96 per cent • Disciplinary skills — 93 per cent • Employability skills — 93 per cent • Enterprise skills — 79 per cent

  17. QILT Website • QILT website aims to inform student choice through provision of survey results by institution and course • Two phases: • Initial release • August 2014 release • Existing data from the AGS and UES, to be provided by department • Limited scope – undergraduates at Table A and B universities • Further development • Developed progressively to August 2015 • Expanded scope – all institutions and levels of study • Enhanced functionality, e.g. search and comparison functions enhanced • Development and release of mobile app • Website updated every six months from August 2014 with latest available QILT survey data

  18. Governance • QILT Working Group will provide advice to department on implementation of QILT • QILT Working Group largely comprises former members of AQHE Reference Group • QILT Working Group will include representation from non-university higher education institutions (NUHEIs)

  19. General methodology • Independent administration • Random stratified sample • Stratification variables are specified for each instrument • HEIMS data to be made available for constructing sample frames • Additional information for the construction of sample frames, along with student and graduate contact details, to be collected from institutions

  20. Quality assurance • Biases and unrepresentativeness minimised • Quality assurance is based on achieving required confidence interval, on specific items, for a set proportion of strata • UES (entire educational experience) and GOS (further full-time study) • Interval of + or – 7.5 pp, at 90% confidence level • 75% of strata at the institution by subject area level, excluding strata with a population of fewer than six • ESS (overall supervisor rating) • Interval of + or – 7.5 pp, at 90% confidence level • 8 out of 10 broad fields of education • 25 responses for each field at each institution • Confidence intervals calculated as per 2013 UES

  21. NUHEIs • Currently around 85 Non University Higher Education Institutions (NUHEIs) accredited under HESA • Limited participation in AGS to date, not in scope for UES • NUHEIs in scope for QILT: • 2015 ESS • 2015 UES(trial of NUHEIs in 2014 UES) • 2016 GOS (from October 2015 round) • Survey results for NUHEIs to be included on QILT website subject to data being robust

  22. Tender • Tender opened 13June • Tender closed 11 July • Professor Shirley Alexander, UTS, to assist Department in assessing tenders • Expect to announce outcome by the end of July • Market testing through tender for 2014-16 triennium

  23. Confidence interval paper - anu http://unistats.anu.edu.au/surveys/toolkit/Confidence_Intervals_Sheet.pdf

More Related