1 / 39

INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc

INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc. Institutional Reporting from the Australian Graduate Survey. Leone Nurbasari Planning and Performance Measurement July 2014. Presentation Overview. Key Performance Indicators

vesna
Download Presentation

INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly, acu Hiediwilkinson, usc

  2. Institutional Reporting from the Australian Graduate Survey Leone NurbasariPlanning and Performance MeasurementJuly 2014

  3. Presentation Overview • Key Performance Indicators • Program (Course) Review reports • Discipline reporting on CEQ/PREQ • Graduate destination reporting • Comments analyses • Other reporting

  4. Key Performance Indicators • ANU Strategic Plan • College Operational Plans and KPIs

  5. KPI generation • Raw unit record data at 6 digit FOE, assigned to College scores based on taught disciplines (rolling 5 year average, minimum 10% of load taught by that College) • Benchmarking against Go8 data that is weighted according to ANU discipline weightings

  6. Example of some underlying 6 digit FOE data and calculations for KPI generation

  7. Program (Course) Review Reports • Program reviews • Data warehouse reports – CEQ, Employment & Further study • Supplemented by additional detailed analyses (including item level results) • Supplemented by anonymised open-ended comments

  8. Discipline reporting – CEQ/PREQ • Pseudo colleges (custom groupings of 2, 4 and 6 digit FOEs) • Pseudo departments (also custom groupings of FOE) • Benchmarking – Go8 and national, based on ANU taught FOE (excludes non ANU disciplines) • Percentage of individual student-item response in disagreement/neutral/agreement categories, aggregated to scales

  9. Graduate destination reporting • Data warehouse reports – employment rates, sector, industry, salary, employer, job search • Cohort-specific analysis • Ad-hoc data requests

  10. Comment analyses • CEQuery with charting via Excel • Various student cohorts and org. units • Adopting Geoff Scott’s method of proxies for importance and quality

  11. Other reporting • Topic Papers – comprising data from various surveys and student feedback sources, eg. HDR Satisfaction Report • Ad-hoc reporting from internal pivot tables – GDS, CEQ, PREQ

  12. Questions? • Email Leone.Nurbasari@anu.edu.au • Web http://unistats.anu.edu.au/surveys

  13. AGS Data and Reporting at ACUPamela Sarly | Acting Manager, Statistical Analysis & Surveys2014 Survey Manager Information Forum17 July 2014

  14. A bit about ACU… ONE OF THE FIRST EDUCATION PROVIDERS IN AUSTRALIA Merger of education colleges, forming ACU Teacher Training in NSW and VIC 1857 1963 2014 Good Samaritan Sisters 1900 1991 One of the fastest growing universities in Australia Teacher Training in NSW, VIC, QLD and ACT

  15. Australia's leading Catholic university which is supported by more than 2,000 years of Catholic intellectual tradition. 29,513 Enrolments 22,245 EFTSL 12 Schools across 4 Faculties 490 Higher Degree Research Students 6,263 Postgraduate Students 21,934 Undergraduate Students 826 Non-Award Students 2,839 international students • Staff FTE • 1,051 Academic • 982 Professional 12,952 commencing students 16,561 continuing students Data as at 11 July 2014

  16. AUSTRALIAN GRADUATE SURVEY Data Governance at ACU A central contact for surveys: Office of Planning and Strategic Management Publish aggregated data through internal sites: Staff Site (www.acu.edu.au/opsm) SharePoint site Data updates communicated to relevant staff via email

  17. AUSTRALIAN GRADUATE SURVEY Institutional Reporting Strategic Plan - Traffic Light Report Quarterly Report TEQSA Risk Indicators New Course Application Course Review and Renewal

  18. TRAFFIC LIGHT REPORT • Published in July and December • Progress against the University’s Strategic Goals and Key Result Areas

  19. STRATEGIC SCORECARD

  20. QUARTERLY REPORT Strategic Plan Goal 1: Student Experience • Published in January, April, July and October • High level quantitative data that is linked to Strategic Goal 1 (Student Experience)

  21. AUSTRALIAN GRADUATE SURVEY For the future… • Use of SPSS TAS for qualitative data analysis and reporting • Better linkage to Student Evaluation on Unit and Teaching surveys • Future of AGS – to inform our new Strategic Plan 2015+

  22. Thank youQuestions?

  23. Australian Graduate Survey Communicating results for effective decision making Note that the figures contained within this presentation are fictitious.

  24. Why do we need to communicate results? AGS results are used at USC to: Inform strategic decision making Assess program and institution performance Influence improvements to programs Review learning and teaching Develop internal policies

  25. Why use visualisations? “Numbers have an important story to tell. They rely on you to give them a clear and convincing voice.” Stephen Few

  26. Choosing the right medium Textual Visual Oral

  27. Tapping into human nature Over 50% of the cerebral cortex is involved with the processing of visual inputs. -Lu & Dosher There are over 130 million sight receiving cells in the retina. - Vries 2011 Source: Atranik 2011

  28. Tableau

  29. Knowing your audience USC’s Organisational Chart

  30. "The purpose of visualization is insight, not pictures" Shneiderman1999 Know your goal Enable insight

  31. GDS Dashboards Flowchart

  32. CEQ Dashboards Flowchart

  33. Levels of Analysis Graduate Destination Survey Benchmark Group Dashboard Course Experience Questionnaire Longitudinal Dashboard Graduate Destination Survey Faculty Dashboard Graduate Destination Survey Program (Course) Dashboard Course Experience Questionnaire Program (Course) Dashboard

  34. Director • - Quick reference point • Program leaders • - Program reviews • Executive • - Marketing and recruitment • - Identifying strengths and weaknesses • - Information sharing How dashboards are used Are they being used? Who is using them? What are they using them for?

  35. Reports by Field of Education • Beyond Graduation Survey • University Experience Survey • Internal Surveys • Interactive Dashboards • Qualitative Data Visualisations What next?

  36. Atranik.org 2011, http://antranik.org/functional-areas-of-the-cerebral-cortex. • Azzam, T & Evergreen, S 2013, Data Visualization, Part 2: New Directions for Evaluation, Number 140, Google eBook. • Card, S, Mackinlay, J & Shneiderman, B 1999, “Readings in Information Visualization - Using Vision to Think”, Morgan Kaufmann, Massachusetts. • De Vries, J 2011, The Five Senses, Random House, Victoria. • Few, S 2006, Information Dashboard Design: The effective visual communication of data, O’Reilly, California. • Lu, Z and Dosher, B 2013, Visual Psychophysics: From Laboratory to Theory, MIT Press, ISBN: 9780262019453. • University of Leicester (Learning Development) 2012, Presenting numerical data, <http://www2.le.ac.uk/offices/ld/resources> Bibliography

  37. Questions? If you have any questions after today: hwilkins@usc.edu.au

More Related