1 / 21

Colour coding, KPIs and the Road to Damascus

Colour coding, KPIs and the Road to Damascus. Shedding new light on reporting the student experience at the University of Sydney Rachel Y. Symons Quality Assurance Officer (Learning and Teaching). “We can learn from experience if we are ready to adapt that experience to changed conditions”

yetta
Download Presentation

Colour coding, KPIs and the Road to Damascus

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Colour coding, KPIs and the Road to Damascus Shedding new light on reporting the student experience at the University of Sydney Rachel Y. Symons Quality Assurance Officer (Learning and Teaching)

  2. “We can learn from experience if we are ready to adapt that experience to changed conditions” JC Masterton (1972): The Double Cross System in the war of 1939-1945

  3. Scene setting • Reports Marks 1 & 2 • Colour coding • Taxonomy upgrade • NVivo7 • Reports Mark 3 • What next?

  4. Reports Mark One (2003 – 2006) • Areas of best practice • Suggested improvements • Ordered by percentage of comments received • Not linked to SCEQ scales • Redundant with introduction of KPIs

  5. Reports Mark Two (2007) • Aligned with KPIs • Generic Skills • Good Teaching • Overall Satisfaction • List of key issues • Minimal sample comments • Results as raw numbers • Undergraduate and postgraduate coursework combined on one page

  6. Feedback and A new broom • Dense • Not user friendly • Difficult to interpret • Percentages not raw numbers • Confusing to novice • Need for succession planning

  7. Colour coding and KPIs • Easy identification of related data across all documentation • Taxonomy • Recording analysis • Communicating information • Linked to KPIs / SCEQ scales

  8. Taxonomy enrichment • Describes the concepts • Coverage of each component • Answers the question • Illustrates with student comments

  9. The Road to Damascus • NVivo7 + Boolean logic = technological conversion • Combines manual and automated analysis • Use of pre-set base queries • Faculty customisation

  10. Benefits of NVivo7 methodology • No need to print source documents • Simultaneous coding • Time saving • Direct transfer of comments into reports • Reduces impact on environment

  11. Revitalising the reports • Consultation with stakeholders and communications experts • Developed quality product • Beneficial to all stakeholders • More actionable than previous reports

  12. Reports Mark Three (2008) • KPI alignment • SCEQ and CEQ data • Executive summary • Trends • Comparative quantitative and qualitative data (2001 – 2007) • Key issues (2007) • Customisation by faculty • Separate reports undergraduate and postgraduate coursework

  13. Reports Mark Three (2008) Key results • Faculty scores for Good Teaching (KPI 15; Section 2) • have increased by 8% since 2005 for current students (SCEQ) • have been consistently above the University average for current students (SCEQ) since 2000 • are lower than previous years for graduates (CEQ) • are lower than the University average for graduates (CEQ) • domestic and international students are recording similar experiences • Analysis of the qualitative data from current students (SCEQ) indicates that experiences of teaching within the Faculty are evenly divided between students who consider this to be one of the best areas of their experience and those who suggested improvements. For graduates (CEQ), a higher percentage of comments received from respondents focused on areas of improvement rather than areas of best practice.

  14. Reports Mark Three (2008)

  15. Reports Mark Three (2008)

  16. Reports Mark Three (2008)

  17. Have we succeeded? • Improved arrangement • Easy identification of areas • colour coding • collation by KPI • Used to inform • curriculum review • marketing • Aids in prioritising areas for improvement • Incorporation of qualitative and quantitative data plus executive summary particularly helpful

  18. What next? • Investigate strategies for “closing the loop” • Report on faculty/ student administration • Cross university reports • eLearning • Research-enhanced learning and teaching • Library • International students • Alignment with AUSSE Scales and survey items

  19. What next? • Commitment to quality improvement • Reports continue to be actionable and accessible • Taxonomy continuous w-i-p • Flexibility of NVivo7 will accommodate range of analytical reports

  20. “We can learn from experience if we are ready to adapt that experience to changed conditions”

  21. Useful information • Taxonomy; methodology; reportshttp://www.usyd.edu.au/learning/evaluating/open_ended_responses.shtml • Contact information r.symons@usyd.edu.au 02 9351 6560

More Related