1 / 12

Dr Susan Shannon Spencer Gulf Rural Health School

Evaluating and reporting at a distance: quality experiences with cost effective web supported evaluations. Dr Susan Shannon Spencer Gulf Rural Health School The University of Adelaide and The University of South Australia susan.shannon@adelaide.edu.au. Evaluation Clients. CASE STUDY 1

ludwig
Download Presentation

Dr Susan Shannon Spencer Gulf Rural Health School

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluating and reporting at a distance: quality experiences with cost effective web supported evaluations Dr Susan Shannon Spencer Gulf Rural Health School The University of Adelaide and The University of South Australia susan.shannon@adelaide.edu.au

  2. Evaluation Clients CASE STUDY 1 N U P H I S S National Undergraduate Public Health Internship and Scholarship Program, December 2001 to February 2002 4 Universities, placements in WA, Qld, SA and ACT 6 week placements for end-3rd year students in private or public clinical setting. Students undertook research projects, under supervision.

  3. Evaluation Clients CASE STUDY 2 S G R H S Spencer Gulf Rural Health School All 1st year medical students spend a Rural Week in one of two country districts in April, July or Sept All 2nd year students undertake clinical shadowing for a week in a rural health setting 2003 Ten 5th years spent their year in the Spencer Gulf Learning in 5 locations

  4. External Requirement for evaluation Case Study 1 NUPHISS Ongoing funding contingent on successful pilot program. External evaluation mandated. Case Study 2 SGRHS Parameter of Commonwealth Contract for Rural Clinical School …‘transparent internal evaluation”

  5. The Web to the rescue! CASE STUDY 1NUPHISS VGallery web-based Survey tool Web based Discussion Board CASE STUDY 2SGRHS Email Evaluation Questions Optical Mark Recognition Software Surveys

  6. V Gallery online survey tool

  7. vGallery Advantages Professional appearance; Speed of feedback from respondents and Reporting to Excel automatically conducted; No data keying by evaluation personnel; Evaluation time can be spent on evaluation, ie understanding what data means rather than gathering data; Good response rates (students 89%; academics 100%; work placement supervisors 68%) Disadvantages universal web access amongst respondents is required keyboard familiarity is assumed

  8. Web based Discussion Board Advantages • Interactive • Responsive • Group focused • Threaded by evaluator to add further qualitative data Disadvantages • Asynchronous • Universal Web access required • Key board familiarity is assumed • Can be talking to an empty “room” • Whole question of the evaluator as correspondent with the evaluatee comes more vexed

  9. Smart Board and Video Conferencing

  10. Optical Mark Recognition Instruments purpose made evaluations

  11. Tool or Technique Advantages Disadvantages VGallery web-based Survey tool Self-prepared Professional appearance Speed of feedback from respondents Reports automatically to Excel No data keying by evaluation personnel Evaluation time can be spent productively Good response rates No Paper – nothing to lose Supports any type of question Universal Web access required Key board familiarity is assumed Survey form must be carefully prepared to avoid glitches All respondents must be enrolled as vGallery users on University of Adelaide PeopleSoft database to access Survey (provided with a username and password) Web based Discussion Board Interactive Responsive Group focused Asynchronous Universal Web access required Key board familiarity is assumed Can be talking to an empty “room” Email Evaluation Questions Ubiquity of email as relevant and informal communication protocol Cheap Requires no particular platform – hotmail and private isp providers utilised by students as preferred email addresses as well as official University student email No paper – nothing to lose Email access problems at some sites Attachments would not upload at some remote learning centres (bandwidth issues) Response rates diminished during the year No automatic accumulation of responses to database Optical Mark Recognition Software Surveys Response rtes remained high OMR software collates, analyses and reports digitally Speed (potentially) and accuracy ensured in reporting Reports emailed No loss of data richness University- supported system Paper based – document storage issues with originals Current Software set-up only accommodates some types of questions Need to involve LTDU in process of preparing Surveys, analysing etc.

More Related