130 likes | 290 Views
Auditing Subject Knowledge in Initial Teacher Training using Online Methods. Auditing Subject Knowledge in Initial Teacher Training using Online Methods. Project team: Kerie Green Lynne Jones Barbara Kurzik David Longman Technical support: Dan Jones Allan Theophanides.
E N D
Auditing Subject Knowledge inInitial Teacher Training usingOnline Methods
Auditing Subject Knowledge in Initial Teacher Training using Online Methods Project team: Kerie Green Lynne Jones Barbara Kurzik David Longman Technical support: Dan Jones Allan Theophanides
Rationale • Subject audits are not new and have been used in our institution to determine existing knowledge for some considerable time. • The introduction of the standards for ITT as described in circulars 4/98 (England) and 13/98 (Wales) was the stimulus for this. • Subsequently, experimental electronic versions were created for core subjects. • These were designed to provide instant assessment and feedback for staff and students. • Recognition of the value and potential of electronic audits led to further developments in the areas of English and mathematics. • The objective of this project was the design and testing of online methods of assessment in these two subjects.
Questions: Answers: What? Who? Why? How? Analysis Limitations Observations Progression
What do we mean by ‘auditing subject knowledge’? • Is it diagnostic? • Is it assessment for learning? • Is it assessment of learning? • Is it all of these? • Through this project formative and summative assessment methods were explored and developed.
Who were to be audited? • First year trainees on the BA (Hons) Primary Studies Programme were the target for the English audit. • Second year trainees on the same programme were subjected to a similar model in the online examination. • First year trainees on the Secondary BSc programme were the target for the mathematics audit.
Why do students need to audit knowledge and skills? • An entry requirement for our Primary Studies Programme is that trainees need the equivalent of GCSE English at grade B. • Similarly, trainees are recruited to the Secondary BSc degree on the understanding that they are competent to a level equivalent to a GCSE Higher Tier. • However, trainees enter these courses with shortcomings that need to be identified and addressed.
How were the audits created? • The tool of convenience was QuestionMark Perception (QMP). • Audit questions were provided by subject specialist tutors and audits were extensively trialled with colleagues, trainees, teachers and Y12 students. • The English audit and examination used a variety of question formats including multiple choice, and free text entry. • The mathematics audit was presented as multiple choice format. • Audit items were drawn randomly from banks of equivalent questions. • With both audits trainees received instant and automatic feedback on their answers with links to supporting materials on each question. • English audit example • Maths audit exampleTypical audit feedback
Analysis • QMP allows for some data analysis but the most productive tool was found to be Excel. • Conditional formatting provided instant visual highlighting of areas for concern. • The usual numerical analysis was, of course, also useful so averages, standard deviations and graphs could be applied where needed. • English examination dataMathematics data
Limitations • QMP is designed for the non-specialist to create on-line surveys or tests. It includes a range of fixed template styles and operations. • Our requirements for the display of questions and feedback could not be easily accommodated by QMP.
Observations • The need for evaluation and checking of content accuracy and operational efficiency cannot be underestimated. • Question format is a challenge and it takes time and skill to design questions that are unambiguous and test objectively. • The method provides for consistency and accuracy of marking. • The value of the data analysis in terms of highlighting cohort weaknesses has proved compelling as it has in identifying individuals’ subject difficulties. • There is significant potential for the use of formative audits as part of professional development for trainees and to enhance partnership with schools.
Progression • Replace QMP with custom made system based on SQL and ASP.NET. Funding constraints have prevented progress so far but the intention remains. • Extend the audit process to cover literacy and numeracy for the same and other programmes (and schools?). This is a current project that is nearing completion. • Incorporate audit procedures in Personal Development Planning. Institutional discussions are taking place but the method has not yet been implemented. • Engage with external agencies such as DCELLS. Meetings to share ideas have taken place with WJEC.
Summary • This project was conducted during 2006-7. • The objective was to design and test online methods of assessment in the subject areas of English and mathematics. • In English an existing knowledge audit was further developed to incorporate feedback to individual students with topic specific support links. • Additionally a version of this audit model was adapted for use as an online examination. • In mathematics an audit was created as a formative exercise including links to support materials. • The data generated also informed tutors about areas for development across the cohort. • The findings have clear implications for curriculum design as well as professional development.