280 likes | 438 Views
Developing and Assessing the Research Skills of Students in Engineering Courses. Dr Said Al-Sarawi Research Skill Development and Assessment ALTC Project Member School of Electrical and Electronic Engineering University of Adelaide A seminar for Curtin Engineering Faculty. Outline.
E N D
Developing and Assessing the Research Skills of Students in Engineering Courses Dr Said Al-Sarawi Research Skill Development and Assessment ALTC Project Member School of Electrical and Electronic Engineering University of Adelaide A seminar for Curtin Engineering Faculty
Outline • Motivation • Students and staff issues • RSD Framework in Australia • Benefits of using RSDF • What is RSD Framework? • Case Studies – From 1st Year to Master Level • Other Dimensions of RSD Framework • RSD at the Program Level ! • RSD in conclusion • Discussion and future work
Motivation (1/2) • PhD completion rate were doubled for students who had participated in undergraduate research (Bauer & Bennett, 2003) • Performance based research funding (old RQF, ERA) • Performance based learning and teaching support – Learning and Teaching Performance Fund (DEST, 2006) • Undergraduate research has been associated with higher level of student satisfaction and generic skills development (Kardash, 2000) • Factors affecting skills implementation are (Lucas et al, 2000) • Scepticism of the message, the messenger and its vocabulary • The skills demanded lack clarity, consistency and recognisable theoretical base • The skills are dependent on discipline area
RSDF in Australia • The University of Adelaide (John Willison) • Human Biology (Eleanor Peirce & Mario Ricci) • Electrical Engineering Masters by Coursework (Said Al-Sarawi and Brian Ng) • Clinical Nursing (Frank Donnelly) • Petroleum Engineering (Steve Begg) • Introductory Academic Program (Richard Warner) • English (Joy McEntee) • Dentistry (Vicki Skinner and Leonard Crocombe) • Oral Health (Sophie Karanicolas and Cathy Snelling) • Software Engineering (Li Jiang) • Veterinary Science (Susan Hazel)
RSDF in Australia (Conts.) • Macquarie University (Psychology: Judi Homewood) • Monash University (Business Ethics: Jan Schapper; Sue Mayson: Business; Glen Croy: Tourism) • University of Melbourne (Business Law: Eu-Jin Teo) • University of South Australia (Introduction to Tertiary Learning, 2008: Rowena Harper)
The facets of student research In researching, students: • embark on an inquiry and so determine a need for knowledge/understanding • find/generate needed information using appropriate methodology • critically evaluate information/data and the process to find/generate • organise information collected/generated • synthesise and analyse and apply new knowledge • communicate knowledge and the processes used to generate it, with an awareness of ethical, social and cultural issues. (Willison & O’Regan, 2007)
Learning to Frame Research Questions Level 1 Facet A) Embark on Inquiry Respond to questions / tasks arising explicitly from a closed inquiry. Facet E) Synthesis, analysis, application Ask questions of clarification / curiosity.
Learning to Frame Research Questions Level 2 Facet A) Embark on Inquiry Respond to questions / tasks required by and implicit in a closed inquiry. Facet E) Synthesis, analysis, application Ask relevant, researchable questions.
Learning to Frame Research Questions Level 3 Facet A) Embark on Inquiry Respond to questions / tasks generated from a closed inquiry. Facet E) Synthesis, analysis, application Ask rigorous, researchable questions based on new understandings.
Why use RSD approaches? (1/2) • Benefits of using the RSD for students (according to Eleanor Peirce and Mario Ricci, Medical Sciences, Uni of Adelaide) • Their research skills in our course have improved. • They understand much more clearly what is expected of them. • They know exactly where they need to develop, thanks to feedback.
Benefits for Lecturers (2/2) (according to Eleanor Peirce and Mario Ricci, Medical Sciences, Uni of Adelaide) • We can give feedback on assessment tasks more accurately and efficiently; we can give the same quality feedback with less writing, and faster. • We can get a much better idea of where our students are from a quick analysis of the RSD results than from a detailed analysis of standard marks. • We can easily match assessment tasks with course objectives, and course objectives with the University’s Graduate Attributes.
Case Study 1: Human Biology - 1st Year • Read the two short then complete tasks 1 and 2. • Task 1: • Integrate the information presented in the two articles to write your own dot-point notes • on the worksheet attached. To do this: • Identify 3-4 key ideas from the articles • Use these key ideas to formulate headings and underline each • Make bullet-point notes and list them under these headings. • After each point, indicate its source, i.e. whether the idea came from article 1, article 2, or both • Provide a title that embodies the content of your notes. Task2: Which of the two articles do you consider to be the better source? On what characteristics/features of the article have you based your choice?
Case Study 1: Human Biology - 1st Year Literature Research Skill Stream Laboratory Research Skill Stream Field and literature research Semester 2 Semester 1
Novelty in the engineering case The differentiation points: • Clearly state the need to research skill development • The students are required to identify a gap in knowledge for each of the chosen topics (the unknown) • Rigorous literature research
Case study 2: Photonic and Communication – Master Course • Masters (coursework) course at EEE • Course run by senior colleague in EEE • 2 units of lectures/exams • 1 unit of literature research project • Students seek supervisors individually • Topic chosen by student, but requires approval by supervisor • Demographics • Almost all are international students; usually East Asian background • Technically capable, but generally lack prior experience in conducting research • Goal: produce a high quality review paper on chosen topic • Mostly negative experiences in 2005 • Cohort lacked basic literary research skills • Unstructured approach towards project • Low quality final review papers
Process Details • Initial diagnostic task • Supervisor supplies two technical papers Different levels (magazine, journal) for contrast • Students summarise and compare the key points from both sources in one structured, bulleted list • Detailed supervisor feedback in written form • Knowledge accumulation phase • More articles added to the reading list Continually add to an organic structured, bulleted list • Strategy on further reading Student applies critical evaluation on suitability of sources with supervisor input • Fortnightly workshops for group presentations & discussions Supervisor supplies feedback; optional: external advice (CLPD) • Writing phase • Student writes review paper based on list • Supervisor feedback on first draft approx a week before submission
Outcome and Evaluation • Outcomes from 2006 • Total of 6 students plus1 external (industry) student as control • Qualitatively, much improved papers compared to 2005 • Coherent structures, logical arguments, conciseness, respect of referencing practice • Student perspective • struggled to cope with demands of research among the pressures of regular coursework • English as second language remained a great barrier • Framework “matrix” useful for quantitative assessment • Unexpected benefit – reduction in plagiarism • Turn-it-in software reported major improvements from 2005 • Reports tend to be on conservative side.
Case 3: Final Year Project (in Progress) “We have been concerned about a degree of disconnect between the desired leaning outcomes and the assessment methods used in these projects.” HoS of School of EEE, The University of Adelaide.
Variety of RSD approaches • Five distinct approaches have emerged in the use of the RSD so far: • Rubrics base to assess the profile of skills for each student, as demonstrated by Human Biology rubrics. this is the most common approach, and is used by most disciplines. • A lock-step approach, whereby students are kept 'in formation' and progressively and corporately develop one level of skills at a time. This is shown by the Nursing RSD rubrics. • Grading with a specific grading, by incorporation into SOLO taxonomy to define grading within a specific level set, as used by Dentistry. • Skill Evaluation, to evaluate the skills and levels required by existing assessments. • Identification of research skills of higher degree students, Masters by Research and PhD students and/or their supervisors/advisors to locate their present skill set and plot future directions and development needs. (SOLO: Structured Observed Leaning Outcomes)
Other Dimensions of RSD • Degree of Autonomy • Degree of Academic Rigour • Degree of Conceptual Demand • Degree of disciplinary knowledge required • Status of knowledge being pursued
RSD Framework at Program Level ! No studies that consider student outcomes of the explicit development and assessment of research skills over a whole undergraduate or masters-by-coursework program • Undergraduate Level • Bachelor Oral Health, Adelaide Uni, AU • Bachelor of Media Studies , Adelaide Uni, AU • Bachelor of Science, Adelaide Uni, AU • Bachelor of Nursing, Adelaide Uni, AU • Bachelor of Business, Monash Uni, AU • Bachelor of Science (Psychology), Macquarie University, AU • Postgraduate Level • Master of Engineering (Advanced) (Electrical), Adelaide Uni, AU • Bridging Program for International PhD students, Adelaide Uni, AU • Masters and PhD (Nursing), PhD (Nursing) • Trinity College Dublin, Ireland • Master of Business, Monash University, AU
RSD in conclusion • Provides the Big Picture and relates this to the assessment details for course coordinators, lecturers, tutors, and especially students • Informs assessment-first curriculum redesign • Same ‘facets’ for multiple assessments, various levels • Explicit &Transparent assessment criteria • Coherent & Incremental skill development • Revisited & (potentially) Cyclic Conceptual Framework
Discussion and Future work • How to integrate the framework into other Programs? • How will the implementation of RSD framework affect academic workload? • How to monitor and assess student’s progress? • How can this be implemented for larger class sizes? • How can the framework adopted for non-literary research skills?
Acknowledgement References This session was funded by an Australian Learning and Teaching Council Grant • Bennett, N., Dune, E. & Carré, C. (2000). Skills Development in Higher Education and Employment. (Society for Research into Higher Education & Open University Press). • Stevens, C. & Fallows, S.J. (2000). Integrating Key Skills in Higher Education: employability, transferable skills and learning for life. Routledge, ISBN 0749432659. • Lucas, U., Cox, P., Croudace, C. and Milford, P. (2004). “Who Writes This Stuff?”: Students’ Perceptions of Their Skills Development. Teaching in Higher Education, 9(1), 55-68. • Willison, J.W. & O’Regan, K. (2007). Commonly known, commonly not known, totally unknown: A framework for students becoming researchers. Higher Education Research and Development, 26(3). RSD Web Site http://www.adelaide.edu.au/clpd/rsd