1 / 34

Assessing Confidence in the Chinese Learner

Assessing Confidence in the Chinese Learner. Stephen Bruce Napier University, Edinburgh. Assessing Confidence in the Chinese Learner. Chinese students at Napier Rationale for confidence-based assessment Student responses to the test Academic self-concept, confidence and achievement research

hope-shelton
Download Presentation

Assessing Confidence in the Chinese Learner

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessing Confidence in the Chinese Learner Stephen Bruce Napier University, Edinburgh

  2. Assessing Confidence in the Chinese Learner • Chinese students at Napier • Rationale for confidence-based assessment • Student responses to the test • Academic self-concept, confidence and achievement research • Future work

  3. English Foundation Programme • Set up in 2001 to prepare Chinese students for entry to Napier University, both in terms of linguistic and study skills. • The needs of these students were complex, the “learning curve” for them was extremely steep, and that the demands they faced on a linguistically complex and culturally unfamiliar one-year Masters would be considerable, even given the requisite IELTS pass. • Proportion of Chinese students on Napier Business Masters programmes has increased in recent years, which has resulted in staff and the institution facing a very different cohort with different strengths, and also needs, to their predecessors. • Presents a major challenge to lecturers as they struggle to adapt their teaching approaches to suit both the increased number of overseas students and also the home students on their modules. Dr Lesley Gourlay, Napier University (2004). Crossing Boundaries: A Case Study Masters Level Chinese Students, LTSN (In Press).

  4. Linguistic and Cultural Issues Challenges for Students LANGUAGE • Extreme difficulties in understanding / taking notes in lectures • Difficulties with tutorial participation due to lack of confidence in English • Some difficulties with tutorial tasks due to slow reading speeds • Fear of failing exams due to difficulties in writing English in a time limit EDUCATIONAL CULTURE • Feeling “lost” in semester 1 • Some students unclear about expectations in UK-style coursework & reading • Experience extremely stressful for some Lesley Gourlay, Napier University (l.gourlay@napier.ac.uk)

  5. Linguistic and Cultural Issues Challenges for Staff LANGUAGE • Unsure if students are understanding lectures • Required to spend extra time explaining lecture content at the end • Some difficulties with tutorial participation and integration EDUCATIONAL CULTURE • Some staff faced with numerous examples of plagiarism in coursework • Some difficulties with tasks involving critical appraisal • Some students struggled with application of theory to practice Lesley Gourlay, Napier University (l.gourlay@napier.ac.uk)

  6. Potential responses by the university Lesley Gourlay, Napier University (l.gourlay@napier.ac.uk)

  7. Confidence and achievement • Learner self-esteem has been found to be positively associated with academic achievement (Brookover et al., 1964; Prendergast & Binder, 1975; Song & Hattie, 1984) • Comparative studies support a view that the self-esteem of young Chinese students is lower than UK and American (Chan, 2000). • self-effacing and modest values in Chinese culture, strongly influenced by the Confucian tradition of a ‘humble’ character • traditional authoritarian style of education or the highly competitive pressures created by schools, families and society • gap in living standards in the UK and Chinese cultures • Recent advances in academic self-concept and achievement research

  8. Assessing Confidence in the Chinese Learner This short pilot study is interested in two questions: • Would the purported reticence of the Chinese student be reflected in confidence-based assessment ? • Would their confidence levels in the test bear any relation to their academic self-concept ?

  9. Rationale for confidence-based MCQ • A student’s ability in answering may fall into a number of categories • I know it • I’m not quite sure, but I think I know it • Perhaps I can identify the answer by a deductive process on the distracters • If I guess I’ve a 25% chance of being correct (for 4 answer choices) • I really haven’t a clue and perhaps worst of all • I really know it …… what do you mean I’ve got it wrong !! Davies (2002) • The standard MCQ cannot distinguish between the above • the student certain of their knowledge should be rewarded • the student should not be rewarded for guesswork • Would the student use this knowledge to make a decision or perform an action (usable knowledge) ? Hassmen and Hunt (1994)

  10. Confidence-based MCQ • Select an answer in the usual way for a multiple choice question • Indicate your confidence that you are correct. C=1 (low) C=2 (mid) C=3 (high) Gardner-Medwin and Gahan (2003)

  11. Scoring confidence-based MCQ What is my score if my answer is or ? C = 3 (High) C = 2 (Mid) C = 1 (Low) Confidence level 1 3 2 -4 0 -1 More severe penalties for True/False format Gardner-Medwin and Gahan (2003)

  12. Encouraging good confidence judgement Good confidence judgement means more marks ! • If you are sure that your answer is correct …. • select C=2 (mid) or C=3 (high) confidence level • get the marks your confidence deserves! • If your are unsure (or guessing !) …. • select C=1 (low) confidence level • don’t lose marks through misplaced confidence!

  13. Confidence-based MCQ feedback A 100% confidence-based score is 40 answers at C=2 (mid) confidence …. so that 40 correct at C=3 would be 150% Gardner-Medwin and Gahan (2003) A summary of confidence levels….

  14. Confidence-based MCQ feedback Is the confidence-based score higher than the % correct score? No • Check the instances of INCORRECT answers at C=3 (high) and C=2 (mid). These can indicate areas of knowledge where you are misinformed. A large number of these perhaps indicates general overconfidence. • Check your number of CORRECT answers at C=1 (low). A large number indicates that you know more than you are willing to admit and are perhaps underconfident.

  15. Confidence-based MCQ feedback Is the confidence-based score higher than the % correct score? Yes You have shown good confidence judgement! You are willing to express an appropriate level of confidence when considering your knowledge. This is important when making decisions and performing actions based on that knowledge.

  16. The student group • 31 Chinese students (54% sample) • 18 male, 13 female • 24 postgraduate and 7 undergraduate • Ages: 18-25 (19), 26-32 (8), 33-40 (4) • Current on English as a Foreign Language (EFL) Foundation Programme • 35 weeks of 20 hours of EFL study • Gateway and Starter routes dependant upon English proficiency • Students recently completed their IELTS exam (International English Language Testing System). • Entry on to a Napier Programme conditional on a student’s IELTS score • At the time the students did not know their results • Students complete a confidence-based academic vocabulary test (40 Qs)

  17. Chinese student responses to confidence-based MCQ test • No indications that Chinese students are reluctant to select high confidence levels (C=3). • On average, C=3 (high) confidence was selected most often • No significant difference between genders observed.

  18. Chinese student responses to confidence-based MCQ test • Instances of C=3 (high) confidence increased as the question difficulty decreased. • ‘Difficult’ language comprehension test as some small grammatical differences separated answers and distracters. Harder Easier

  19. Correlating confidence-based test and final exam (IELTS) • only 7 students (of 31) improved on their % correct score with good confidence judgement. • 6 of these students were the highest test scorers in the class. • no apparent correlation between IELTS exam and confidence-based test • IELTS exam a true measure of achievement • several randomised confidence-based tests required for any correlation to identified • Students will become more proficient with practice

  20. Student opinions of the test • Strongly agree • Agree • Neutral • Disagree • Strongly disagree

  21. Student comments “It is a real good interesting test and I can receive different aspects of my knowledge. However, it will cost lots of time to finish this exam. As a result, I suggest we can do the exam at a regular time such an once a week.” “It is an interesting test and I would like to do it at my university. I think that is obviously fair for students.” “It is interesting and a little difficult to understand the result. I think I should get a higher confidence-based score because I chose (c3) nine times, and the number answered correct is 7. Maybe I have not understood this well.” “I think it is quite interesting and helpful.it is also a good way to show me the link between confidence and academic study.”

  22. Confidence and academic self-concept • This system asks learners to rate their confidence. Therefore learners are being assessed on their perceived confidence. • It may be of interest to examine the aspects of self-perception (self-concept) and how they relate to academic achievement. • As a staff developer encouraging uptake, it would be useful for me to give some indication of the factors that might be influencing a student’s decision to select a confidence level. • some non-academic self-concept ? • general academic self-concept ? • subject specific self-concept ? Does it matter in practice ? • perhaps not important for assessing a student’s usable knowledge • important for informed interventions for enhancing academic self-concept • new research making use of recent advances in theory Craven (1996) • scope for applying confidence-based feedback in primary/early secondary education • exploit the reciprocal effects of academic self-concept and achievement

  23. A model for self-concept • Posited to be multifaceted and hierarchical in nature • Shavelson, Hubner and Stanton (1976) • Numerous studies now based or extended from this model Hattie (1992)

  24. Self-concept and academic achievement • From the literature… • Support for the multifaceted nature…. • academic self-concept and academic achievement strongly correlated • Marsh, Byrne and Shavelson (1988), Hattie (1992) • little correlation between social self-concepts and academic achievementSong & Hattie (1984), Waugh (1999) • Support for the hierarchical nature less clear cut…. • more support for a hierarchical model for adolescents and • a unitary structure for younger children Hattie (1992) • achievement and academic self-concept deserve special interpretations at the level of specific subjects Marsh (1990) • standard self-description questionnaires based on the Shavelson model for preadolescents, adolescents and late adolescents Marsh (1992a, b, c)

  25. Measuring academic self-concept • Your perceptions of your ability and achievement • adapted from Marsh(1992), Song & Hattie (1984) and Waugh (2001) • indicate your relative agreement with each of the 20 statements: • All the time, or nearly all the time • Most of the time • Some of the time • None of the time, or almost none of the time • 1st 10 general university experience • eg. I am capable of getting good marks at university • I am proud of my achievements at university • 2nd 10 experience of English Language classes only • eg. I am sure of myself in English Language classes • I am achieving at a high level in English Language classes Questionnaire feedback For your general university experience, your responses total 18 points (30 maximum). For your English Language classes experience, your responses total 19 points (30 maximum).

  26. Measuring academic self-concept • Your attitudes to academic life • locus of control inventory adapted from Trice (1985) • select True or False to indicate your agreement with each statement: • eg. My academic marks most often reflect the effort I put into classes. • I came to university because it was expected of me Questionnaire feedback

  27. Correlating academic self-concept, confidence and achievement • perception of ability and achievement scores (general and EFL scales), increase in tandem. • as locus scores become increasingly internal (towards 0), perception of ability and achievement scores increase accordingly • General perception 19 ± 4.6 • EFL perception 19 ± 4.7 • Locus of control 12.2 ± 4.2

  28. Gender observations in the questionnaire responses Scores from questionnaire: Females tended to respond with lower scores than males for the perception of ability and achievement (general and EFL) scales. The responses indicating the most internal of locus of control scores, were male students.

  29. Correlating academic self-concept, confidence and achievement • Student’s perception of ability and achievement in EFL shows a moderate correlation with their IELTS exam result. • in line with academic self-concept and achievement research – strongly subject based. • No significant gender differences in the confidence-based test scores. • Females attained higher scores in the IELTS exam.

  30. Correlating academic self-concept, confidence and achievement • Perception of EFL ability & achievement scores are moderately correlated with the number of high confidence levels selected. • These figures may indicate a trend, but may be noise • one confidence-based test is insufficient to identify genuine correlations • there is little correlation between the confidence-based test and the IELTS results (true measure of achievement)

  31. Future work • Provide a series of confidence-based tests for EFL students • Deliver a system of numerous randomised tests (numerical and reading) • Provide a visual ‘confidence profile’ • Enhancing self-concept may positively affect other desirable academic behaviours such as persistence on academic tasks, academic striving behaviours, self-worth, self-efficacy, self-attributions in success and failure situations and academic achievement. Craven (1996) • Examine academic self-concept , achievement and confidence-based assessment • Focus on younger learners who may enjoy a game perspective • Harness the confidence-based feedback for academic self-concept enhancement interventions • The testing and feedback may dovetail with the reciprocal relationship between academic self-concept and positive academic behaviours.

  32. Bibliography Brookover, W.B., Thomas, S. and Paterson A. (1964) Self-concept of ability and school achievement, Sociology of Education, 37, pp. 271-279. Craven, R. (1996) Enhancing Academic Self-Concept: A Large-Scale Longitudinal Study in an Educational Setting, PhD thesis, University of Sydney Davies, P. (2002) There’s no confidence in multiple-choice testing, Proceedings of the 6th International CAA conference, Loughborough, pp. 119-130. Echternacht, G.J. (1972) The use of confidence testing in objective tests, Review of Educational Research, 42:2, pp. 217-237. Gardner-Medwin, A.R. and Gahan M. (2003) Formative and summative confidence-based assessment, Proceedings of the 7th International CAA conference, Loughborough, pp. 147-155. Gourlay, L. (2004) Crossing Boundaries: A Case Study Masters Level Chinese Students, LTSN (In Press) Hassmen, P. and Hunt, D.P. (1994) Human self-assessment in multiple-choice testing, Journal of Educational Measurement, 31, pp. 149-160.

  33. Bibliography Hattie, J. (1992) Self-Concept, Lawrence Erlbaum Associates, New Jersey Khan, K.S., Davies, D.A. and Gupta, J.K. (2001) Formative self assessment using multiple true-false questions on the Internet: feedback according to confidence about correct knowledge. Medical Teacher, 23, pp. 158-163. Marsh, H.W (1992a) Self-Description Questionnaire (SDQ)III: A theoretical and empirical basis for the measurement of multiple dimensions of preadolescent self-concept: A test manual and research monograph, MacArthur, New South Wales, Australia, University of Western Sydney, Faculty of Education. Marsh H.W., Byrne, B.M. and Shavelson, R.J. (1988) A multifaceted academic self-concept; Its hierarchical structure and its relation to academic achievement. Journal of Educational Psychology, 80, 366-380. Marsh H.W. (1990b) The structure of academic self-concept: The Marsh/Shavelson model, Journal of Educational Psychology, 82, 623-636. Prendergast, M.A. and Binder D.M. (1975) Relationships of selected self-concept and academic achievement measures, Measurement and Evaluation Guidance, 8, pp. 92-95.

  34. Bibliography Song, I.S. and Hattie, J. (1984) Home environment, self-concept and academic achievement measures, Journal of Educational Psychology, 76, pp. 1269-1281. Waugh, R.F. (2001) Measuring ideal and real self-concept on the same scale, based on a multifaceted, hierarchical model of self-concept, Educational and Psychological Measurement, 61:1, pp. 85-101.

More Related