The influence of question types on the comprehension of expository texts

1. The influence of question types on the comprehension of expository texts Fernando Marmolejo-Ramos Julia Miller Chad Habel University of Adelaide

2. Outline • Introduction • Literature review • Text comprehension • Expository texts • Implicit/Explicit questions • Methodology • Findings • Discussion • Conclusions

3. Introduction • Text comprehension is essential for academic achievement. • 3 kinds of academic text: • Argumentative – makes an argument • Narrative – tells a story • Expository – informs or explains Does question type influence the comprehension of expository texts? • Explicit questions • Implicit questions

4. Assessment of reading comprehension gives educators an indication of students’ understanding of texts (Ozuru et al., 2007). • The results of our findings could have implications for university teachers who set questions on texts.

5. Literature review • Questions can provide ‘contextual support’ to help readers understand the text (Ozuru et al., 2007, p. 405). • Texts activate students’ prior knowledge of a subject (Ozuru et al., 2007, p. 424).

6. Text comprehension • Readers of short texts often do not refer to the text when answering questions on it (Pressley et al.,1990, in Ozuru et al., 2007). • Text availability affects comprehension. • Non-availability measures cognitive processing speed and retrieval, also drawing on prior knowledge • Availability measures processing based on resources. (Ozuru et al. 2007)

7. Expository texts • Expository texts are harder to understand than narrative texts (Gersten et al., 2001; Wolfe & Woodwyk, 2010). • No oral prompts • More abstract • More complicated structure – this influences comprehension • Reader may lack prior knowledge in area • Expository texts tend to activate prior knowledge (Wolfe & Woodwyk, 2010). • It is hard to measure the comprehension of expository texts (Ozuru et al., 2007).

8. Explicit/Implicit questions • ‘readers do not construct novel inferences when reading unfamiliar as opposed to familiar texts (Noordman, Vonk, & Kempff, 1992) unless the task situation demands readers to do so (Singer, Harkness, & Stewart, 1997)’ (Ozuru et al., 2007, p. 405). • 2 types of questions in our study: • Text-based (explicit) • Inferential (implicit)

9. Research gap • Few studies have addressed the issue of the question type and passage availability. • Ozuru et al. (2007) studied question administration (with-text, without-text) but examined multiple choice vs open questions. • Wolfe & Woodwyk (2010) compared narrative and expository texts. • This study examines both passage availability and explicit/implicit question types, and their effects on text comprehension. Participants were not told in advance that they would have to answer questions. • No measure of reader’s confidence with answers has been investigated (here we measure confidence with answers), let alone the relationship between confidence and answer correctness.

10. Research goals • Study the effect that reading instruction has on comprehension in relation to the type of questions. • Investigate how participants’ confidence with the answers is self-assessed in relation to the reading situation. • Determine the way answer’s correctness relates to the readers’ confidence with the answer.

11. Methodology • Participants • 50 students at Adelaide University (18 males, M age= 24.82, SD = 8.68) • 56% English speakers • 44% non-native speakers (half were Chinese) • Different fields (Psychology, Commerce, Arts, Engineering, etc) Materials • One expository text entitled “wine production” (translated from a text originally written in Italian and used in a recent study) • The translation resulted in a 267-word text • 3 explicit + 3 implicit questions

12. Methodology (2) Procedure • 4 versions of text booklet • 2 with continuous text • 2 with bullet text • For each type of text, one version allowed participants to refer back. The other did not allow them to refer back. • Before they read the text, participants were not informed that they would have to answer questions on it.

13. Procedure • The text, the questions, and rating questions about participants’ confidence with their answers and helpfulness of the questions were given in a fixed order and in the form of a booklet (each printed in one page). • After reading the text participants were asked to rate their knowledge of the topic BEFORE reading and AFTER reading. • Independent judges rated the correctness of participants’ answers to the questions about the text.

14. Findings Answer correctness Confidence ratings ** ** * * p < .05 ** p < .01

15. Findings (2) Type of relationship between correctness and confidence - correlation τ = -.123, p = .457 τ = .395, p = .043 τ = .121, p = .493 τ = -.195, p = .235 Kendall tau (τ) correlations showed that only when readers were allowed to go back to the text did the correctness of the answer and reader’s confidence with the answer correlate positively. In no other case this did this situation happen. + correlation The closer the value to 0, the larger the consistency between correctness and confidence (i.e., high correctness – high confidence and low correctness – low confidence), i.e., signals a positive correlation The farther from 0, the lesser the consistency between correctness and confidence (i.e., low correctness – high confidence and high correctness – low confidence), i.e., signals a negative correlation

16. Discussion • There is a positive correlation between correctness and confidence only when explicit questions are asked, especially when readers can refer back to the text. • Text comprehension can be enhanced by choice of question types and encouragement to refer back to the text. • Consider putting explicit questions first. • Take account of this in exam conditions. • In this study, bullet vs continuous text structure has no effect on answer correctness. • Further research in this area is needed.

17. Limitations • Original text was in Italian and based on the situation in Italy, where sparkling wine is usually white. Australia, however, has sparkling red wine too. This may have caused difficulty in answering some questions. • We trusted participants not to look back at the text if instructed not to, but couldn’t control this. • Sample size. • More nuanced questions to enable use of a full scale of correctness. • Control for disciplinary background.

18. Conclusion • Implications for university teachers in setting readings, exams and discussion questions. • Question setters should be aware of the types of questions they set in relation to the types of comprehension they want to test.

19. References Gersten, R., Fuchs, L.S., Williams, J.P., & Baker, S. (2001). Teaching reading comprehension strategies to students with learning disabilities: A review of research. Review of Educational Research, 71(2), 279-320. Gillam, S L., Fargo, J.D., & Robertson, K.St.C. (2009). Comprehension of expository text: Insights gained from think-aloud data. American Journal of Speech-Language Pathology, 18, 82-94. Ozuru, Y., Best, R., Bell, C., Witherspoon, A., & McNamara, D. (2007). Influence of question format and text availability on the assessment of expository text comprehension. Cognition and Instruction, 25 (4), 399-438. Pazzaglia, F., Meneghetti, C., de Beni, R., & Gyselinck, V. (2010). Working memory components in survey and route spatial text processing. Cognitive Processing, 11, 359-369. Wolfe, M.B.W., & Woodwyk, J.M. (2010). Processing and memory of information presented in narrative or expository texts. British Journal of Educational Psychology, 80, 341-362.

20. Wine production The grape harvest is held at different times of the year according to the type of wine: grapes for the production of sparkling wine are harvested during the first week of September, while grapes for table wine are harvested at the end of September. Sparkling wines have a high degree of acidity and an alcohol level of 9 - 12%. Non sparkling wines have low levels of acidity and an alcohol level of 12 - 15%. Generally, sparkling wines appeal more to women and table wines appeal more to men. The grapes are picked and put into 18 kilogram baskets, and crushed on the day of harvest to avoid early fermentation. The must obtained from crushing is placed in open vats in order to eliminate carbon dioxide. There are two types of vinification, or wine making, depending on whether white or red grapes are used. To produce red wine, the must is left to ferment with the grape skins at a temperature of 15°-18°C for 15 days. After this process, the skins of the red grapes are separated and crushed to produce second grade wine. Crystallisation then occurs. i.e. the wine is kept at sub-zero temperatures for two days, allowing the excess tartar to sink to the bottom. The wine is then bottled immediately and stored in cool, dry cellars. To produce white wine, the skins are immediately discarded and the grapes are crushed at once. As with red wine, the must is left to ferment at a temperature of 15°-18°C for 15 days. The process of crystallisation and bottling is the same as for red wine.

21. …and the questions Explicit questions • Which are harvested first: grapes for sparkling wine or grapes for table wine? • Do men or women usually prefer table wine? • How quickly are grapes crushed in order to produce white wine? Implicit questions • What is “must”? • What happens if grapes are not crushed on the day of harvest? • Why are the grape skins discarded immediately when making white wine?

22. Effect of language Answers’ correctness Knowledge on topic * * * p < .05 ** p < .01