Centre for Research in English Language Learning and Assessment

1. Centre for Research in English Language Learning and Assessment Однажды…

2. Centre for Research in English Language Learning and Assessment Однажды… ….. в 1974-oмгоду …

4. Centre for Research in English Language Learning and Assessment Assessing reading – how and why? An introduction to assessing reading for language teachers Dr Lynda Taylor

5. Designing tasks for testing L2 reading

6. We need to think about…. What types of mental (or cognitive) processing are typically involved in reading? What types of text are typically involved in reading activity? How can different test tasks (question formats, e.g. MCQ, short answer) help us to elicit the relevant cognitive processes using the relevant text-types in our reading tests?

7. Starting from the right place…. the nature of the cognitive processes in reading 2. the nature of the reading texts 3. the nature of the reading question formats

8. Starting from the WRONG place? the nature of the reading question formats the nature of the reading texts 3. the nature of the cognitive processes

9. Good reading task design What types of mental (or cognitive) processing are typically involved in reading? What types of text are typically involved in reading activity? How can different test tasks (question formats, e.g. MCQ, short answer) help us to elicit the relevant cognitive processes using the relevant text-types in our reading tests?

10. What actually happens when we read?A brief look at cognitive processes in reading

12. Visual input Clearly, the starting point for any reading activity is the set of marks on a handwritten or printed page, or on a computer screen – i.e. a combination of letters, symbols, pictures, etc.

13. Word recognition Word recognition involves matching the form of a word in a written text with a mental representation of the orthographic forms of the language. Field (2004:234) refers to this as “the perceptual process of identifying the letters and words in a text”.

14. Lexical (vocabulary) access Accessing the lexical entry containing stored information about a word’s form and its meaning from the lexicon. The form includes orthographic and phonological mental representations of an item and possibly information on its morphology. The lemma (the meaning-related part of the lexical entry) includes information on word class and the syntactic structures in which the item can appear and on the range of possible senses for the word.

15. Syntactic parsing Once the meaning of words is accessed, the reader has to group words into phrases, and into larger units at the clause and sentence level to understand the text message.

16. Establishing propositional meaning at the clause or sentence level An abstract representation of a single unit of meaning: a mental record of the core meaning of the sentence without any of the interpretative and associative factors which the reader might bring to bear upon it.

17. Inferencing (1) Inferencing is necessary so the reader can go beyond explicitly stated ideas as the links between ideas in a passage are often left implicit. Inferencing in this sense is a creative process whereby the brain adds information which is not stated in a text in order to impose coherence.

18. Inferencing (2) A text cannot include all the information that is necessary in order to make sense of it. Texts usually leave out knowledge that readers can be trusted to add for themselves. If there was no such thing as inferencing, writing a text which includes every piece of information would be extremely cumbersome and time consuming.

19. Constructing a mental model Once the reader has processed the incoming sentence and elaborated it where necessary and possible through inferencing, the new information needs to be integrated into a mental representation of the text so far. This process entails an ability to identify main ideas, to relate them to previous ideas, distinguish between major and minor propositions and to impose a hierarchical structure on the information in the text.

20. Creating a discourse-level structure At a final stage of processing, a discourse-level structure is created for the text as a whole. The skilled reader is able to recognise the hierarchical structure of the whole text and determines which items of information are central to the meaning of the text. The skilled reader determines how the different parts of the text fit together and which parts of the text are important to the writer or to reader purpose.

21. Establishing a mental representation across texts In the real world, the reader sometimes has to combine and collate macro-propositional information from more than one text. The need to combine rhetorical and contextual information across texts would seem to place the greatest demands on processing.

22. Cognitive processing at A2 to C2

23. Goal setter The goal setter in the left hand column is critical in that the decisions taken on the purpose for the reading activity will determine the relative importance of some of the processes in the central core of the model.

24. Types of reading Reading is either careful or expeditious and comprehension takes place at the local and global level.

25. Local comprehension Local comprehension refers to the understanding of propositions at the level of microstructure i.e., the sentence and the clause. Basic comprehension questions are used to assess lexical, syntactic, and semantic abilities and the ability to understand important information presented in sentence-level propositions. The information used in the question and the information required for the answer are usually in the same sentence.

26. Global comprehension Global comprehension refers to the understanding of propositions beyond the level of microstructure, that is, any macro-propositions including main ideas, the links between those macro-propositions and the way in which the micro-propositions elaborate upon them.

27. Careful reading Careful reading is intended to extract complete meanings from presented material. This can take place at a local or a global level, i.e. within or beyond the sentence right up to the level of the complete text or texts. The approach to reading is based on slow, careful, linear, incremental reading for comprehension.

28. Expeditious reading Expeditious reading involves quick, selective and efficient reading to access desired information in a text. Expeditious reading would appear likely to include skimming, search reading, and scanning.

29. Skimming Skimming is generally defined as reading to obtain the gist, general impression and/or superordinate main idea of the whole text.

30. Search reading Search reading involves locating information on predetermined topics. The reader only wants information necessary to answer set questions or to provide data for example in completing written assignments. It differs from skimming in that the search for information is guided by predetermined topics so the reader does not necessarily have to establish a macro-propositional structure for the whole of the text.

31. Scanning Scanning involves reading selectively, to achieve very specific reading goals, e.g. looking for specific words/phrases, figures/percentages, names, dates of particular events or specific items in an index at the local word level.

32. Types of reading tested at levels A2 to C2

33. Let’s look at a reading test task • Look at Task 1 which comes from a reading test. • Which of the core cognitive reading processes we have discussed does this task seem to elicit? • Which processes does this task apparently NOT elicit?

34. Let’s look at a reading test task

35. Let’s look at a reading test task

36. When we design a reading test task… We need to be explicit about: the types of reading that the task requires the various cognitive processes we believe the reading task is eliciting how well matched the cognitive processes are to the level of our students E.g. Is there a shift from tasks that focus on decoding to tasks that focus on meaning building from main ideas, to a text level representation to intertextualrepresentation

37. Cognitive validity in the testing of Reading The extent to which the tasks we employ in a reading test elicit the cognitive processing involved in target reading contexts beyond the test itself.

38. What is the nature of the texts we read?A brief look at what makes a text difficult to read

39. Contextual features of texts and tasks in reading tests This relates to the appropriateness of both : the linguistic and content demands of the text to be processed(i.e. read and comprehended) and the features of the task setting that impact on task completion (e.g. responding to comprehension questions or writing a summary)

40. Controlling contextual features of reading texts and tasks – 3 questions What are the key contextual features (within-text) that we need to think about when selecting texts for reading test tasks? What degree of complexity should we aim for in these features across each of the proficiency levels we want to test? What are the methods that can help us to understand the difficulty level of a reading text?

41. Contextual features in reading • Some illustrative features relating to: • lexical complexity (decoding) • structural complexity (syntactic parsing) • cohesion(the construction of meaning)

42. Some key lexical parameters L1 Syllables per word L2 Type token ratio L3 Word frequency L4 Lexical density L5 Proportion of academic words

43. L1 : Average syllables per word the mean number of syllables per content word multisyllabic words take longer to read and process than monosyllabic words [Rayner & Pollatsek, 1989] “In general, the more syllables per word and the more words per sentence, the higher the associated grade level of the text” [White, S. (2011) Understanding Adult Functional Literacy: Connecting Text Features, Task Demands, and Respondent Skills. Taylor & Francis]

44. L2 : Type token ratio the number of unique words divided by the number of tokens of the words Each unique word in a text is a word type. Each instance of a particular word is a token. When the type: token ratio is 1, each word occurs only once in the text; comprehension should be comparatively difficult because many unique words need to be encoded and integrated with the discourse context. A low type: token ratio indicates that words are repeated many times in the text, which should generally increase the ease and speed of text processing. [Templin, M (1957) Certain Language Skills in Children: Their development and interrelationships. Institute of Child Welfare Monograph Series No. 26. Minneapolis, MN: University of Minnesota Press]

45. L3 : Word frequency • the relative frequency of occurrence of words Frequency effects have been shown to facilitate decoding: • frequent words are processed more quickly and understood better than infrequent ones (Haberlandt & Graesser, 1985; Just & Carpenter, 1980). • rapid or automatic decoding = strong predictor of L2 reading performance (Koda, 2005) • texts which assist such decoding (e.g., by containing a greater proportion of high frequency words) are easier to process…. The more frequent a word, the more likely it is to be processed with a fair degree of automaticity, thus increasing reading speed (even among lower level learners) and freeing working memory for higher level meaning building. (Crossley, Greenfield and McNamara, 2008)

46. L4 : Lexical density • depends on distinguishing between different word types, i.e. lexical (content) and function words • lexical: verbs, nouns, adjectives, adverbs • function: auxiliaries, determiners, pronouns, prepositions, conjunctions Accessing the meaning of lexical items requires accessing the mental lexicon, function words can be dealt with by pattern matching. Reading focuses mainly on lexical items and readers tend to skip function words.

47. L5 : Proportion of academic words the incidence of academic words in a text proved to be a good predictor of level in a study of FCE, CAE and CPE reading texts (Weir et al, 2012) MeanSD FCE (B2) 1.61% 1.26% CAE (C1) 1.63% 1.41%CPE (C2) 5.82% 2.84%

48. Some key syntactic parameters S1 Sentence length S2 Readability formulae S3 Higher level constituents

49. Syntactic complexity linear processing of text in careful reading, with the reader decoding word by word assembly of decoded items into larger scale syntactic structure (Just & Carpenter, 1987; Rayner & Pollatsek, 1994) cognitive demands imposed vary considerably according to how complex the structure is (Perfetti, Landi & Oakhill, 2005) (Crossley ,Greenfield and McNamara, 2008)

50. S1 : Sentence length in Cambridge ESOL reading papers