What Is the “Context” for Contextual Vocabulary Acquisition?

1. What Is the “Context” forContextual Vocabulary Acquisition? William J. Rapaport Department of Computer Science & Engineering Department of Philosophy Center for Cognitive Science NSF ROLE Grant REC-0106338

2. Outline • People can figure out a meaning for a word “from context” • What does “context” mean in this context?

3. Definition of “CVA” “Contextual Vocabulary Acquisition” =def • the acquisition of word meanings from text • “incidental” • “deliberate” • by reasoning about • contextual cues • background knowledge • Including prior word-meaning hypotheses, language knowledge… • without external sources of help • no dictionaries • no people

4. CVA: From Algorithm to Curriculum • Computational theory of CVA • Based on: • algorithms developed by Karen Ehrlich (1995) • verbal protocols (case studies) • Implemented in a semantic-network-based knowledge-representation & reasoning system • SNePS (Stuart C. Shapiro & colleagues) • Educational curriculum to teach CVA • Based on our algorithms & protocols • To improve vocabulary & reading comprehension • Joint work with Michael Kibby • Center for Literacy & Reading Instruction

5. People Do “Incidental” CVA • We know more words than explicitly taught • Average high-school grad knows ~45K words  learned ~2.5K words/year (over 18 yrs.) • But only taught ~400/school-year • ~ 4800 in 12 years of school (~ 10% of total)  Most word meanings learned from context • “incidentally” (unconsciously) • How?

6. People Also Do “Deliberate” CVA • You’re reading; • You understand everything you read, until… • You come across a new word • Not in dictionary • No one to ask • So, you try to “figure out” its meaning from “context” • How? • guess? derive? infer? deduce? educe? construct? predict? … • our answer: Compute it! Via inferential search of “context”/KB • But what KB?

7. CVA as Cognitive Science • Studied in: • AI / computational linguistics • Psychology • Child-language development (L1 acquisition) • L2 acquisition (e.g., ESL) • Reading education (vocabulary development) • Thus far: “multi-”disciplinary • Not yet: “inter-”disciplinary!

8. What does ‘brachet’ mean?

9. (From Malory’s 15th century Morte d’Arthur[page # in brackets]) • There came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them. [66] • People: brachet = animal? inanimate object? don’t know. • Computer: brachet = physical object • (because only physical objects have color) • As the hart went by the sideboard, the white brachet bit him. [66] • People: brachet = animal • Computer: brachet = animal • (because only animals bite)

10. Malory, continued 3. The knight arose, took up the brachet and rode away with the brachet.[66] • People: brachet = animal / small animal • Computer: brachet = small animal • (because: picked up and carried) 4. A lady came in and cried aloud to King Arthur, “Sire, the brachet is mine”. [66] • People: brachet = pet / small, valuable animal • Computer: brachet = small, valuable animal • (because: what’s wanted is valuable)

11. Malory, continued • There was the white brachet which bayed at him fast. [72] • People: brachet = dog • Computer: brachet = hound (i.e., dog that hunts) • (because only hounds, which are hunting dogs, bay) • The hart lay dead; a brachet was biting on his throat, and other hounds came behind. [86] • People: brachet = hound • Computer: brachet = hound (i.e., dog that hunts) • (because “x and other y”  x is a y)

12. How (Not) to Teach CVA:Vague Strategies • Clarke & Nation 1980: a “strategy” (algorithm) • Look at word & context; determine POS • Look at grammatical context • E.g., “who does what to whom”? • Look at wider context • [E.g., for clues re: causal, temporal, class-membership, etc.] • Guess the word; check your guess

13. Vague strategies: • “guess the word” = “then a miracle occurs” • Surely, we computer scientists can “be more explicit”!

14. A More Precise, Teachable Algorithm • Treat “guess” as a procedure call • Fill in the details with our algorithm • Convert the algorithm into a curriculum • To enhance students’ abilities to use deliberate CVA strategies

15. Figure out meaning of word from what? • context (i.e., the text)? • Werner & Kaplan 52, McKeown 85, Schatz & Baldwin 86 • context and reader’s background knowledge? • Granger 77, Sternberg 83, Hastings 94 • context including background knowledge? • Nation & Coady 88, Graesser & Bower 90 • Note: • “context” = text  context is external to reader’s mind • Could also be spoken/visual/situative (still external) • “background knowledge”: internal to reader’s mind • What is (or should be) the “context” for CVA?

16. Some Proposed Preliminary Definitions(to extract order out of confusion) • Unknown word for a reader =def • Word or phrase that reader has never seen before • Or only has vague idea of its meaning • Different levels of knowing meaning of word • Notation: “X”

17. Proposed preliminary definitions • Text =def • (written) passage • containing X • single phrase or sentence … several paragraphs

18. Proposed preliminary definitions • Co-text of X in some text =def • The entire text “minus” X; i.e., entire text surrounding X • E.g., if X = ‘brachet’, and text = • “There came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them.” Then X’s co-text in this text = • “There came a white hart running into the hall with a white ______ next to him, and thirty couples of black hounds came running after them.” • Cf. “cloze” tests in psychology • But, in CVA, reader seeks meaning or definition • NOT a missing word or synonym: There’s no “correct” answer! • “Co-text” is what many mean by “context” • BUT: they shouldn’t!

19. Proposed preliminary definitions • The reader’s prior knowledge =def • the knowledge that the reader has when s/he begins to read the text • and is able to recall as needed while reading • “knight picks up & carries brachet” ? small • Warnings: • “knowledge”  truth • so, “prior beliefs” is better • “prior” vs. “background” vs. “world”, etc. • See next slide!

20. Proposed preliminary definitions • Possible synonyms for “prior knowledge”, each with different connotation: • Background knowledge: • Can use for information that author assumes reader to have • World knowledge: • General factual knowledge about things other than the text’s topic • Domain knowledge: • Specialized, subject-specific knowledge about the text’s topic • Commonsense knowledge: • Knowledge “everyone” has • E.g., CYC, “cultural literacy” (Hirsch) • These overlap: • PK should include some CSK, might include some DK • BK might include much DK

21. Steps towards aProper Definition of “Context” • Step 1: • The context of X for a reader =def • The co-text of X • “+” the reader’s prior knowledge • Both are needed! • After reading: • “the white brachet bit the hart in the buttock” most subjects infer that brachets are (probably) animals, from: • That text, plus: • Available PK premise: “If x bites y, then x is (probably) an animal. • Inference is not an enthymeme! (because …)

22. Proper definition of “context”: • But (inference not an enthymeme because): • When you read, you “internalize” the text • You “bring it into” your mind • Gärdenfors 1997, 1999; Jackendoff 2002 • This “internalized text” is more important than the actual words on paper: • Text: “I’m going to put the cat out” • Misread as: “I’m going to put the car out” • leads to different understanding of “the text” • What matters is what the reader thinks the text is, • Not what the text actually is • Therefore …

23. Proper definition of “context”: • Step 2: • The context of X for a reader =def • A single KB, consisting of: • The reader’s internalized co-text of X • “+” the reader’s prior knowledge

24. Proper definition of “context”: • But: What is “+”? • Not: mere conjunction or union! • Active readers make inferences while reading. • From text = “a white brachet” & prior commonsense knowledge = “only physical objects have color”, reader might infer that brachets are physical objects • From “The knight took up the brachet and rode away with the brachet.” & prior commonsense knowledge about size, reader might infer that brachet is small enough to be carried • Whole > Σ parts: • inference from [internalized text + PK]  new info not in text or in PK • I.e., you can learn from reading!

25. Proper definition of “context”: • But: Whole <Σ parts! • Reader can learn that some prior beliefs were mistaken • Or: reader can decide that text is mistaken (less likely) • Reading & CVA need belief revision! • operation “+”: • input: PK & internalized co-text • output: “belief-revised integration” of input, via: • Expansion: • addition of new beliefs from ICT into PK, plus new inferences • Revision: • retraction of inconsistent prior beliefs together with inferences from them • Consolidation: • eliminate further inconsistencies

26. Prior Knowledge Text PK1 PK2 PK3 PK4

27. Prior Knowledge Text T1 PK1 PK2 PK3 PK4

28. Integrated KB Text T1 internalization PK1 PK2 PK3 PK4 I(T1)

29. B-R Integrated KB Text T1 internalization PK1 PK2 PK3 PK4 I(T1) inference P5

30. B-R Integrated KB Text T1 internalization PK1 PK2 PK3 PK4 I(T1) T2 inference P5 I(T2) P6

31. B-R Integrated KB Text T1 internalization PK1 PK2 PK3 PK4 I(T1) T2 inference T3 P5 I(T2) P6 I(T3)

32. B-R Integrated KB Text T1 internalization PK1 PK2 PK3 PK4 I(T1) T2 inference T3 P5 I(T2) P6 I(T3)

33. Note: All “contextual” reasoning is done in this “context”: B-R Integrated KB Text T1 internalization PK1 PK2 PK3 PK4 P7 I(T1) T2 inference T3 P5 I(T2) P6 I(T3)

34. Proper definition of “context”: • One more detail: X needs to be internalized • Context is a 3-place relation among: • Reader, word, and text • Final(?) def.: • Let T be a text • Let R be a reader of T • Let X be a word in T (that is unknown to R) • Let T-X be X’s co-text in T. • Then: • The context that R should use to hypothesize a meaning for R’s internalization of X as it occurs in T =def • The belief-revised integration of R’s prior knowledge with R’s internalization of T-X.

35. This definition agrees with… • Cognitive-science & reading-theoretic views of text understanding • Schank 1982, Rumelhart 1985, etc. • & KRR techniques for text understanding: • Reader’s mind modeled by KB of prior knowledge • Expressed in KR language (for us: SNePS) • Computational cognitive agent reads the text, • “integrating” text info into its KB, and • making inferences & performing belief revision along the way • When asked to define a word, • Agent deductively searches this single, integrated KB for information to fill slots of a definition frame • Agent’s “context” for CVA = this single, integrated KB

36. Distinguishing Prior Knowledge from Integrated Co-Text • So KB can be “disentangled” as needed for belief revision or to control inference: • Each proposition in the single, integrated KB is marked with its “source”: • Originally from PK • Originally from text • Inferred • Sources of premises

37. Some Open Questions • Roles of spoken/visual/situative contexts • Relation of CVA “context” to formal theories of context (e.g., McCarthy, Guha…) • Relation of I(T) to prior-KB; e.g.: • Is I(Ti) true in prior-KB? • It is “accepted pro tem”. • Is I(T) a “subcontext” of pKB or B-R KB? • How to “activate” relevant prior knowledge. • Etc.

38. Summary • People can figure out a meaning for a word “from context”, where… • “Context” = belief-revised integration of: • reader’s prior knowledge, with • internalized information from the text • This clearer concept of relevant notion of “context” will help us: • evaluate other research • develop our curriculum