Categorization vs. logic in category conjunction: towards a model of overextension

1. Categorization vs. logic in category conjunction: towards a model of overextension Dr. Fintan Costello, Dept of Computer Science, University College Dublin. Fintan.Costello@ucd.ie

2. Classification in conjunctive categories The ability to form new complex categories by the conjunction of existing categories is a basic logical operation and an important part of classification and reasoning. Any worthwhile AI model of categorization should be able to reason about conjunctive categories in a way that mirrors human reasoning about such conjunctions. In this talk I describe a particular problem which AI models must face in attempting to mirror human conjunctive classification: that of overextension. I will then describe a computational model that attempts to address this problem, and will demonstrate that model in an experimental test.

3. Overextension in conjunctive categories: a challenge to logic In the logical, set-theoretic, view of category conjunction, an item is a member of a conjunction A&B only if it is a member of the single category A and a member of the single category B. However, people regularly and reliably break this logical rule when classifying items in some conjunctive categories. They regularly classify items as non-members of a constituent category A (or B) but as members of conjunction A&B. This deviation from set logic is called overextension. Are ‘blackboards’ furniture? no Are ‘blackboards’ school furniture? yes Is ‘pool’ a sport? no Is ‘pool’ a game that is also a sport? yes Overextension and similar effects also occur, for example, with drawings of faces (‘is this person intelligent? No. Is this person happy and intelligent? Yes.’), in economic decisions, and in probability judgements.

4. Most accounts of overextension and similar ‘conjunction fallacy’ effects(e.g. Hampton, 1996, Tversky & Kahnemann, 1983) are based on an idea of representativeness or similarity to a conjunctive prototype. According to this view, for example: • ‘blackboards’ are more similar to the prototype of ‘school furniture’ than to that of ‘furniture’, and so are members of the first but not the second. • This gives a mechanism for how overextension can occur, but does not explain why this apparent error should persist and be widespread in human reasoning. Explainingoverextension I will argue that overextension is a logical consequence of a more basic requirement in human reasoning: the need to have an open, as opposed to closed, system of category conjunction. In an open category system, items can be classified in new, previously-unseen category conjunctions: conjunctions for whose existence there has been no previous support.

5. Membership(A |A) Membership in A&B Membership(B |B) To classify an item as a member of a category A, we use two sources of evidence : positive evidence (if the item has features that occur in known members of A) & negative evidence (the item has features that occur in known members of other categories: in A’s contrast setA). Classification in a conjunction A&B: Positive and negative evidence for classification Combine these to get conjunctive membership Use both to compute constituent membership Get positive and negative evidence Positive: A Negative:A Positive: B Negative:B Suppose A&B is a new conjunction – there are no known members of AB. Then A B, and B A . This means that positive evidence for membership in A is negative evidence for B, and vice versa. This means that no items will get good classification scores in a new conjunction A&B. This is a closed system of categories.

6. To produce an open system for conjunctive classification, we must ensure that positive evidence for membership in A or B does not get counted as negative evidence for membership in the conjunction A&B. Let the contrast set for A&B to be AB: items outside both A and B. Positive: A Negative: AB Membership(A | AB) Membership in A&B Positive: B Negative: AB Membership(B | AB) An open system for conjunctive classification Combine these to get conjunctive membership Use both to compute constituent membership Get positive and negative evidence In a system like this, positive evidence for membership in A will not count as negative evidence for membership in B, or vice versa. This is an open system: items will be classified in A&B even though A&B is a new conjunction (there are no known members of AB).

7. When classifying an item in a conjunction A&B, positive evidence comes from A and B and negative evidence comes from AB (A&B’s contrast set is AB). This is logically correct for ‘open’ conjunctions. However, if X’s features are rare in the conjunctive contrast set AB, X will have very little negative evidence for membership in A&B, and so will be classified as a member of the conjunction A&B. In an open system of classification, when classifying an item in a single category A, positive evidence comes from known members of A and negative evidence comes from known non-members of A (the contrast set for A isA). This is logically correct for single categories. An open system produces overextension This means that an item X with features that are relatively frequent in single category A but that are very frequent in single category B (and where AB= ) will have negative evidence for memberhip in A, and so will be classified as a non-member of the single category A. This is an overextension response for the category and conjunction.

8. An example: diagnostic evidence model of classification I will illustrate these ideas using a simple model of single and conjunctive classification: the diagnostic evidence model. This model assumes that people represent categories by storing sets of category members in memory. From these sets, diagnostic features are computed: these features act as evidence to identify category members. An item’s classification in a single or conjunctive category is a function of the diagnosticity of its features for that category or for the constituentcategories of that combination. A continuous-valued logic is used to combine evidence from multiple diagnostic features, and to combine evidence for membership in the constituent categories in a combination.

9. Feature diagnosticity: category and contrast set A new item is placed in a category C if it has diagnostic features for C: features common in stored members of C but rare in its contrast set Kc (the set of stored items outside C). Diagnosticity of feature x for category C (relative to contrast set KC) An item with diagnostic features for a category gives evidence for membership in that category. A crucial factor in this computation is the contrast set. As discussed earlier, for a single category A the contrast set is the set of items outside A. For a conjunction A&B, however, the contrast set is the set of items that are outside both A and B.

10. Diagnosticity in combinations: change in contrast set Single category contrast set All stored items that are not members of that single category Conjunctive category contrast set  All stored items not members of anyconstituent of combination  If a feature occurs often in a single category’s contrast set, but less often in the contrast set for a combination containing that category, the feature will be less diagnostic for the single category, but more diagnostic for the combination. This change in diagnosticity is what causes overextension. If there is no difference in feature occurrence in the contrast set for a single category and a combination, overextension will not occur.

11. Multiple features: Overall evidence for membership Continuous-valued logic. Variables between 0(false) and 1(true) not A = 1-A A and B = A*B A or B=1-(1-A)*(1-B) Membership in single categories An item i with features X1 X2X3 gives evidence for membership in category C if feature X1or X2or X3 is diagnostic for C: E(i|C|KC)= 1-(1-D(x1|C|KC))* (1-D(x2|C|KC))* (1-D(x3|C|KC)) Membership in combined categories Item i gives evidence for membership in conjunction C1&C2 if i has some features diagnostic for C1 and others diagnostic for C2(relative to K1&2) E(i|C1&C2|K1&2) = E(i|C1|K1&2) * E(i|C2|K1&2)

12. Testing the model: an experiment with single categories and conjunctions Method: Investigates classificationand overextension (logical errors) using a controlled set of patient-descriptions (items), symptoms (features on 3 dimensions) and categories (diseases A, B, and C). Training phase: 18 participants get a set of patient descriptions (training items) with certain diseases and symptoms, and learn to identify diseases (to criterion). Test phase: Participants get 5 new patient descriptions (test items) with new symptom combinations. For each test item participants separately rate patient as having disease A, B, C, A&B, A&C, B&C. Each test item therefore occurs 6 times in the test phase (with 6 different rating questions). Results. Classification scores and frequency of overextension errors are compared with those predicted by the diagnostic evidence model.

13. Training items These disease categories have a family-resemblance structure: there are no simple rules linking an item’s symptoms and category membership. Participants learned categories by studying items like these. Different participants got different symptom-words in the training materials, but all had the same symptom distribution. Participants then classifed new “test” items in categories and category conjunctions. Item Symptoms Category EYES SKIN MUSCLES

14. Test items Symptoms rated as member of category or conjunction Participants learned training items and then classified the test items as members or non-members of the categories and conjunctions. The model was given the training items and used the feature distribution there to compute membership for each test item in each category and conjunction.

15. Results Classification The model’s classification scores were a good fit to participants’ average classification scores for items as members of single categories (r=.91, <.001, %var=.82) and as members of category conjunctions (r=.85, <.001, %var=.73) in the experiment.No free parameters were used in fitting the model to the data. Overextension Nearly half of participants (46%) classified test item 2 as a non-member of category A but a member of disease conjunction A&C. The next most frequent case of overextension occurred for test item 4 30% of participants classified test item 4 as a non-member of single disease category B but a member of disease conjunction B&C. All remaining cases of overextension were produced by less than 15% of participants.

16. Model’s predicted overextension To produce overextension predictions from the model a “membership threshold” was applied to the model’s classification scores, so that if an item was given a score above the threshold in a category or conjunction it was taken as a member; if the score was below that threshold, the item was a non-member of the category or conjunction. Excluding extreme (very high or very low) threshold values, only one membership threshold value produced an overextension response in the model’s computed classification scores. With this threshold, test item 2 was classified as a non-member of category A but a member of conjunction A&C. This was the item, category and conjunction for which overextension was most frequent in the experiment. The chance of the model selecting one case of overextension at random, and that being the most frequent case of overextension in the experiment, is 1 in 45 (p < 0.05).

17. The basic ideas in this talk can be summarised as follows: Conclusions Overextension is an apparent deviation from logic which AI models that aim to reason about and use conjunctive categories must address. Looking deeper, however, overextension is a logical consequence of (i) the need to use positive and negative evidence in classification, (ii) the need to have an ‘open’ system of conjunctive categories: one which can classify items in conjunctions for whose existence there has been no previous support. Take-home message: If you want an open system of conjunctive classification, and you want to use positive and negative evidence in classification, you will have to think carefully about sources of negative evidence. And you will have to resign yourself to overextension and other apparent logical errors.