The Category Test: History, development, and future directions

1. Nick A. DeFilippis & Jonathan N. Dodd Georgia School of Professional Psychology The Category Test: History, development, and future directions

2. Early Tests of Concept Formation • Ach (1921), a German researcher developed a means of studying concept formation (Sakharov, 1930; Hanfmann & Kasanin, 1937). • Vygotsky’s variation (Hanfmann & Kasanin, 1937). • Hanfamann & Kasanin (1937) translated the Vygotsky test into English and identified 3 basic factors of concept formation.

3. Early Tests of Concept Formation • Weigl Color-Form Test (1941) consisted of 12 cardboard figures that could be grouped together according to their colors or forms. • The Goldstein and Scheerer Sorting Test (1941) required the subject to sort 7 of 33 common objects together and describe how they are alike and deliniated abstract concept formation into 3 levels of graded sophistication: Concretistic, Functional, and General.

4. Early Tests of Concept Formation • Ward Halstead developed an object sorting task similar to Goldstein and Scheerer’s known as the Halstead Object Sorting Test • Straus & Werner (1942) conducted the first studies with this instrument comparing children with acquired brain injuries with those with mental retardation.

5. Birth of the Category Test • Due to variable success with the Halstead Object Sorting Test, Ward Halstead developed the Halstead Category Test, designed to measure “grouping behavior.” • The initial version of the Category Test contained 9 subtests totaling 360 items. • The test was administered via an apparatus that contained a rotating drum and reading lens.

6. Evolution of the Category Test • During the 1940’s and 1950’s the method for administering the CT changed. • Incorporated the use of slides, the number of items was reduced to 208, and subtest reduced to 7, and it was placed in a console. • Variations of test administration included shortened versions, paper-and-pencil forms, a card version, a computer version, a booklet version, and an adapted version based on item response theory.

7. Evolution of the Category Test • The Booklet version of the Category Test, known as the “Booklet Category Test” (BCT) has enjoyed much popularity due to it’s ease of transportation and administration, as compared to the original version using a cumbersome and heavy console. • The booklet version consisted of 208 8 x 1 paper cards containing stimuli identical to the original and bound into two separate booklets.

8. Shortened versions of the Category Test • Charter, Swift, and Blusewicz (1997) developed a shortened version of the CT composed of all items in subtests 1 through IV and 20 items from Subtests V from the original CT. • Charter et al.’s sample consisted of 67 diffusely brain damaged and 109 non-brain damaged hospital patients matched for age and education. • Limitations include limited demographic data for the control group and risk of selection bias.

9. Paper-and-pencil version • Adams and Trenton (1981) modified the original CT into a paper-and-pencil version in efforts to reduce the cumbersomeness. • They also created an alternative version of their paper-and-pencil version with good split-half-relability. • Adams and Trenton noted significant practice effects; thus, repeat testings between the two versions are not recommended.

10. Booklet Category Test • Research has shown that the BCT correlates significantly with the original Halstead Category Test (McCampbell and DeFilippis, 1979). • Comparing Category Test performance of 30 chronic alcoholics to that of 30 normal control college students, with order of test administration (original Halstead slide vs. booklet version) varied in administration, strong correlations of .913 and .804 were found between normal and alcoholic participants respectively. • Significant practice effects b/w first and second administrations of the CT in both normal and alcoholic groups.

11. Psychometric Properties of the CT • The CT’s overall internal consistency is approximately 97% (Lopez, Charter, & Newman, 2000). • Split-half reliability is 0.98 (Shaw, 1966). • The CT shows significant age and education effects (Heaton, Grant, & Matthews, 1991).

12. Psychometric Properties • Most researchers agree that the CT is a multidimentional instrument measuring several independent but complimentary cognitive abilities including: • Concept formation, abstract reasoning, nonverbal problem solving, and attention (Johnstone, Holland, & Hewett, 1997) • Learning (Laatsch & Choca, 1994) • Judgment (Lopez, Charter, & Newman, 2000)

13. Psychometric Properties:Construct Validity • Research indicates that the BCT correlates with measures of intelligence, memory, and nonverbal problems solving (Allen, Goldstein, & Mariano, 1999). • Lansdell & Donnelly (1977) conducted a factor analysis that indicated that the CT loaded highly on visuomotor factors, as well as WAIS Performance subtests, Block Design, Picture Arrangement, and Object assembly.

14. Psychometric Properties: Construct Validity • Leonberger, Nicks, Goldfader, & Munz (1991) found that the Category Test loaded on tests of visual, but not verbal memory. • Jonstone, Holland, & Hewett (1997) also conducted a factor analysis finding that

15. Psychometric Properties: Construct Validity • Johnstone, et al. (1997) fourn that there was not a single CT factor of abstract reasoining. Rather, they identified three different factors (cognitive demands) including: • Symbol Recognition/Counting • Spatial Positional Reasoning • Proportional Reasoning

16. Psychometric Properties: Lateralization Effects • Dodd, McDermott, Goldstein, & DeFilippis (2008) found that the Spatial Positional Reasoning (SPR) Index correlated with patients with right hemisphere lesions and Performance IQ scores. • In contrast, the Proportional Reasoning (PI) index correlated with patients with left hemisphere lesions and their Verbal IQ scores.

17. Psychometric Properties: Effort • Using item analysis, Laatsch & Choca (1991) found that all items from Subtests I and II did not discriminate between brain-injured and normal subjects. • Thus, most test-takers answered all items from Subtest I and II correctly, regardless of overall performance. • Hence, performance on Subtests I and II can be used as measures of effort, rather than neurocognitive functioning.

18. Subscales of the Category Test • The Category Test has received criticism for providing only a single Error score; although it requires the use of many component cognitive abilities. • Several efforts have been made at developing new scales to the Category test including scales for: • Categorization • Set Maintenance • Perseveration • Memory

19. Categorization Scale • Rosenblum, et al. (2005) developed a Category scale for the Booklet Category Test (BCT-CAT), which is generated from clusters of similar test items that run consecutively through each subtest. • Moderate correlation between the BCT-CAT and the Categories completed scale of the WCST (r = .478, p < .01). • Moderate convergent validity with stronger correlation between the WCST-CAT and the BCT-CAT than the BCT-Error score (r = .478, p = .01 vs. .396 p = .01).

20. Categorization Scale • Discriminant validity was established against the WCST-Failure to Maintain Set and WCST-Perseveration Errors. • The BCT-CAT also had higher correlations with visual and verbal memory tests than the BCT-Memory scales (r = .644 & .410, p = .01 vs. .408 & .383, p = .01). • Land, DeFilippis, Hill, & Dodd, (2007) demonstrated further concurrent validity of the CT-CAT scales with the Halstead Impairment Index.

21. Perseveration Scale • Pelham (2001) made the first known attempts at developing BCT subscales intended to measure perseveration, loss of set, and memory. • Minassian, et al., (2003) published a series of experiments attempting to develop further validity and reliability for Pelham’s scales.

22. Perseveration Scale • Pelham found moderate correlations with the WCST Perseverative Responses scale (r = .407, p < .01). • Pelham also described how to score Subtest IV, but not Subtests III, V, and VI. • Minassian, et al., (2003) demonstrated that the BCT-P correlated moderately with the WCST Perseverative Response scores, as well as with the WMS-R scores, which demonstrated good convergent, but poor discriminant validity of the BCT-P.

23. Loss of Set scale • Pelham’s Loss of Set scale (BCT-LS) was calculated based on the total number of errors made after three consecutive correct responses. • This scoring rule was applied to subtests III, IV, V, and VI. • The BCT-LS scale did not correlate with any other measures of attention or set maintenance.

24. Loss of Set scale • Minassian, et al., (2003) also were unable to demonstrate correlations between the BCT-LS scale and measures of attention and the WCST FTMS scale.

25. Memory Scale • Pelham’s memory scale (BCT-M) is the percentage of possible correct scores from Subtest VII. • Pelham showed strong correlations between the BCT-M scale and WMS-R scores. • Minassian, et al., (2003) demonstrated a significant positive correlation between the BCT-M score and the CVLT-II learning score for the first 5 trials.

26. Scales in Development • Revised scoring criteria for the above subscales has been proposed by Dodd & DeFilippis. • New methods and underlying constructs are currently in development for: • Category scale • Set Loss scale • Perseveration scale

27. New Category Scale (CAT-2) • The CAT-2 is established after 3 consecutive correct responses are achieved within a subtest. If the patient loses set and is unable to pick back up within 3 items, the CAT-2 “re-sets,” as it is assumed that the previous three items correct was by chance, and the patient never truly grasped the category/concept.

28. New Set Loss scales • Set loss errors occur whenever a patient has established the CAT-2 and then responds incorrectly. • 2 types of set loss • Attentional set loss • Conceptual set loss

29. Attentional Set Loss (SL-A) • The SL-A is scored only after an incorrect response is given after the CAT-2 is established and within a subgroup of items that have similar stimulus structure. • These are considered “true” set loss and represent an unexpected change in the patient’s approach to the test resulting from a sudden dip in attention or impulsivity.

30. Conceptual Set Loss (SL-C) • The SL-C is an incorrect response given after CAT-2 is established, between subgroups within a subtest in which the core stimulus percept changes. • This represents weak concept formation, poor abstraction, poor mental/conceptual flexibility, and easy confusion.

31. New Perseveration scale • Perseveration on the BCT occurs when a patient persists with the incorrect response or principle for two or more items despite being told “incorrect.” • Further analysis of the patient’s approach to the test may reveal more concrete forms of perseveration verses conceptual forms of perseveration.

32. New Perseveration scale • The two most common forms of perseveration that are feasibly scored without prompting the patient include: • counting on subtests III, V, and VI, and • continual misplacement of roman numerals III and IV on Subtest IV.

33. New Perseveration scale • For Subtest III, the most common form of PSV-W involves persisting with a counting principle from the previous subtest. • PSV-W for Subtest IV manifests after the numbers disappear from the matrix, and the patient regresses back to assigning the III and IV to the traditional placements within the quadrants, rather than the clockwise progression of the roman numerals that is presented in the test.

34. New Perseveration scale • Perseveration on subtests V and VI is scored when the patient persists with the counting principle. • Again, the examiner would only qualify an error as a perseveration if the patient persists with either counting objects that are the same (PSV- CS) or counting objects that are different from the other figures in that item (PSV- CD).

35. References Goldstein, K. & Scheerer, M. (1941). Abstract and concrete behavior: An experimental study with special tests. Psychological Monographs, 53, (2, Whole Number 239).