Creation of a Short Form Boston Naming Test for Individuals with Aphasia

1. Conclusions Creation of a Short Form Boston Naming Test for Individuals with Aphasia Christina M. del Toro1,3, Diane L. Kendall1,3, & Craig Velozo2,4 Malcom Randall VA RR&D Brain Rehabilitation Research Center1,Malcom RandallVA HSR&D/RR&D Rehabilitation Outcomes Research Center2, University of Florida, Department of Communication Sciences & Disorders3, University of Florida, Department of Occupational Therapy4 Figure 1. Graves et. al,(2004) BNT Short Form Item Map <more>|<rare> 3 . + | | T| | | 58PALETTE | . | 2 + |T | . | | | | 55SPHYNX . | 1 S+S | . | | 54TONGS . | | 57TRELLIS | . | 42STETHOSCOPE 45UNICORN 0 +M 44MUZZLE 53SCROLL . | 41PELICAN | 24SEAHORSE 31RHINOCEROS | 48NOOSE . | M| | 35DOMINOES . | -1 +S . | 19PRETZEL | | | . | | |T 32ACORN -2 + . S| | | | | | | -3 . + <less>|<frequent> Figure 3. New BNT Short Form Item Map <more>|<rare> 4 . + | | T| | | 59PROTRACTOR 3 . + | 58PALETTE | | . | |S 56YOKE 2 + . | | 55SPHYNX | . S| | 1 + 54TONGS . | 57TRELLIS | | 42STETHOSCOPE . | 41PELICAN | 24SEAHORSE 0 . +M | | 35DOMINOES . | M| | 19PRETZEL -1 . + | | | 32ACORN . | | 10TOOTHBRUSH -2 + |S . | | | S| -3 + . | 5WHISTLE | | | | -4 + | |T . | | | -5 + 1BED | | | | | -6 . + less>|<frequent> Phase 3 Results & Discussion Background Phase 1 Results & Discussion • Aim 3: Rasch Analysis of a new short form • Total N analyzed: 85 out of 100 participants. 15 participants scored 0 correct providing insufficient data and were dropped from analysis • Range of person ability & item difficulty: -6 to 4 logits, an increase from the previous forms. The larger range at the low end of the hierarchy (-6) indicates the form captures lower levels of ability not previously measured in phase 1 and 2 forms. • Mean ability level of the participants is higher than the difficulty level of the test. • No items misfit based on Infit Mean Square (≥1.4); • 1 item misfit on Outfit Mean Square (≥2.0) ranges. • Person separation = 1.97 • Number of Strata = 2.96, indicating the short form can divide people into approximately 3 ability levels, similar to the other forms. • Person reliability = 0.80 • Item reliability = 0.97 • No items were redundant based on the unit of measure (item difficulty; figure 3) • Aim 1: Rasch Analysis of Graves et. al (2004) short form • Total N analyzed: 74 out of 100 participants. 26 participants scored 0 correct providing insufficient data and were dropped from analysis • Range of person ability & item difficulty: -3 to 3 logits • Mean ability level of the participants is lower than the mean difficulty level of the test., (M in Figure 1), indicating the test should include easier items to measure individuals of lower ability. • No items misfit based on Infit Mean Square (≥1.4), and Outfit Mean Square (≥2.0) ranges, an indication of the unidimensionality of the short form. • Person separation = 1.83 • Number of Strata = 2.7, indicating the short form can divide people into approximately 3 ability levels. • Person reliability = 0.77 • Item reliability = 0.89 • 3 of the 15 (20%) items were redundant (items that occur at the same unit of measure or difficulty level; Figure 1), suggesting a rationale for item reduction. The Boston Naming Test (BNT; Kaplan, Goodglass, & Weintraub, 1983) as well as the short form included in the second edition by Mack, Freed, Williams, & Henderson (1992), were developed using traditional standardization methods of classical test theory. Standardization is performed on the test as a whole rather than on the individual items, without attention to the difficulty level of items. Although item statistics can be generated post hoc, they apply only to thatgroup of subjects on those particular items. Rasch analysis however takes advantage of recent advances in psychometrics, allowing for analysis at the item level. Item response theory claims that the probability of a person’s response to an item is the combined function of that person’s ability and the difficulty level of the item (Bond & Fox, 2001). One of the short forms of the BNT (Graves et al, 2004; Bond & Fox, 2001) employed Rasch analysis to responses from neurologically-healthy individuals and individuals diagnosed with various forms of dementia. However, considering the differences in underlying mechanisms of anomia for dementia compared with aphasia, it is not known if this particular short form is valid for use with individuals with aphasia. Additionally, the Mack et. al (1992) short form currently published with the BNT was developed using data from individuals with Alzheimer’s disease and age-matched healthy controls. Thus, there is a need for empirical evidence that these forms are appropriate for assessing naming function in aphasia. Purpose The primary aim of this study was to investigate the validity of items from the Boston Naming Test for use in a 15 item short form for an aphasia population. We measured item difficulty, item fit, item redundancy, person separation, reliability, and the original proposed hierarchy of lexical frequency. To address this aim we analyzed two previously published forms (Graves, Bezeau, Fogarty, & Blair, 2004; Mack, Freed, Williams, & Henderson, 1992)and one developed for this study. Figure 2. Mack et. al (1992) BNT Short Form Item Map <more>|<rare> 4 . + | . T| |T | 58PALETTE | | 3 + | . | | | | 55SPHYNX | 2 . + S| |S . | | | | 42STETHOSCOPE 45UNICORN 1 . + | 39HAMMOCK 52TRIPOD . | 29BEAVER | | . | | 0 +M 13OCTOPUS . M| 23VOLCANO 36CACTUS | | . | | | 26CANOE -1 . + | | 10TOOTHBRUSH | . | |S | 20BENCH -2 S+ . | | | | | . | -3 + 4HOUSE | 7COMB | | |T . | | -4 . T+ less>|<frequent> Phase 2 Results & Discussion Methods • Aim 2: Rasch Analysis of Mack et. al (2004) short form • Total N analyzed: 83 out of 100 participants. 17 participants scored 0 correct providing insufficient data and were dropped from analysis • Range of person ability & item difficulty: -4 to 4 logits, indicating this form captures a wider range of ability than Graves et. al (2004) form. • Mean ability level of the participants is nearly equivalent to the mean difficulty level of the test (M on Figure 2). This form includes several easy items thus, measuring individuals of lower ability not measured by the Graves et. al (2004) form. • No items misfit based on Infit Mean Square (≥1.4); 3 items misfit based Outfit Mean Square (≥2.0) ranges, suggesting unexpected responses away from a person’s ability (naming a harder item or not naming an easier item). • Person separation = 2.09 • Number of Strata = 3.12, indicating the short form can divide people into approximately 3 ability levels similar to the Graves et. al (2004) form. • Person reliability = 0.81 • Item reliability = 0.96 • 3 of the 15 (20%) items were redundant (items that occur at the same unit of measure (item difficulty; Figure 2), suggesting a rationale for item reduction. • Participants: • 100 individuals with aphasia • a single left hemisphere stroke at least 6 months prior to enrollment • right-handed • monolingual English speaking • no prior CVA or pre-existing neurological illness • No severe impairment in vision or hearing • Overall, a wide range of difficulty level (easy and hard items) is needed to capture the large range of ability levels seen in aphasia. • The redundancy observed in the 60 item corpus, suggests the need for a shortened version. Thus, for individuals who have aphasia as a result of stroke, it is recommended to use 15 items which span a large difficulty range without redundancy in the measure of difficulty. • The original hypothesis for the creation of the BNT items was based on lexical frequency however, discrepancies between item difficulty and word frequency suggests other factors may be affecting word production (e.g. syllable length, phonotactic probability, etc). Thus, our lab will conduct two further studies: • Data Collection: • Archival data from the patients screened at the VA Brain Rehabilitation Research Center • All participants were given the full 60 item BNT by a licensed speech-pathologist following original directions for administration and scoring • Only spontaneous responses without cueing were included • Data Analysis: • Rasch analysis was computed using the responses of 100 individuals • with aphasia in 3 phases: • 1) 15 items from the Graves et. al (2004) short form • 2) 15 items from the Mack et. al (1992) short form • 3) 15 items newly chosen for this study based on the results of phases 1 & 2 • Items from the Graves et. al (2004) and the Mack et. al (1992) forms were kept based on Infit and Outfit Mean Squares • Overlapping items were excluded • 4 items were added to fill in gaps in the hierarchy of difficulty level • and expand the range to include easier items • Participant responses were analyzed with the WINSTEPS Rasch analysis computer program (Bond & Fox, 2001; Linacre, 2004, 1994). • Infit mean squares greater than or equal to 1.4 were considered to misfit • Outfit mean squares greater than or equal to 2.0 were considered to misfit • The original 60-items of the BNT will be analyzed using responses from the rating scale scoring system which includes the semantic and phonologic cues. Inclusion of participants responses with cues will provide additional information of each participant’s ability allowing a more fine-grained comparison to the test’s difficulty. • The original 60-items of the BNT will also be analyzed using a hierarchy of psycholinguistic variables such as phonotactic probability, syllable length, cluster effect, and neighborhood density. This analysis will determine if the BNT is strictly a test of lexical/semantic frequency. This presentation is based upon work supported with resources and the use of facilities from the Office of Research and Development, Rehabilitation R&D Service, and the Brain Rehabilitation Research Center, of the Department of Veterans Affairs.