PRAXIS Speech-Language Pathology and Audiology Tests Barbara Parr Assessment Specialist, Licensure Development Group

PRAXIS Speech-Language Pathology and Audiology Tests Barbara Parr Assessment Specialist, Licensure Development Group Assessment Development Division November 18, 2006

Overview • The Praxis Series • Licensure Tests • Test Development Process • Test Preparation Products and Services

The Praxis Series Professional Assessment for Beginning Teachers • Licensure • State regulated process • Grants permission/legal authority to practice • Indicates the professional has met the standards for training and practice experience • Indicates readiness for independent professional practice • Indicates minimal competence

The Praxis Series • Licensure Testing • High stakes • Required, not voluntary • One of multiple requirements • Measure knowledge, skills, and abilities deemed important for professional practice • Does not predict performance • Must be legally defensible and professionally credible

The Praxis Series • Age of the program: over 55 years old • User states & jurisdictions • Each state or agency adopts the most appropriate tests for its licensure titles • Each state or agency sets its own cut score for each test • Praxis staff work with states and agencies to set cut scores (standard setting studies) • Number of test titles: 140+

The Praxis Series • Number of administrations per year: 8 • Number of test centers across the country: 650+ • Number of test takers for selected tests: • PPST>150,000 • Elementary Education: Curriculum, Instruction, and Assessment> 35,000 • Speech-Language Pathology >5,000 • Audiology > 1,500

The Praxis Series • Test Offerings: • Praxis I: PPST (Pre-Professional Skills Tests) Measure basic proficiency in reading, writing, and mathematics Computer and paper/pencil delivery • Praxis II: Subject Assessments • Specialty Area tests (140+ titles) • Principles of Learning and Teaching (PLT) tests (4 titles) • Multiple Subjects Assessment for Teachers (MSAT) (2 titles)

The Praxis Series • Praxis III: Classroom Performance Assessments Direct observation in the classroom • Related Programs: • School Leaders Series • School Leaders Licensure Assessment (SLLA) • School Superintendent Assessment (SSA) • ParaPro Assessment

Praxis Speech-Language PathologyandPraxis Audiology • Intended for examinees who have completed a degree program. • Recognized as the national examination in speech-language pathology and audiology • The test is one of several requirements for the Certificate of Clinical Competence issued by the American Speech-Language-Hearing Association (ASHA). • Some states use the examination as part of the licensure procedure. • Complete information may be about certification/licensure from the authority (ASHA or state agency) from which certification or licensure is sought. • .

TEST DEVELOPMENT COMMITTEE ETS RESEARCH ASHA • Format outcomes of the domain analysis into a survey • Conduct a large-scale confirmation survey • Analyze Standards • Analyze Domain • Define Subset of Domain • Draft Test Specifications • Recommend Test Design • Test Development Committee consists of ASHA representatives, content experts, and higher education faculty. • The committee is diverse in terms of race, gender, and educational setting (elementary, secondary, college, etc.)

ASHA NATIONAL ADVISORY COMMITTEE • Reviews outcomes of the job survey • Confirms Domains to Be Tested • Approves Test Specifications • Confirm Test Design • Reviews each new test form • Reviews outcomes of each new test form

Item Development • Item writers are referred to ETS by ASHA. • Beginning an online initiative to access writers throughout the country • Each item that appears on the test undergoes external and internal reviews.

Item Types • Discrete multiple choice • Case studies • Assess knowledge of possible applications to clinical situations and issues. • Research articles • Assess ability to synthesize information and to apply it to specific examples

Praxis Speech-Language Pathology and Audiology • 2 hours • 120 Multiple Choice Questions • Cost $75

Registration All information is available online www.ets.org

Test Takers with Disabilities • All requests for testing accommodations are subject to approval by ETS and must meet ETS documentation criteria. • Reasonable testing accommodations are provided to allow candidates with documented disabilities (recognized under the ADA) an opportunity to demonstrate their skills and knowledge. • http://www.ets.org/disability/index.html

Test Preparation Products and Services • “Test at a Glance” available free online • Test specifications • Sample questions, answers with explanations • SLP eStudy Guides will be available in January, 2007 • Workshops available for faculty of institutions of higher education

Questions and Comments

Praxis: Speech Pathology and AudiologyStatistics Kathryn L. Ricker Associate Psychometrician Center for Statistical Analysis ETS, Princeton, NJ

Purpose of the Praxis Tests The Audiology and Speech Language Pathology tests measure examinees’ academic preparation in and knowledge of the field…it is one of several requirements for the Certification of Clinical Competence…also used by some states for licensure *These tests are to aid in determining preparedness to enter the profession—they are NOT diagnostic, and scores beyond the cut score are NOT meant to be predictive of success in the profession

What’s a psychometrician? • Oversees all statistical processes related to the test title • Equating • Item Analysis • Differential Item Functioning • Test Analysis • Consults on measurement issues • Scoring, test delivery • Ensures that statistics are used appropriately to ensure fairness and validity (tools, not weapons, or magic)

What is Item Analysis? • A statistical description of how examinees responded to each item in a test • Reveals the difficulty of the items • Shows the relative attractiveness of each option in a multiple-choice item • Indicates how well the item discriminates among the examinees

What Can IA Help Detect? • Wrong key • Typographical errors which make the key incorrect • Key has become incorrect because it is obsolete

Test Form Equating • Test forms are not exactly equal in difficulty—scores must be aligned statistically to be FAIR • Scaled scores mean the same thing across forms, raw scores do not! • Use a set of items common to an old form-performance on these items helps establish equivalence of the forms

Differential Item Functioning • Statistical means of examining tests for bias for one group over another • Males vs. Females, Ethnicity (if sample size allows) • Conditions on total test score (i.e., compares item performance relative to “like” test scores) • Used as a guide for careful examination by committees of item reviewers

Some Notes on the Test Stats

Notes from the Test Stats • Repeaters-tend to have lower mean test scores by about 1-1.5 standard deviations—repeaters need to try to use the diagnostic information and test prep materials that are provided—this requires time! • Subscores (category scores) are less reliable than total test scores—be cautious when interpreting raw scores at the level of the content categories!

What can these stats tell me to help my students prepare? • Individual level feedback • Institutional level feedback

Understanding Individual Scores

Understanding Individual Scores Raw category score

Understanding Individual Scores Raw category score Best possible score

Understanding Individual Scores Rough estimate of “typical” performance Raw category score Best possible score

Understanding IHE Statistics • Sample size is important! • Be careful when interpreting statistics with small samples (especially less than 30-50 candidates) • Score distributions • By gender, ethnicity • Detailed Category Score Information • Detailed Score Information by Quartile Why are National and State-wide data both included for comparisons?

FAQs • I took the test twice and got too different raw scores (when I add up my number correct in each category) but the same scaled score. What’s up with that? • I got a scaled score of 590, but the conditional standard error of measurement (CSEM) is 10. Doesn’t that really mean I could have gotten a 610? • My friend and I took the same test at the same time and our number correct are the same, but our scaled scores are different. Doesn’t that mean there was some kind of mistake made?

It Sure Isn’t the National Exam That I Took… Beth Mineo Mollica, Ph.D. University of Delaware Praxis SLP National Advisory Committee

Striving for… • Balance across content areas • Keeping reading burden reasonable • Currency • Logical association with practice demands

Covering the Breadth of the Field

Reflects Practice Demands • Application of knowledge to clinical situation • Question posed in context of clinical situation • Series of questions tied to clinical scenario • Extracting information from research • Integration of information • Working knowledge of law/regulations and their application

Logistics • 120 questions • 2 hours in duration • Multiple-choice format • Item “clusters” tie multiple questions to a single reading passage

Establishing Test Items • Every item reviewed for currency and accuracy • Large inventory of new items generated and extensively vetted

This investigation was motivated by observations that when persons with dysarthria increase loudness, their speech improves. Some studies have indicated that this improvement may be related to an increase of prosodic variation. Studies have reported an increase of fundamental frequency (F0) variation with increased loudness, but there has been no examination of the relation of loudness manipulation to specific prosodic variables that are known to aid a listener in parsing out meaningful information. This study examined the relation of vocal loudness production to selected acoustic variables known to inform listeners of phrase and sentence boundaries: specifically, F0 declination and final-word lengthening. Ten young, health women were audio-recorded while they read aloud a paragraph at what each considered normal loudness, twice-normal loudness, and half-normal loudness. Results showed that there was a statistically significant increase of F0 declination, brought about by a higher resetting of F0 at the beginning of a sentence and an increase of final-word lengthening from the half-normal loudness condition to the twice-normal loudness condition. These results suggest that when some persons with dysarthria increase loudness, variables related to prosody may change, which in turn contributes to improvement in communicative effectiveness. However, until this procedure is tested with individuals who have dysarthria, it is uncertain whether a similar effect would be observed. Watson & Hughes (2006) – The relationship of vocal loudness manipulation to prosodic F0 and durational variables in healthy adults. JSLHR, 49, 636-644.

Question #1 Which of the following represent the independent variable(s) used in the Watson and Hughes study? (A) Prosody of dysarthric speech (B) Gender of the speaker (C) F0 declination and final-word lengthening (D) Vocal loudness (E) Speech intelligibility and communicative effectiveness

Answer to Question #1 An independent variable is manipulated or selected by the researcher to determine its effect on the dependent variable(s). Watson and Hughes manipulate vocal loudness to determine its effect on prosodic F0 and durational variables. Therefore, (D) is the correct answer. (A) and (E) are incorrect, as those variables were not part of the Watson and Hughes study. As only women were studies, the effect of speaker gender was not addressed, so (B) is incorrect. (C) is incorrect, as both F0 declination and final-word lengthening represent dependent variables that change as a consequence of vocal loudness.

Question #2 Watson and Hughes are cautious when suggesting that the speech of some persons with dysarthria improves due to the prosodic changes that result from increasing vocal loudness. Of the following, which is the most likely reason for this caution? (A) Only women were studied. (B) Several acoustic variables related to speech prosody were not included. (C) The prosody of persons with dysarthria may not show similar loudness effects. (D) The results were not statistically significant. (E) Reciting a paragraph aloud is unlike spontaneous speech.

Answer to Question #2 At the end of the abstract, Watson and Hughes suggest a relationship between increased vocal loudness and “improvement in communicative effectiveness” in some persons with dysarthria. This relationship is not directly supported by their study, as individuals with dysarthria were not tested, so (C) is the correct answer. (A), (B), and (E) are incorrect as they do not address the relationship of ddysarthria, vocal loudness, and improved communicative effectiveness. (D) is incorrect for the same reason and, furthermore, the results were statistically significant.

Question #3 Which of the following best describes the experimental design of the Watson and Hughes study? (A) A longitudinal design (B) A multiple-baseline design (C) A between-subjects design (D) A within-subjects design (E) A mixed between- and within-subjects design

PRAXIS Speech-Language Pathology and Audiology Tests Barbara Parr Assessment Specialist, Licensure Development Group