1 / 28

“You Can’t Always Get What You Want: Accountability and Limits in Score Reports to Boards”

Expect the Unexpected: Are We Clearly Prepared?. “You Can’t Always Get What You Want: Accountability and Limits in Score Reports to Boards”. Censored image of psychometrician in Mick Jagger style tight leather pants. Troy Elliott Association of Social Work Boards.

yasuo
Download Presentation

“You Can’t Always Get What You Want: Accountability and Limits in Score Reports to Boards”

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Expect the Unexpected: Are We Clearly Prepared? “You Can’t Always Get What You Want:Accountability and Limits in Score Reports to Boards” Censored image of psychometrician in Mick Jagger style tight leather pants Troy Elliott Association of Social Work Boards Council on Licensure, Enforcement and Regulation 2006 Annual Conference Alexandria, Virginia

  2. Who We Are Association of Social Work Boards (ASWB) 49 states, DC, USVI, 7 provinces Public protection mission Social work licensure examinations—about 25,000 administrations annually Other services: Disciplinary databank, continuing education provider approval, credentials registry, communication, etc. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  3. The ASWB Examinations • Five categories (Associate, Bachelors, Masters, Advanced Generalist, Clinical) • 4 option multiple choice • 150 scored, 20 pretest • CBT • 4 hour time limit Psychometric and administration support through ACT, Inc. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  4. The Score IRT process – no single raw score applicable to all forms Slight variations to maintain consistent overall difficulty – anchor exam Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  5. The score, continued, unfortunately So what everyone gets is the equated score, right? If only it were that easy… Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  6. The score, continued, unfortunately The Scaled Score Zone, or Ketchup vs. Catsup— the debate rages on “70” versus “75” Where does it exist? Why does it exist? How many English majors does it take to develop an apt simile? 70 75 Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  7. Now that the basics are covered… Who gets what right away: Passing Candidates Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  8. Now that the basics are covered… Who gets what right away: Failing Candidates Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  9. Now that the basics are covered… Who gets what right away: Regulatory Boards Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  10. Summary reports Every year, to boards Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  11. Choose your cliché The greatest thing since sliced bread….or something else entirely? Our great idea: reports to schools Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  12. School data What the schools get Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  13. How could anything go wrong? • Misuse – program outcome, marketing • Requests for more data • Fingerpointing Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  14. What does this have to do with my topic? Reporting service highlighted relationship between regulatory boards and schools Some boards interested in school data Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  15. Data salad, hold the context Requests and questions from ASWB member boards • Subscores? • Jurisdictional pass rates? • School data shared with everyone? Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  16. Data salad, hold the context ASWB’s initial answers: • No • No • No Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  17. What’s our problem anyway? The three Ps: philosophy, psychometrics, potential litigation Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  18. What’s our problem anyway? Philosophy Whose data is it? Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  19. What’s our problem anyway? Psychometrics 150-item test with content areas ranging from 3 – 22 percent of the examination. Schools—and sometimes, entire jurisdictions—with small testing populations, even smaller first-timer numbers. Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  20. What’s our problem anyway? Potential litigation • Misuse/misrepresentation of data • Hiring/firing/program changes based on poor data Would we prevail? Probably. But at what cost? Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  21. Coming soon to an association near you Protection of the exam meets service to members Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  22. Under consideration… Compilation of jurisdictional pass rates, with identification where allowed All school data reported to boards (in addition to existing program) Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  23. What did we learn? A victim of our own success? Licensing (and the exam)more important than ever • Fewer exemptions • More importance among employers • Recognition by educational programs Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  24. What did we learn? Looking for outcomes in all the wrong places • Test data can get caught up in political and economic pressures faced by schools Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  25. What did we learn? Member education never stops • Turnover not just limited to board members • Multidisciplinary boards increase the need for education (“If they can do it why can’t you?”) • Culture of profession regulated impacts expectations, available options Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  26. What do we still need to learn? Information culture – has the data train left the station? • Expectations • “Good” data vs. “transparency” • Whose data is it anyway? Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  27. What do we still need to learn? Association culture – when is service a disservice? • Long range planning, goal identification: don’t forget overall profile • Tough decisions: sometimes you have to make them Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

  28. And you thought I’d never shut up Troy Elliott, Communications Director Association of Social Work Boards 400 South Ridge Parkway, Suite B Culpeper, VA 22701 800-225-6880 Email: info@aswb.org www.aswb.org Presented at the 2006 CLEAR Annual Conference September 14-16 Alexandria, Virginia

More Related