Our Literature , Our Field: Findings and Trends From Postsecondary Disability Literature

1. Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University of Connecticut Presentation at the Association on Higher Education and Disability Annual Conference Baltimore, MD July 11, 2013 Center on Postsecondary Education and Disability, ; AHEAD 2013

2. About Us • Allison • Assistant Professor, Department of Educational Psychology, Neag School of Education, UConn • Research Associate, Center on Postsecondary Education and Disability (CPED) • Adam • Doctoral Student, Department of Educational Psychology, Neag School of Education, UConn • Joe • Associate Professor , Department of Educational Psychology, Neag School of Education, UConn • Director of CPED, Neag School of Education, UConn • Member, NPSO Advisory Board Center on Postsecondary Education and Disability, ; AHEAD 2013

3. Special Thanks Lyman L. Dukes III Michael Faggella-Luby Nicholas Gelbar Jennifer S. Kowitt Melissa Root Center on Postsecondary Education and Disability, ; AHEAD 2013

4. Session Objectives To explain the rationale for conducting this comprehensive literature review To explain the background and methods used To present specific key findings to date To present suggestions for future research To facilitate discussion regarding future research directions Center on Postsecondary Education and Disability, ; AHEAD 2013

5. Project Background The Vocational Rehabilitation Act of 1973 required all institutions of higher education to ensure access to qualified students with disabilities (SWD). Forty years after the passage of the Act, 11% of college freshmen report having a disability (U.S. G.A.O., 2009). The profession of disability services is now longer a nascent field in higher education Center on Postsecondary Education and Disability, ; AHEAD 2013

6. Project Background • To date, a comprehensive analysis of the literature dealing with disability and higher education has not been conducted • This literature is broad in scope and dispersed across a variety of disciplines (e.g., special education, higher education, psychology, sociology) • The 40-year anniversary of the passage of Section 504 provides an anniversary to review the field’s literature: • What topics have been studied? • What populations have been studied? • What methodologies have been employed? • How much of the literature is research? How much is not research? • What aspects of the field have substantial evidence and support? • What aspects are lacking substantial evidence and support? • What research areas within the field are likely to receive greater attention in coming years? • Relevance to practitioners, researchers, policy makers Center on Postsecondary Education and Disability, ; AHEAD 2013

7. Project Background • Initially looked to a process used in secondary transition by the National Secondary Transition Technical Assistance Center (NSTTAC: Test, Fowler, Kohler & Kortering, 2010) • To review evidence-based practices in secondary transition based on quality experimental studies • Method: • Procedures: (a) electronic search, (b) reviewing reference lists, (c) hand searches of journals, and (d) updating by replicating the initial procedures • Criteria: (a) publ. after 1984, (b) SWD in subjects 11-22 yrs, (c) IV or DV aligned to five areas of Kohler’s Taxonomy • Focus: Included systematic reviews and group or single subject design studies • Lens: Applied NSTTAC decision rules for determining levels of evidence (Strong, Moderate, Potential) • Total: 240 reviews and intervention studies Center on Postsecondary Education and Disability, ; AHEAD 2013

8. Broad Definitions (Helsel, Hitchcock, Miller, Malinow, & Murray, 2006; Twyman, 2008) Center on Postsecondary Education and Disability, ; AHEAD 2013 From Cameto, Mazzotti, & Test (2011)

9. Findings • Identified 33 evidence-based practices in secondary Transition • Categorized using Kohler’s Taxonomy for Transition Programming • 3in Student-Focused Planning (e.g., involving students in IEP) • 26 in Student Development (e.g., life skills, purchasing skills) • 1in Family Involvement (training parents about transition) • 3 in Program Structure(extending services beyond secondary school) • No practices identified in the area of Interagency Collaboration • Only 2 evidence-based practices have a strong level of evidence: • teaching life skills, teaching purchasing skills • 28 practices had a moderate level of evidence • For more information, see: Test, D. W., Fowler, C. H., Richter, S. M., White, J., Mazzotti, V., Walker, A. R., & Kortering, L. (2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32(2), 115-128. Center on Postsecondary Education and Disability, ; AHEAD 2013

10. Project Background • Genesis was a request from NSTTAC to present information about evidence-based practices regarding: • Successful transition to postsecondary education • Success in postsecondary education • Our plan: • Initially, to follow the NSTTAC meta-analysis procedures • But, postsecondary education lacks a taxonomy for the literature • Postsecondary education does not use the evidence based practice standards required in secondary education • No prior sorting of the literature, either by topical or research categories • Required a regrouping and new direction Center on Postsecondary Education and Disability, ; AHEAD 2013

11. Our Method • Began with review of 80+ JPEDarticles from 2000-2010 • Identifying common themes and topics • Development of broad content “domains” • Identification of respective sub-domains • Domains sent to past two JPED editors for feedback • An electronic rating form was developed and revised • JPED articles from 10 issues reviewed by four coders • Reliability determined, team debriefing, further refinement of domains, rating form Center on Postsecondary Education and Disability, ; AHEAD 2013

12. Initial Domain Descriptions Center on Postsecondary Education and Disability, ; AHEAD 2013

13. Our Method • Discussion revealed overlaps, determination of key terms • e.g., “policies and procedures”, “experiences” • Where does eligibility “belong”? • Difference between institutional and program legal compliance? • What about studies of instruments and proposed constructs? • Domains collapsed and updated: • Student level • Program level • Faculty/staff level • Construct level • JPED articles from an additional 5 issues reviewed by four coders • Reliability determined at 75%-85% • Debriefing lead to 100% agreement; refinement of terms, inclusion and exclusion criteria Center on Postsecondary Education and Disability, ; AHEAD 2013

14. Our Method • Concurrently, 500 articles from other sources collected • Sorted into domains; reliability measured • 88% - 96% for sorting • Articles provided a broader perspective and lead to further refinement of the subdomains • Validity check by 8 former editors or co-editors of JPED • Measured the clarity of domain definitions • all were strongly agree or agree that the definition is clear • Requested suggestions for missing domains • Fit of the subdomains • Suggestions for missing subdomainsand clarification of subdomains • (e.g., legal compliance at the program or institutional level) Center on Postsecondary Education and Disability, ; AHEAD 2013

15. Inclusion and Exclusion Criteria • Inclusion criteria: 1. The article is about Postsecondary Education for Students with Disabilities (broadly considered to include faculty, disability services, etc.) 2. The article is about one of the following topics/populations: a. Programs for accepted students into degree granting programs at a 2 or 4 year college or university b. Programs, services, or experiences of matriculated students c. Articles about the experiences of students with disabilities who have dropped out of degree granting programs at a 2 or 4 year college or university d. Articles about the experiences of students with disabilities who are graduates of degree granting programs at a 2 or 4 year college or university • Exclusion criteria 1. Articles that are primarily about secondary students in transition or transition aged programs. Center on Postsecondary Education and Disability, ; AHEAD 2013

16. Domain Descriptions Center on Postsecondary Education and Disability, ; AHEAD 2013

17. Sub Domains

18. Sub Domains

19. Sub Domains

20. Sub Domains

21. Method Continued literature review and collection of articles Search Terms Included: • Academic Accommodation • Accommodation • ADD • ADHD • Blind • College • College Admission • College Student • Deaf • Disabilities • Disability • Disabled • Dyslexia • Dyslexic • Handicap • Handicapped • Hearing Impairment • Postsecondary Education • Mental Illness • Mobility Impairment • Postsecondary Education • Student Affairs • Student Personnel • Student Services • University • University Student • Visual Impairment

22. Method • 1,210 articles identified by searches of multiple data bases (e.g., Academic Search Premier, EBSCO, PSYCinfo) • Published between 1955 and 2012 • Articles grouped into domains, reliability measured • Coding resulted in some articles shifting domains • Will focus on student-level, program or institution-level, faculty/non-disability-level studies today • Future steps will focus on coding construct development studies Center on Postsecondary Education and Disability, ; AHEAD 2013

23. Method An electronic database was developed that included the reference citation for each article and unique alpha-numeric codes (e.g., 201225Shaw4). The codes were designed to be entered into the electronic coding instrument, so that coding could be linked back to the reference citation. Each article randomly assigned to two coders Reliability determined for each domain Discrepancies discussed Center on Postsecondary Education and Disability, ; AHEAD 2013

24. Instrument • An electronic coding instrument was designed and refined with two pilots, multiple coders. • The instrument allowed for the researchers to code: • Did the article meet inclusion criteria? • Did the article present original data? • If not research based, what type? (e.g., lit review, legal analysis) • If research based, what type? (with multiple layers) • What was the setting? (US, Canada, international, 2- or 4-year) • Sample information? (numbers, gender, disability, race, etc..) • Domain and sub-domain • Across coding sheet, 148 choices were possible • To achieve agreement, coders selections must be exact Center on Postsecondary Education and Disability, ; AHEAD 2013

25. Inter-Rater Reliability Frequency and Reliability by Domain • For today’s presentation, three subsets of articles (Domains 1, 2, and 3) were analyzed. • Each article coded twice to check for inter-rater reliability. • Discrepancies discussed and reconciled Center on Postsecondary Education and Disability, ; AHEAD 2013

26. Articles by Domain (Initial Sort) Center on Postsecondary Education and Disability, ; AHEAD 2013

27. Frequency of Articles by Domain Over Time Center on Postsecondary Education and Disability, ; AHEAD 2013

28. Journals with the Highest Frequency of Articles About Higher Education and Disability Unique Journals: 305 Center on Postsecondary Education and Disability, ; AHEAD 2013

29. Journals with the Highest Frequency of Student-Level Articles Unique Journals: 172 Center on Postsecondary Education and Disability, ; AHEAD 2013

30. Journals with the Highest Frequency of Program/Institutional-Level Articles Unique Journals: 109 Center on Postsecondary Education and Disability, ; AHEAD 2013

31. Journals with the Highest Frequency of Faculty/Non-Disability Staff-Level Articles Unique Journals: 71 Center on Postsecondary Education and Disability, ; AHEAD 2013

32. Journals with the Highest Frequency of Construct Development-Level Articles Unique Journals: 70 Center on Postsecondary Education and Disability, ; AHEAD 2013

33. Proportion of Data-Based vs. Non-Data-Based Articles for Student-Level Studies Center on Postsecondary Education and Disability, ; AHEAD 2013

34. Frequency of Data-Based vs. Non-Data-Based Student-Level Studies Over Time Center on Postsecondary Education and Disability, ; AHEAD 2013

35. Proportion of Student-Level Studies by Research Methodology Center on Postsecondary Education and Disability, ; AHEAD 2013

36. Proportion of Student-Level Studies With and Without Control/Comparison Groups Center on Postsecondary Education and Disability, ; AHEAD 2013

37. Proportion of Student-Level Studies Containing a Control/Comparison Group by Experimental vs. Quasi-Experimental Methodology Center on Postsecondary Education and Disability, ; AHEAD 2013

38. Proportion of Student-Level Studies by Location Note: 17 studies include multiple locations Center on Postsecondary Education and Disability, ; AHEAD 2013

39. Location of Student-Level Studies Over Time Center on Postsecondary Education and Disability, ; AHEAD 2013

40. Proportion of Student-Level Studies Including Clear Sample Size Data Center on Postsecondary Education and Disability, ; AHEAD 2013

41. Proportion of Student-Level Studies Including Data on the Race/Ethnicity of Participants Mean = 6,136 SD = 91,021 Min = 1 Max = 1,502,658 Center on Postsecondary Education and Disability, ; AHEAD 2013

42. Proportion of Student-Level Studies Including Disability-Related Demographic Information Center on Postsecondary Education and Disability, ; AHEAD 2013

43. Proportion of Student-Level Articles Including Data About the Gender of Participants Center on Postsecondary Education and Disability, ; AHEAD 2013

44. Proportion of Student-Level Articles Including Data on the Class Standing of Participants Center on Postsecondary Education and Disability, ; AHEAD 2013

45. Proportion of Student-Level Articles Including Data About the Non-Student Participants Center on Postsecondary Education and Disability, ; AHEAD 2013

46. Thirteen Subdomains of Student-Level Studies and Their Frequencies(Articles could be coded as multiple subdomains) Center on Postsecondary Education and Disability, ; AHEAD 2013

47. Proportion of Data-Based vs. Non-Data-Based Program/Institution-Level Studies Center on Postsecondary Education and Disability, ; AHEAD 2013

48. Frequency of Data-Based vs. Non-Data-Based Program/Institution-Level Studies Over Time Center on Postsecondary Education and Disability, ; AHEAD 2013

49. Proportion of Program/Institution-Level Studies by Research Methodology Center on Postsecondary Education and Disability, ; AHEAD 2013

50. Proportion of Program/Institution-Level Studies With and Without Control/Comparison Groups Center on Postsecondary Education and Disability, ; AHEAD 2013