1 / 51

Online Professional Development for Transition: Evaluating the Effectiveness

This study evaluates the effectiveness of online professional development programs for transition in special education. It examines the impact of non-instructor led online training modules on practitioners' knowledge and implementation of best practices in transition planning.

hectord
Download Presentation

Online Professional Development for Transition: Evaluating the Effectiveness

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Online Professional Development for Transition:Evaluating the Effectiveness Mary E. Morningstar, PhD – mmorningstar@ku.edu Dana L. Lattin, MSEd – dlattin@ku.edu Amy Gaumer Erickson, PhD – aerickson@ku.edu Ryan Kellems, MEd – rkellems@ku.edu October 18, 2007 DCDT International Conference – Orlando, FL

  2. 1980’s • National study of high school programs, 1985-88 • 1985 Department of Defense Dependents’ Schools Summer Institute on career/ vocational • Summer transition workshops late 1980s • Secondary Special Education/ Transition personnel preparation grant, 1989-91 • 1990’s • Kansas Systems Change Project - Statewide Transition Training 1993-1998 • Kansas Transition Network –contract with KSDE 1998-2000 • KU Secondary/Transition Personnel Preparation Program 1994-1999 • KU Secondary/Transition Outreach project 1997-2000. • 2000-2004 • Interprofessional/Transition Personnel Preparation, 2000-03 • National Transition Outreach Project (National Transition Coalition), 2001-2004 • National Online Specialization in Transition Project, 2001-2004 • 2004 - Present • KU TransCert: Online Transition Graduate Certificate Program 2004-2008 • Evaluating the Effectiveness of Online Professional Development for Transition 2004-2007 • KU Secondary Transition Leadership Program 2005-2009 Late 1960’s Department of Special Education commitment to secondary special education and career development starts in late 1960s • 1970’s • Federal grant for secondary teacher education personnel preparation model, 1972-76 • Federal RETOOL project for higher education faculty on career education for students with disabilities, 1975-79 First, a little background…. Late 1960’s 2000-2004 1990’s 2004-? 1970’s 1980’s

  3. Non-Instructor Led Online Professional Development • User friendly and interactive information • Making sense of complex information • Applying information • Connect & network Online Training Online Resources Searchable Databases Models of Success

  4. Currently Available Working with Families Transition Assessment: The Big Picture Best Practices in Transition and IDEA 2004 Cultural Diversity and Transition Students with ED/BD and Employment In Design Phase Aligning Transition IEPs with Standards Self-Determination SBR & Transition Students with Significant Disabilities & PBS Online Training Modules

  5. Online Module Framework • Learning Objectives • Case-Based Examples & Activities • Conceptual Models/Unifying Frameworks • Important Point • Now It’s Your Turn • More Information & My Library • Interactivity: Click & Compare; True/False; Games; Audio/video • Session Summary General Content • Pre/Post Test • Learning activities • Reflective questions • Impact on teaching • Barriers to implementation • Reflection of general learning content User Portfolio

  6. Learning Objectives

  7. Case-based Examples & Activities

  8. Conceptual Models & Unifying Frameworks

  9. Important Point

  10. Now It’s Your Turn

  11. More Information & • My Library

  12. True/False or Myth/Reality

  13. Click & Compare

  14. Games

  15. Audio/Video

  16. Session Summary

  17. Demographic Data of All Users • 2405 practitioners • 68% educators (sped, gen ed, transition, related services, administrators) ~ 50% serving secondary-aged students • 18% College/university students • Types of disability groups served • 30% LD; 18% multiple groups; 5% MR; 4% ED/BD; 2% autism; 5% SMD; 25% Programmatic • 57% regular schools; 7% special schools; 6.5% community agencies • Level of Education • 44% Masters/specialist; 44% bachelors; 4% doctorates; 3% associates March, 2007

  18. Educators • Certification Status • 43% fully certified for current teaching assignment; 3% emergency certif.; 7% certif. in field other than teaching; 8% provisional; 17% NA • Years Teaching • 36% 10+; 8% 7-9yrs.; 15% 4-6yrs; 20% 1-3yrs; 9% student teaching; 11% not teaching • Transition Training & Information Sources • 46% inservice training; 26% transition course; 6% 2+ courses; 9% infused content; 17% conferences; 17% newsletters/online; 43% on-the-job training March, 2007

  19. Satisfaction with Training Modules March, 2007 Best Practices Families Assessment

  20. PRO’s Ease of use Graphics Navigation Flexibility Portfolio Information was current, useful & to the point Examples and interactive case studies CON’s Pre/Post Test Limited information Portfolio Printer-friendly version More reflective practice and application Links not working Qualitative Data Results: Online Training Modules

  21. N= 282; Post-test (M=11.51, SD 2.77) significantly greater than pre-test (M=13.90, SD 2.86); t(164) = -10.85 p<.00, d=.84 Evaluation of Online Modules: Knowledge GainRepeated measures design - Paired Sample t-test Working with Families during Transition Best Practices in Transition Planning N= 282; Post-test (M=11.51, SD 2.77) significantly greater than pre-test (M=13.90, SD 2.86); t(164) = -10.85 p<.00, d=.54 Transition Assessment N= 282; Post-test (M=11.51, SD 2.77) significantly greater than pre-test (M=13.90, SD 2.86); t(124) = -14.318 p<.00, d=1.28

  22. Random Sample-Control Group Study: 2 Online Modules Kim (2006). Kim & Morningstar (2006). • Working with Families & Working with CLD Families and Youth Modules • Control group (43; no intervention) & Intervention group (43; A-B; B-A) • Content Knowledge test (pre/post) • Competencies (level of preparation) + Attitudes (level of importance) Survey (.79-.94 reliability)

  23. Results: Knowledge & Competency Gains • 28/42 of control group completed pre & post tests • 25/42 of intervention group completed study • Comparison of groups • Chi Square tests = no significant difference among 2 groups for years teaching, types of students, training, certification, sources of training in transition • Content Knowledge: Experimental group showed significant changes in knowledge & control group showed no change • Perceived competencies: WWF: significant difference; WCLD: control & experimental groups showed no change • Perceived importance (Attitudes): no change pre/post both; WWF’s between groups ANCOVA on posttest was significant for control

  24. Online Professional Development Evaluation Recruitment & Sampling Control Group 15-20 practitioners No training No online CoP Training + CoP 15-20 practitioners 2 training modules Online CoP Training Only 15-20 practitioners 2 training modules • Comparisons • Knowledge • Competencies & skills • Frequency of implementation • Mixed Measures • Pre-post test • Competencies surveys • Performance-based content analysis • Qualitative data Access to ALL online training modules upon completion of the study Access to ALL online training upon completion of the study

  25. Recruitment of Participants • Initial recruitment in 3 states (CO, ID, AZ) • Contacted SDE staff at other states to find other participants • NEW box on TC website – available to anyone • Online agreement to participate • Blocked from all training modules until research started • Needed 150 participants with min. 90 completing research – recruited 156

  26. Procedures • Communication with participants was through email • Participants in all groups (C, T, TCoP) were asked to begin research at the same time. • Participants received email reminders and prompts to complete the research and were given approximate times for how long it would take to complete each step. • All participants received incentives for completion of research (honorarium or graduate credit) and for completing various steps of the research within specific timelines (jump drives, bonuses, transition books) • Phone support was provided to those participants needing intensive technical support using the website.

  27. Participant Demographics • 81 (86%) Educators 8 (9%) School Related Services Providers • 5 (5%) Other • Community Service providers, Family members, Administrators • 88 (94%) Serve secondary aged students • Types of disability groups served • Across Multiple Disabilities (55%); High Incidence (33%) Low Incidence (7%); None (4%) • Level of Education • 64% Masters/specialist; 30% bachelors; 4% doctorate; 1% associates; 1% HS Diploma 94 Completed Research (C=35; T=30; TCoP=29)

  28. Educators • Educators Only - Certification Status • 75% fully certified for current teaching assignment; 6% provisional/emergency certif.; 3% certif. in field other than teaching; 16% NA • Educators Only - Years Teaching • 64% 10+; 6% 7-9yrs.; 16% 4-6yrs; 13% 1-3yrs; 1% not teaching • Educators Only - Transition Training & Information Sources • 75% inservice training; 72% conferences; 64% newsletters/articles/books; 62% on-the-job training; 48% online; 20% professional associations; 9% 1+ transition course; 13% infused content

  29. Measures • STTS (pre/post) • Best Practices pre/post tests • Transition Assessment pre/post tests • Satisfaction Surveys for each module • Performance-based Assessment of 3 specific activities within each module (total of 6 activities across both modules)

  30. DOMAINS Instructional Planning Curriculum and Instruction Transition Planning Assessment Collaboration Additional Competencies RELIABILITY TESTS: 557 teachers nationally Alpha scores: .96 Preparation subscale .94 Frequency subscale

  31. Control Group Pre STTS 1 Best Practices Pre test Transition Assessment Pre test Best Practices Post test 2 Transition Assessment Post test Post STTS

  32. Training Group Pre STTS Transition Assessment 1. Pre test 2. Complete Module and all Activities 3. Post test 4. Satisfaction Survey Best Practices 1. Pre test 2. Complete Module and all Activities 3. Post test 4. Satisfaction Survey Post STTS

  33. Training + CoP Group Pre STTS Transition Assessment 1. Pre test 2. Complete Module and all Activities 3. Post test 4. Satisfaction Survey Best Practices 1. Pre test 2. Complete Module and all Activities 3. Post test 4. Satisfaction Survey Participate in Community of Practice Post STTS

  34. Performance-based Assessment • Developed, validated, user-tested, modified rubrics for 6 activities • 3 reviewers met to reach consensus on scoring methods • Compared scores on sample • Scored all to determine Inter-rater Reliability • Range 75%-90% • Average 86% • Mean Scores • Comparisons between T & T+CoP groups

  35. Results: Knowledge Gained • Measure: pre/post tests for modules • Best Practice Module • Statistically significant gain in knowledge for both training groups; high effect size • Statistically significant difference in knowledge on post-tests between control and training groups, accounting for 35% of the variance in scores • Assessment Module • Statistically significant gain in knowledge for both training groups; high effect size • Statistically significant difference in knowledge on post-tests between control and training groups, accounting for 8% of the variance in scores

  36. Results: Satisfaction with Training

  37. Results: Perceived Skill & Frequency • Measure: STTS pre/post tests • Statistically significant increase in perceived skills for training groups • CoP group showed more realistic perceptions of their skills and frequency on the post-test

More Related