A Delphi Study of Critical School Counseling Research Questions

1. A Delphi Study of Critical School Counseling Research Questions ASCA-ACES Research Summit St. Louis, Missouri June 28, 2003

2. National Center for School Counseling Outcome Research John Carey Carey Dimmitt http://www.umass.edu/schoolcounseling/

3. Center Mission • The National Center for School Counseling Outcome Research is dedicated to improving the practice of School Counseling by developing the research base that is necessary for responsible and effective practice.

4. Center Mission • The Center provides national leadership in the measurement and evaluation of the outcomes of school counseling interventions and programs. The Center helps K-12 leaders and practitioners use research findings and methods to improve school counseling practice by enabling them to set standards, establish benchmarks, implement evidence-based interventions, measure student learning outcomes, and evaluate program effectiveness.

5. Why the Need for Empirical Studies of School Counseling Outcomes? • Need to Identify Best Practices • Need to Identify Evidence-based Interventions (NCLB) • Need to be Accountable • Need to Demonstrate Impact on Student Achievement • To Develop as a Profession

6. Leadership Needed • To Evaluate Existing Evidence • To Identify Critical Questions • To Identify the Most Important Outcomes • To Identify or Develop Good Outcome Measures • To Develop New Research Approaches • To Coordinate Multi-Site Efforts • To Communicate the Implications of Research Findings to Practitioners

7. Center ActivitiesNational Focus • Research Brief Listserve, 10,000 subscribers • Summer Institute, “Using Data to Help All Children Succeed” (July 14 thru 18) • Review of Outcome Studies and Measures • Delphi Study of Critical School Counseling Research Questions

8. Center ActivitiesRegional Focus • Urban Guidance Directors Leadership Group • Task Force on a Massachusetts School Counseling Model • Program Evaluation and Leadership Consultation • Curriculum Development Consultation • Research Partnerships with School Districts

9. Center New Initiatives • National Board for Evidence-Based Practice in School Counseling • Establish Criteria • Select Critical Outcomes • Evaluate Evidence • Identify Needed Research • Make Recommendations for Counselor Education

10. Center New Initiatives • Electronic Journal “School Counseling Outcomes” • Promote empirical identification of best practices • Promote the development of effective research approaches • Promote the development of quality outcome measures • Disseminate findings broadly to multiple constituencies • Establish credibility for school counseling practice

11. Delphi Study of Critical School Counseling Research Questions • Delphi Research Uses Expert Opinion to • Forecast Future Trends and Directions • Identify Critical Issues • Achieve Consensus and/or Map Points of Disagreement • First Phase in Evidence-Based Professional Boards • Delphi Research is Especially Appropriate for Identifying Approaches to Complex Problems in Areas with Multiple Constituencies • Delphi Research Has Been Used Successfully by National Agencies to Develop Priorities for Research Funding

12. Delphi Study of Critical School Counseling Research Questions • Panel Selection • Phase I: Identify the Purposes of School Counseling Research • Phase II: Identify Specific Research Questions For Each Purpose • Phase III: Prioritize Research Questions and Identify Criteria for Judging Importance

13. Judy Bowers Carol Dahir Teesue Fields Norman Gysbers Trish Hatch Richard Hayes Peggy Hines Reese House Curly Johnson Dawn Kay Judy Madden Brenda Melton Linda Miller Rachelle Perusse Lonnie Rowell Russell Sabella Susan Sears Toni Tollerud Jose Villaba Susan Whiston Richard Wong Expert Panel

14. Purposes of School Counseling Research: Phase I • Delphi Study Question 1: Given the history and current political context of school counseling, what ends should research achieve?

15. Purposes of School Counseling Research: Phase I Responses • To identify evidence-based interventions and best practices in terms of school counseling activities, interventions, programs, and models. “Research can provide practitioners with information on which interventions really help students.”

16. Purposes of School Counseling Research: Phase I Responses • To document effectiveness in order to be accountable to external constituencies for school counseling activities, interventions, programs, and models. Research needs “to show the results of a comprehensive school counseling program and demonstrate to stakeholders the effectiveness of the program.”

17. Purposes of School Counseling Research: Phase I Responses • To understand how to effect change in school counselor practice, school counseling programs, and school counselor roles in schools. “More research is needed to discover what factors are contributing to or influencing the changes in school counselors’ beliefs and behaviors regarding the performance of counseling and non-counseling activities.”

18. Purposes of School Counseling Research: Phase I Responses • To identify the most effective approaches to educating and supervising school counselors in training. “If best practices could be determined, they might help provide some kind of standard by which all school counselors could be prepared consistently.”

19. Specific Research Questions: Phase II Questions • “1. Do your ideas about the purposes of school counseling research fit into one or more of these categories? • 2. Would you add to or redefine the definitions of any of these purposes? • 3. Please identify what you consider to be an important, specific research question for each purpose, if possible.”

20. Specific Research Questions: Phase II Responses • Responses were collapsed into 42 distinct Research Questions.

21. Specific Research Questions: Phase III Ratings • Participants were asked to rate the importance of each Research Question • 1=Not Important • 2=Important • 3=Very Important • 4=Critically Important • Participants were asked to identify criteria used in judging importance • Participants were asked to supply additional questions

22. Survey • Please Complete • We would like your completed survey

23. Phase III Results • 23. Which specific School Counseling interventions result in the greatest gains in student’s academic development and achievement? 3.8 • 27. What specific School Counseling interventions are most effective at preventing school dropout? 3.7 • 30. What is the impact of enhancing School Counselors’ ability to use data in planning and accountability on student outcomes? 3.7 • 26. What specific school counseling interventions are most successful at preventing school failure? 3.6

24. Phase III Results • 37. What are the most effective approaches to changing school leaders’ opinions about the value and legitimacy of school counseling? 3.6 • 2. What factors contribute to the success of a school counseling program? 3.6 • 8. What’s the impact of implementing a results-based school counseling program on students’ academic development and achievement? 3.6 • 4. What’s the impact of implementing a Comprehensive Developmental School Counseling Program on students’ academic development and achievement? 3.6 • 36. What are the most effective approaches to changing school leaders’ opinions about school counselor role and function? 3.5

25. Phase III Results • 24. Which specific School Counseling interventions result in the greatest gains in students’ career development? 3.5 • 25. Which specific School Counseling interventions result in the greatest gains in students’ personal/social development? 3.5 • 6. What is the impact of implementing a Comprehensive Developmental School Counseling Program on students’ personal/social development? 3.5 • 9. What is the impact of implementing a Results-based School Counseling Program on students’ career development? 3.5 • 10. What is the impact of implementing a Results-based School Counseling Program on students’ personal/social development? 3.5

26. Phase III Results • 12. What is the impact of implementing an ASCA National Model School Counseling Program on students’ academic development and achievement? 3.5 • 35. What is the impact of enhancing school counselors’ abilities to use data in planning and accountability on school leaders’ perceptions of the value and legitimacy of school counseling? 3.5 • 1. What is a successful School Counseling program? 3.4 • 11. What is the impact of implementing a Results-based School Counseling Program on school leaders’ perceptions of the value and legitimacy of school counseling? 3.4 • 3. What are the factors that contribute to the efficacy of a school counseling program’s implementation? 3.4

27. Phase III Results • 13. What is the impact of implementing an ASCA National Model School Counseling Program on students’ career development? 3.4 • 28. Under what circumstances are whole classroom, small group, and individual counseling maximally effective? 3.4 • 5. What is the impact of implementing a Comprehensive Developmental School Counseling Program on students’ career development? 3.3 • 15. What is the impact of implementing an ASCA National Model School Counseling Program on school leaders’ perceptions of the value and legitimacy of school counseling? 3.3 • 16. What are the best ways to measure the extent to which a school counseling program model is fully implemented? 3.3

28. Phase III Results • 19. What is the impact of reducing student/counselor ratios on program outcomes? 3.3 • 31. What are the most effective approaches to enhancing the leadership skills of School Counselors? 3.3 • 41. What are the most effective professional development approaches to enhance School Counselors’ competence? 3.3 • 7. What is the impact of implementing a Comprehensive Developmental School Counseling Program on school leaders’ perceptions of the value and legitimacy of school counseling? 3.2 • 14. What is the impact of implementing an ASCA National Model School Counseling Program on students’ personal/social development? 3.2

29. Phase III Results • 22. What research and evaluation skills are necessary for effective School Counseling practice? 3.2 • 38. What is the impact of collaboration between guidance leaders and school counselor educators on the quality of school counseling training and the competence of school counseling trainees? 3.2 • 39. What is the impact of aligning a school counselor education program with a school reform agenda on the quality of school counseling training and the competence of school counseling trainees? 3.2 • 40. What is the impact of aligning a school counselor education program with the ASCA National Model on the quality of school counseling training and the competence of school counseling trainees? 3.1

30. Phase III Results • 17. What is the impact of aligning a school counseling program with the school-reform agenda on students’ academic development and achievement? 3.0 • 33. To what extent do School Counselors trained in leadership and systemic change models attempt to effect change in their schools? 3.0 • 42. What is the impact of school counselor educators’ work experience in schools on the orientation and outcomes of counselor education programs? 3.0 • 18. What is the impact of implementing the use of specific competency-based student learning objectives in a School Counseling Program on student outcomes? 2.9 • 32. What are the most effective approaches to developing and enhancing the professional identity of school counselors? 2.8

31. Phase III Results • 34. What are the most effective approaches to enhancing the program organization and management skills of school counselors? 2.7 • 21. Do school counselors who use research to guide practice have a greater impact on student outcomes? 2.6 • 29. What is the impact of peer helper programs on students’ attendance, grades and school behavior? 2.3 • 20. What is the impact of different methods of assigning students to counselors on different outcomes? 2.1

32. Criteria for Judging Importance • Research that demonstrates impact on outcomes is needed because school counseling is “at risk” • Research that would significantly add to our understanding of “best practices” • Research that is intrinsically interesting “to me” • Research that advances the efficacy and effectiveness of school counselor training • Research that helps the profession meet the challenges associated with NCLB and implementing Transforming School Counseling Initiative and ASCA National Model reforms.

33. Additional Questions • What is the technological literacy of school counselors? How do school counselors implement technology? How does implementing technology impact outcomes? • What is the process through which school counselors make decisions and how can decision-making be improved? • What is the impact of school counselor self-reflection on efficacy and effectiveness?

34. National Center for School Counseling Outcome Research Thank You http://www.umass.edu/schoolcounseling/