Should you Conduct a Constituent Survey… If so… how should you do it… and how can you Maximize the Process?

1. Should you Conduct a Constituent Survey…If so…how should you do it…and how can you Maximize the Process? Jeffrey L. Mitchell Headmaster Tuscaloosa Academy

2. Background • Experience as an independent school educator, implementing a data-driven decision-making process, including utilizing constituent surveys • Part of my larger goal to be a data driven school • NAIS StatsOnline • Senior Interview • Marketing data e.g., your “admissions funnel” stats • Even budgeting • Experience as a consultant for schools implementing constituent surveys • Background in stats and methodology • I like it…really! • Contact: jmitchell@tuscaloosaacademy.org

3. Survey on Surveys… • How many have conducted a constituent survey? • What kind of surveys? • What was successful? • What were the challenges? • What (specifically) do you want to know? • Distribute surveys…

4. Outcomes • TPWBAT… • List who you can survey. • Explain why you survey certain constituents. • Provide some details regarding the construction and implementation of constituent surveys. • Make a considered decision regarding the implementation of a survey in your school.

5. Why Survey? (From the Research Methodology class…that you, up to now, have tried to suppress) • To systematically collect data for the purposes of understanding an issue or entity. • Quantify and/or qualify your understanding of constituents: • Satisfaction levels • Values • Thoughts on the present • Thoughts on the future

6. Why Survey? (At the Strategic Level – [for your Board]) • Say… It’s a report card on past performance… political and pragmatic usefulness e.g., fresh start, contract renewal • Say… It’s a commitment to continuous improvement. At no time should an organization, like a school, stop evolving…so you might as well drive this train. • Similarly, say… It’s a way to gather data regarding contemplated initiatives… e.g., homework at PT, dress code at TA • Say… It’s a baseline against which to measure future progress…e.g., whether you’re in your first year as Head, or some kind of milestone change has occurred e.g., TA just implemented a new schedule, TA is considering single-sex classrooms in the MS • Say… It’s a way to help settle an issue… Take your mandate from the market. Heated debates are often brought to closure when you can drop a survey report on the boardroom table and declare, "I hear what you're saying, but that's not what our constituents are telling us.“ • Say… It’s clever marketing… There isn't a market today without exposure to change. A regular cycle of constituent surveys gives you the timely intellectual capital with which to stay on the leading edge of this change. • Say… It’s a good business decision… Efficiently allocate scarce and diminishing resources…it might be an (unfortunately) useful tool if downsizing is required…

7. Why Survey? (At the everyday constituent level) • No matter how many anecdotes you’ve collected, anecdotes are still not data… • Give voice to the silent majority…providing a voice for the 90%... • You win by knowing, good or bad • Segmenting response groups…which segment of your larger group thinks a certain way…e.g., dress code • PR value of asking…often the value for constituents is simply in asking them…

8. Why Survey? (From a marketing perspective – if you need to sell this…with a little panache) • Flag • Screwdriver • Thud

9. Who can you Survey (and why) • Parents, Students, Faculty - assess satisfaction/agreement with all aspects of school life • Young Alumni • assess satisfaction/agreement with all aspects of school life & perhaps best assessment of the quality of your academic program • Advancement survey of alumni • Assess satisfaction/agreement and…could be a powerful fundraising tool • Parents of Graduates • assess satisfaction with all aspects of school life & perhaps the most honest assessment of the school as a whole • Board Self-Assessment • Head of School Assessment

10. The Process Summarized

11. Maximizing Response Rates • Vary by constituency but typically much better response rates than most survey processes • What’s typical… • Parents…at least 60% • Students…almost 100% • Faculty/staff…almost 100% • Alumni…hopefully 25% • Distribution methods: • Paper (mailing , distribute by hand, host survey completion sessions) • Electronic • Combination or the two • Multiple exposures/reminders are important regardless

12. What are your Options? • On your own • A web-based software program designed for the creation of surveys e.g., Survey Monkey • A web-based software program designed for the creation of surveys for independent schools e.g., NAIS Survey Builder • Firm that specializes in independent school surveys • Marketing Firm (many possibilities)

13. Golden Rules that Guidethe Selection Decision • Generally, the more you pay the less you have to do. • Garbage in…garbage out… • Don’t do it if you’re not prepared to share the findings. • Within reason • Story of the “dissatisfied” Associate Head

14. Help Wanted • For this upcoming section on the benefits and drawbacks please think about your experience with surveys and fill in any blanks…

15. Option 1: Flying Solo Benefits Drawbacks You need someone knowledgeable in survey construction Possible “researcher” bias introduced into the process “Person hours” cost could be high Stand alone process…no comparisons to other data • You can construct a completely personalized survey… • You have the power of complete ownership • Cost is minimal i.e., paper

16. Option 2: Online Survey Servicee.g., Survey Monkey(www.surveymonkey.com) Benefits Drawbacks Stand alone process…no benchmarks It’s still up to you to create a good survey (although they do have some basic templates) Researcher bias is still a potential problem • Ease of use • A web-based tool specifically developed for this purpose • Three steps • Design • Collect • Analyze • Powerful & professional • Costs • Free for the basic version • $200 / year for the pro version

17. Option 3: NAIS Survey Builder Seehttp://www.nais.org/ Benefits Drawbacks Ability to benchmark is developing Cookie cutter template without much flexibility Not particularly comprehensive • Similar to Survey Monkey …has features for creating surveys…albeit less powerful • Cost – varies depending on particular survey • e.g., parent survey - $1000 for non-members, $350 for members • Provides standard template

18. Option 4: Specialists in Independent School Surveys (Lookout Management Inc.http://lookoutmanagement.com/index.html) Benefits Drawbacks Expense $6,000 for student and faculty $12,000 for parent and young alumni $25,000 for advancement/alumni • Comprehensive template with as much flexibility as needed • The school is still very much a part of the process • Depth and breadth of analysis – they focus specifically on independent school surveys; thus, cut through the morass of details and get to the essential points • Comparison of your results with a rich data base • Independent presentation (no bias)

19. A (small) example of what you would get with this approach: ??????? vs. “Best Scores” Best Other Weighted Ave.Score Overall Satisfaction 4.2 4.3 Class Size 4.6 4.6 Challenge of Academic Program 4.4 4.6 Quality of Education 4.4 4.5 Respect for Cultural Differences 4.3 4.4 Helpfulness of Support Staff 4.3 4.5 Fine Arts Program 4.2 4.1 Report Card Format 4.2 4.4 Opportunities to Volunteer 4.2 4.0 Safety of Students 4.2 4.3 Interaction between faculty and students 4.2 4.4 Performing Arts Program 4.2 4.4 Report Card Timing and Frequency 4.2 4.3 Sense of Community for Students 4.1 4.4 Volunteer Experiences 4.1 4.0 Academic Facilities 4.1 4.7 Administrative Leadership 4.0 4.2 Teaching Skills 4.0 4.3 Availability of Extra Help 4.0 4.2 Behavior of Students 4.0 4.0 Leadership Opportunities of Students 4.0 4.1

20. Option 5: A “Downtown” Marketing Firm Benefits Drawbacks Cost >= $25K for one survey No benchmark data • They will take care of everything…from interviews, to focus groups, to creating questions… • An independent process • High quality/professional product

21. Should you Conduct a Constituent Survey? • Since it is now possible to do so within any school’s budget...I say yes!

22. Regardless which one you do…you must follow-up • Develop a plan for disseminating the results • It is strongly suggested you follow this pecking order • Board • Senior Admin • Faculty • Parents at large • Students (if developmentally appropriate) • Alumni • Present in multiple formats e.g., paper summary, in-person presentation, online

23. Developing a Survey Process • Overall Satisfaction Ratings across constituents You get longitudinal and latitudinal benefits of conducting surveys over time and across constituent groups Interpretation?

25. Suggested Timeline for a Survey Program

26. The Questions? • What makes a good question? • Returning once more to the Research Methodology class…what are the two master concepts? • Clearly stated • Meaningful to your school • Put some meat and potatoes into your survey • Don’t be afraid to ask controversial questions, especially if change is possible • Where do you get them? • Start with a template • Finish with the issues specific to your school

27. Putting Surveys to Work: TA’s Plan for Accreditation • Four constituent surveys • Parent • Faculty/staff • Student • Alumni • Four committees created to tackle four strategic goal areas • Curriculum and Programs • Financial sustainability • Internal and external relations • Facilities • Each constituent survey has questions organized by the strategic goal area (plus demographics and the example questions provided by SAIS) • We will compile the information collected from the surveys to augment the information gathered in the committees.

28. TA’s Surveys • Comments and Questions • Going beyond a typical survey • Note “forced choice” question • Note substantive feedback sought in terms of what folks want to do e.g., what faculty/staff might suggest for extracurricular activities/clubs

29. Analyzing the Data • Wealth of information available! • Typically descriptive data that summarizes the responses for each question • Cross tabulations with the master question (Overall Satisfaction) are very common and useful e.g., overall satisfaction by any segment of your population • The potential, however, to answer very interesting questions often goes untapped… • Ask how two (or more) variables are related (correlation) e.g., I wonder how satisfaction with the academic, athletic and fine arts programs correlate… • Compare two (or more) variables (t-test, ANOVA) e.g., Is there a statistically significant difference in overall satisfaction among LS, MS and US faculty? • Predict a future outcome (regression analysis e.g., What variables contribute the most to Overall Satisfaction? • Uncover constructs that underlie the data • Analyze components of a Mission Statement e.g., academic excellence, opportunity, individual attention, diversity, service learning

30. Outcomes Revisited • Who can you survey? • Why do you survey certain constituents? • What are some details regarding the construction and implementation of constituent surveys? • Make a considered decision regarding the implementation of a survey in your school

31. Concluding Thoughts & Questions • Make surveys a part of your administrative routine • I would be willing to offer advice or help to any school that wishes to conduct a survey