MTSU Web site Design Research Report Summer 2007

1. MTSU Web site Design Research ReportSummer 2007 Timothy R. Graeff, Ph.D. Scott Seipel, Ph.D.

2. Contents • Purpose of this Research, 3 • The Three Web site Designs, 4 • Survey Methods, 5 • The Sample: Prospective Students, 8 • Results: Prospective Students, 16 • The Sample: Current MTSU Students, 28 • Results: Current MTSU Students, 34 • Further Analysis of the Web site Evaluations: Cross Tabs, 46 • General Conclusions, 54 • Design Conclusion, 55 • Author Contact Information, 56

3. Purpose of this Research In an effort to market more effectively to prospective students, MTSU has decided to change the design of their Web site to be more attractive an appealing to prospective students. The overall purpose of this research is to examine the relative attractiveness of three potential new designs for the MTSU Web site. In addition, this research examines: • The relative importance of Web sites in a students’ choice of university to attend. • The relative importance of various Web site design elements in determining a student’s desire to attend a university. • The reasons why students visit university Web sites.

4. The Three Web site Designs Design 1 Design 2 Design 3

5. Survey Methods

6. Survey Methods • An online survey was developed and administered during June 2007. • An email invitation was sent to prospective high school students and current MTSU students asking them to complete the survey. • In exchange for completing the surveys, respondents were told that their names would be entered into a drawing to win one of a number of prizes: • A staff level (white) parking pass • One of five iPods • One $350 Phillips Bookstore book scholarship • One of two $100 Phillips Bookstore book scholarships • One of several $10 iTunes gift cards • All respondents answered all of the survey questions and evaluated all three Web site designs. The order of presentation of the Web sites was counter balanced to avoid any effects due to order of presentation.

7. Introductory email Letter Sent to Prospective and Current MTSU Students We know you consider many factors when choosing the university that’s right for you. Academic programs, size, location, cost – all these come into play when determining where you’ll spend your college years. But we also know that other subtle details help make that determination – the school’s reputation, its admission materials, or its Web site.  And that’s where we need your help. At Middle Tennessee State University, we’re in the process of redesigning our Web site. Since you’ve shown interest in MTSU, we would like your opinion of the possible new sites. Just for filling out our short survey, you might win one of these prizes • a staff-level parking pass (which allows you to park closer to your classes and with more spaces available to you) • one of five iPods • one $250 Phillips Bookstore scholarship • one of two $100 Phillips Bookstore scholarship • one of several $10 iTunes gift cards Please take a few moments to complete our survey and enter for your chance to win one of the prizes listed above. And while you’re at it, visit us at www.mtsu.edu. Take a look at what we have to offer. You’ll be glad you did. Regards, Warmly Sincerely yours, Go Big Blue, Thank you, Dr. Robert K. Glenn Vice President for Student Affairs and Vice Provost for Enrollment and Academic Services

8. The sampleprospective Students Prospective Students

9. The Sample(Prospective Students) • Sample size: 199 prospective students completed the online survey. Two respondents were identified as duplicates and were eliminated. An additional six respondents were removed as these students did not evaluate all three web pages. • With a sample of 191 people, the margin of error on results involving proportions (at a 95% confidence level) is no greater than ± 7.1%. The margin of error is necessary to estimate the percentage of ALL prospective students who would respond in a similar manner. • For example, if we found that 45% of prospective students in the sample agreed with a statement, we can be 95% confident that between 37.9% and 52.1% of all prospective students in the population would respond the same way. • Other factors such as problems with question wording and question interpretation can introduce additional bias or error into the results. The projection of results from a sample onto a population also requires that the sample be representative of that population. • Email invitations to complete the survey were sent to a total of 2,836 randomly selected prospective students. The response rate for this survey of prospective students is 7%.

10. The Sample (Prospective Students) Out of 181 observations Out of 183 observations

11. The Sample (Prospective Students) Out of 180 observations Out of 178 observations

12. The Sample (Prospective Students) Out of 179 observations Out of 183 observations

13. The Sample (Prospective Students) Out of 181 observations

14. The Sample (Prospective Students) Out of 144 observations

15. The Sample (Prospective Students) • The response rate for this survey of prospective students is 7%. Even though this is a low response rate, it is not unreasonably low given the nature (email survey), timing (during June when many students are on vacation), and duration of this survey (two weeks). • The sample contains responses from prospective students who: • Were more likely to be female than male. • Live mostly in Middle Tennessee. • Have relatively high ACT scores compared to current students (the majority of students have an ACT score of 25 or higher). • Have relatively high High School GPA’s compared to current students (the majority of students have a HS GPA of 3.5 or higher).

16. ResultsProspective Students Prospective Students

17. Have you visited the Web sites for any universities that you are considering attending?(Prospective Students) • Practically all prospective students have visited the Web site for at least one university they are considering attending. Out of 182 observations

18. Why do you visit university Web sites?(Prospective Students) • The most often cited reasons for visiting a university Web site were: • To learn about degree programs and majors • To learn about potential scholarships • To see what the campus looks like • To get a feel for the university • To apply to a university • To learn about academics Out of 183 observations

19. Thinking about the overall look and appearance of a Web site, how important are each of the following in determining your desire to attend a university?(Prospective Students) • The most important aspects of the overall look and appearance of a university Web site in determining students’ desire to attend a university are: • The ease with which the Web site can be used and navigated • The amount of pictures, graphics, and videos on the Web site • The amount of information contained in the Web site Margin of error is approximately ±0.15 at a 95% confidence level.

20. How important are each of the following in determining where you want to go to college?(Prospective Students) • Campus visits are the most important factor in determining where a student wants to go to college. • A university’s Web site is less important than: • word of mouth, and • advice from guidance counselors and teachers. Margin of error is approximately ±0.15 at a 95% confidence level.

21. Have you visited the MTSU Web site?(Prospective Students) • Almost all prospective students in the sample have visited the MTSU Web site. Out of 182 observations

22. The Three Web site Designs Design 1 Design 2 Design 3

23. Evaluating the Three Designs(Prospective Students) • In general, design 3 is the most favorably rated Web site design. • Compared to the other designs, design 3: • Is less boring • Is more interesting • Is more appealing • Makes students want to learn more about MTSU Differences of less than approximately 0.18 are not statistically significant.

24. Evaluating the Three Designs(Prospective Students) • Compared to the other designs, design 3: • Makes students more likely to want to attend MTSU • Gives a more positive impression of MTSU • Is more attractive • Looks more like what a university Web site should look Differences of less than approximately 0.18 are not statistically significant.

25. Overall Evaluations of the Three Designs(Prospective Students) • Design 2 and design 3 were rated more favorably than design 1. • There is no statistically significant difference between the ratings for design 2 and design 3. Differences of less than approximately 0.18 are not statistically significant.

26. Ranking the Three Web site Designs(Prospective Students) • When asked to choose among the designs: • Design 3 was most often chosen as the design that students liked the most. • Design 1was most often chosen as the design that students liked the least. • Although there is a clear preference for design 3 when students are asked to choose among the three designs, the difference in liking between design 2 and design 3 is slight (see previous slide). Differences of less than 10% are not statistically significant.

27. If the MTSU Web site looked like the one that you ranked as liking the most, would you:(Prospective Students) • Prospective students responded favorably when asked if the design they chose as their favorite would have a positive effect on their: • Desire to visit the MTSU Web site more often • Opinion of MTSU • Desire the attend MTSU. • Changing the Web site to the design that students chose as their favorite would not have a negative effect on their desire to attend MTSU. Out of 182 observations

28. The SampleCurrent MTSU students Current MTSU Students

29. The Sample(Current MTSU Students) • Sample size: 1220 current students completed the online survey. Fourteen respondents were identified as duplicates and were eliminated. An additional 29 respondents were removed because they did not evaluate all three web pages. • With a sample of 1177 people, the margin of error on results involving proportions (at a 95% confidence level) is no more than ± 2.9%. • For example, if we found that 45% of current students in the sample agreed with a statement, we can be 95% confident that between 42.1% and 47.9% of all current students in the population would respond the same way. • Email invitations to complete the survey were sent to a total of 8,624 randomly selected current MTSU students. The response rate for this survey of current MTSU students is 13.6%.

30. The Sample (Current MTSU Students) Out of 1160 observations Out of 1169 observations

31. The Sample (Current MTSU Students) Out of 1167 observations Out of 733 observations

32. The Sample (Current MTSU Students) Out of 1055 observations Out of 1166 observations

33. The Sample (Current MTSU Students) • The response rate for this survey of current MTSU students is 13.6. This is a reasonably acceptable response rate given the nature (email survey), timing (during June when many students are on vacation), and duration of this survey (two weeks). • The sample contains responses from current students who: • Were more likely to be female than male. • Live mostly in Middle Tennessee. • Were more likely to be upper division students (juniors, seniors, and grad students) – only 14% freshman or sophomores. • Have moderately high ACT scores (69% had an ACT score of 22 or higher). • Have moderately high High School GPA’s (half of students had a HS GPA of 3.5 or higher).

34. Resultscurrent mtsu students Current MTSU Students

35. Did you visit the Web sites for any universities that you considered attending?(Current MTSU Students) • The vast majority of current MTSU students said that they visited at least one university Web site when considering where to attend college. Out of 1171 observations

36. Why did you visit university Web sites?(Current MTSU Students) • The most often cited reasons for visiting a university Web site were: • To learn about degree programs and majors • To apply to a university • To get a feel for a university • To see what the campus looks like • To learn about scholarships • To learn about academics Out of 1177 observations

37. Thinking about the overall look and appearance of a Web site, how important are each of the following in determining your desire to attend a university?(Current MTSU Students) • The most important aspects of the overall look and appearance of a university Web site on students’ desire to attend a university are: • The ease with which the Web site can be used and navigated • The amount of information contained in the Web site • The amount of pictures, graphics, and videos on the Web site. • Note: Most of the comments offered in the “Other” category were not related to the overall look and appearance of a Web site. And, most of these comments were rated either “4” or “5” on the scale. Margin of error is approximately ±0.06 at a 95% confidence level.

38. How important were each of the following in determining where you wanted to go to college?(Current MTSU Students) • Word of mouth was the most important factor in determining where a student wanted to go to college. • A university’s Web site was less important than campus visits and tours. Margin of error is approximately ±0.06 at a 95% confidence level.

39. Did you visit the MTSU Web site as part of your search for the college you wanted to attend?(Current MTSU Students) • A large majority of current MTSU students said that they visited the MTSU Web site as part of their search for a college to attend. Out of 1099 observations

40. The Three Web site Designs Design 1 Design 2 Design 3

41. Evaluating the Three Designs(Current MTSU Students) • Design 1 was rated as: • More informative • Better able to tell students what they want to know • However, design 1 was also rated as: • More boring • Giving more of a negative impression of MTSU • Compared to design 1, design 3 was rated as slightly more interesting. Differences of less than approximately 0.08 are not statistically significant.

42. Evaluating the Three Designs(Current MTSU Students) • There appears to be no real meaningful differences among the designs on these issues, although many of the differences are statistically significant. Differences of less than approximately 0.08 are not statistically significant.

43. Overall Evaluations of the Three Designs(Current MTSU Students) • Design 2 was rated more positively than design 3 and design 1. Differences of less than approximately 0.08 are not statistically significant.

44. Ranking the Three Web site Designs(Current MTSU Students) • When asked to choose among the designs: • There was no clear preference for any one of the three designs. • Design 1 and design 3 were most often chosen as the designs that students liked the least. Differences of less than 4% are not statistically significant.

45. If the MTSU Web site looked like the one that you ranked as liking the most, would you:(Current MTSU Students) • Current MTSU students responded favorably when asked if the design they chose as their favorite would have had a positive effect on their: • Desire to visit the MTSU Web site more often • Opinion of MTSU • Desire the attend MTSU • Changing the Web site to the design that students chose as their favorite would not have a negative effect on their desire to attend MTSU. Out of 1171 observations

46. Further analysis of the web site evaluations cross tabs

47. Evaluations of the Web sites by ACT Score(Prospective Students) • The pattern of evaluations did not differ significantly across the two levels of ACT score. • Overall ratings for design 1 were significantly lower than ratings for design 2 and design 3. • The differences in the ratings for design 2 and design 3 were not statistically significant. Differences of less than approximately 0.35 are not statistically significant.

48. Evaluations of the Web sites by ACT Score(Prospective Students) • When asked to choose among the Web site designs, design 3 was more often chosen by both groups as the design they liked the most. • Although there is a clear preference for design 3 among students with higher ACT scores, the difference in liking between design 2 and design 3 is slight (see previous slide). Differences of less than 15% are not statistically significant.

49. Evaluations of the Web sites by year in school(Prospective Students) • Design 3 was the most preferred by both high school seniors and students who have already graduated from high school. • The relative preference for design 3 is greater for high school seniors than for students who have already graduated high school. Differences of less than 27% for Seniors and 11% for Graduated HS are not statistically significant.

50. Evaluations of the Web sites by sex (Prospective Students) • Design 3 was the most preferred by females. • The relative preference for design 3 among males was not statistically significant. Differences of less than 18% for Males and 13% for Females are not statistically significant.