Caffeine Junkies: Effects of caffeine on word-search task performance of Dartmouth Students

1. Caffeine Junkies: Effects of caffeine on word-search task performance of Dartmouth Students • “Coffee: creative lighter fluid.” --Floyd Maxwell (1957-)

2. Research Rationale • We find that many of our friends consume caffeinated beverages prior to engaging in academic tasks, exams, or other performance evaluations with the belief that the stimulant will improve performance and enable them to focus to a higher degree on the task at hand. We have designed an experiment in which we examined the actual effects of caffeine on the performance of a simple verbal/spatial task (a word search test). The experimental topic may have implications on the effects of caffeine in a college academic testing environment.

3. Our Hypotheses • Null Hypothesis: • Consumption of a tall (355mL) caffeinated coffee will have no effect of the number of words, or have a negative effect on the number of words, found in a simple word search test in a two minute time period. • Alternate Hypothesis: • Consumption of a tall (355mL) caffeinated coffee will improve performance, i.e. increase the number of words found in a simple word search test in a two minute time period. "Coffee falls into the stomach ... ideas begin to move, things remembered arrive at full gallop ... the shafts of wit start up like sharp-shooters, similies arise, the paper is covered with ink ..." -- Honore de Balzac

4. Parameter • If you choose a subject at random from the Dartmouth College undergraduate student body, then there is some degree to which caffeine will improve his/her performance on a verbal/spatial task. This mean score is the average number of words found by participants after drinking caffeinated coffee; -caff. The parameter that we tested is the difference between -caff and the average number of words found by participants after drinking non-caffeinated coffee, -non. Under the null hypothesis, caffeine will not increase the number of words found in the simple word search task; hence the null hypothesis is that -caff = -non, while the alternate hypothesis is that -caff > -non. • Please note that the subjects were NOT in fact chosen at random for our initial testing; Participants were selected on the basis of their presence at Collis student center on the evening of our testing, and whether or not they agreed to be a part of our study (i.e. whether they had sufficient time to take our test, and had the desire to drink coffee specifically).

5. Experimental Equipment 1 • Our materials were as follows: • 1 testing site. This was a large, common room, with a certain degree of noise and student traffic. This was NOT a controlled environment. • 17 identical coffee cups • 3 pens available for students taking the word search test • 1 blue highlighter • 9 cups of caffeinated coffee brewed by Collis Café (all from one batch of coffee) • 8 cups of decaffeinated coffee brewed by Collis Café (all from one batch of coffee) • 17 copies of a simple word-search task. NOTE: All subjects completed identical word searches. The word search used for experimentation was chosen due to its unusual subject matter, astronomy; we hope to avoid any advantage in word recognition, i.e. the words to be found should be equally familiar to most college students (not derived from any specific movie, hobby, or area of expertise that some students are more likely to be familiar with than others). • 1 watch, to time the “wait period” as well as the word search testing period Coffee should be black as hell, strong as death, and as sweet as love. --Turkish proverb

6. The task...

8. Experimental Equipment 2 • Our experiment required the following personnel: • 1 Pourer. Pourer duties include: pour coffee (caff and non) into cups, label the bottom of the decaffeinated cups with a blue pen. • 2 Distributors. Distributor duties include: give subjects the coffee cups. The distributors are blind to the coffee condition, with no knowledge of which color marking corresponds with which cup. The distributor also hands out the word search task, and keeps time; the initial 15-minute “wait” period, as well as the 2-minute testing period.

9. Experimental Procedure

10. Experimental Procedure 1 • Our Experimental Protocol was as follows: One experimenter approached potential participants in Collis student center, and asked them if they would be willing to be a part of a brief, 17-minute experiment in which they would drink a full cup of coffee and engage in a short word search task. Once students were recruited for participation, the pourer filled and marked as caff or non 17 cups (total). The distributors then handed out the cups of coffee to willing subjects, and timed a waiting period of 15 minutes. Participants were instructed to drink the coffee but to keep their cups. “Decaf: Used to sober up after a night on non-alcoholic beer!” - Unknown

11. Experimental Procedure 2 • 17 subjects total participated in the experiment (9 caffeinated experimental subjects, 8 non-caffeinated control subjects). • Research has shown that caffeine reaches its densest concentration in the blood 15 to 45 minutes after consumption; this is the reason we imposed a 15-minute wait period after participants finish the coffee. Participants remained in the same seat during this period (although they were permitted to talk to others, read, do homework, etc.)

12. Experimental Procedure 3 • After the 15-minute “wait period” delay, the distributor handed out the word search task, as well as pens, to the participants. Subjects were instructed to find and circle as many words as possible during a two minute time period. • At the end of this 2-minute period (measured by the Distributors with watches), subjects were instructed to put their pens down and the Distributors collected the word search tests and the cups. • The distributors and pourer then recorded the number of words found on each puzzle. Subjects’ identity was anonymous, with all data coded according to experimental status (caff or non). "It is inhumane, in my opinion, to force people who have a genuine medical need for coffee to wait in line behind people who apparently view it as some kind of recreational activity." -- Dave Barry

13. Caffeine: is it helping students or just an addiction? “It is caffeine alone that sets my mind in motion. It is through beans of java that thoughts acquire speed, that hands acquire shakes, that shakes become a warning...I am...IN CONTROL...OF MY ADDICTION!” -- From the Minicon Graffiti Wall, 1989

14. Statistical Analysis: Preliminary test statistic • Our primary analysis goal was to determine whether or not the difference between the average number of words found for each independently sampled group is significant. • Due to the fact that our sample size was extremely small (n-caff = 9, n-non = 8) we implement a t-test for the significance of the difference between means. • We assume that the two populations (people who ingest caffeinated vs. non-caffeinated beverages) are normally distributed, and that they have equal standard deviations. (σ-caff = σ-non)

15. Descriptive Statistics • After collecting data, we found the following average scores for our experimental (caffeinated) and control (non) groups: • X-caff = 6.33 • X-non = 5.38 • We also calculated variance with the following equation: • s2 = (x – x)² (n – 1) • This gave us the following resulting variances for each group: • s2 -caff = 6.945 • s2 -non = 2.839

16. Statistical Analysis: Significance level Since we wanted to be able to report these results as statistically significant, we chose the significance level to be 0.05. In other words, we left only a 5% chance of a type 1 error. In this setting, a type 1 error corresponds to concluding that caffeine improves Dartmouth students’ performance on a word search test, when in fact there is no improvement. The equation we used to determine whether or not the difference in average number of words found from the two groups is significant is: t = x-caff – x-non _ √[(s2²/n-caff) + (s2/n-non)] when s2 = (n-caff – 1)s-caff2+ (n-non – 1)s-non2 n-caff + n-non – 2

17. t-score • For s2 = (9 – 1)(6.945) + (8 – 1)(2.839) 9 + 8 – 2 • s2 = 5.029 Therefore • t = 6.33 – 5.38 √[(5.029/9) + (5.029/8) • t = 0.877

18. Degrees of Freedom • Using the t-table, we must take into account the degrees of freedom. Due to the fact that we have two samples, we find that • df = n-caff + n-non – 2 • df = 9 + 8 – 2 • df = 15

19. Statistical Analysis: Critical region • Because our Alternate Hypothesis is that -caff will be significantly greater than µ-non, we are only concerned with the critical region under the right-sided tail of our distribution, at the .05 significance level. Therefore, to find the critical region, we use degrees of freedom (dF= 15) and the t-table (see Appendix A of our class text) to find the area under the standard normal curve equal to the significance level of 0.05. • The critical region for our experiment, given the following: • Null hypothesis: -caff = -non • Alternate hypothesis:  -caff >  -non • Significance level:  = 0.05 • Thus the Critical region is: t > 1.753

20. INsignificant results! • Because our calculated t value of 0.877 does not fall within the above critical region, the difference between the means of our two groups (caffeinated and non-caffeinated) is NOT significant at the 0.05 level. • As we discussed in our proposal, we expected not only that caffeinated subjects would find a significantly higher average number of words in their word search test, but also that the power of our hypothesis would be approximately 40% (i.e. caffeinated subjects would find at least 40% more words than would subjects given a non-caffeinated beverage). Given our results, the likelihood that caffeine does in fact help subjects on a word search task, to the extent that we believed it would, is extremely low.

21. Confounding factors 1 • Our experiment dealt with the question of how caffeine affects performance amongst Dartmouth College students. Unfortunately, we were unable to conduct an experiment in which all variables were controlled, or which was extensive enough, to truly draw any conclusions on this topic. • Our primary difficulty is rooted in our extremely small sample size. The optimum size for a two-independent-sample test is N-1 + N-2 > 60. We had hoped to include at least 30 participants in each group (caffeinated and non-caffeinated subjects); however, we were unable to recruit an adequate number of participants.

22. Confounding factors 2 • A second confounding factor in the implementation of our experiment was our inability to control the participants’ caffeine consumption prior to testing. We would have liked to recruit subjects at least 24 hours before testing, and to have instructed them to refrain from ingesting caffeinated substances. However, due to time constraints, our experiment was conducted without knowledge of our subjects’ blood caffeine content. This may have affected our results. • Additionally, we did not recruit a random sample for our testing. Participants were selected on the basis of their presence at Collis student center on the evening of our testing, and whether or not they agreed to be a part of our study (i.e. whether they had sufficient time to take our test, and had the desire to drink coffee specifically).

23. Confounding factors 3 • Finally, although our test was double-blind (one student poured the coffee, and differentiated between caffeinated and non-caffeinated, while the other two students distributed the coffee) in order to avoid all experimenter bias we would have liked to have a non-affiliated math 5 student to keep time, pass out the tests, and collect the tests and coffee cups. We also would have preferred making our own coffee, so that we could have measured caffeine concentration. Also, we did not measure the volume of coffee given to each participants, although we made a visual approximation to the best of our skill.

24. Conclusion • In conclusion, although we were disappointed that our experiment did not yield significant results, we believe that the topic of caffeine consumption and effects on student health and academic welfare should be further explored in more controlled studies. Some possible areas of research include: what are student perceptions of the effects of caffeine consumption, and are these perceptions correct? What are the effects of continued caffeine use across the college years? And finally, why does Dirt Cowboy close at like 4pm when I need my coffee at 11pm? :-)