An Experiment to Increase Magazine Sales in a Supermarket

1. An Experiment to Increase Magazine Sales in a Supermarket Gordon Bell Johannes Ledolter Arthur J. Swersey

2. Introduction Experimental design methods have been widely applied to manufacturing problems but applications in service areas have been very infrequent Great opportunities exist for operations researchers to apply these powerful methods to problems in marketing and service operations Ledolter, and Swersey, Testing 1-2-3: Experimental Design with Applications in Marketing and Service Operations, Stanford University Press, 2007 (a shameless promotion of our book) Box, Hunter, and Hunter, Statistics for Experimenters (second edition, Wiley, 2005)

3. The Current Study Digest-sized weekly magazine�one of the top 10 circulated magazines in the U.S. Test employed a Plackett-Burman design and took place in 48 supermarket stores over a two-week period The 10 factors were tested simultaneously and related to magazine location and in-store advertising and promotion In contrast, the traditional approach is to test one factor at a time

4. The 10 Test Factors and Their Levels

5. Alternative Designs A full factorial design for n factors requires runs; with 10 factors that would mean 1024 runs! In fractional factorial designs the number of runs N is a power of 2 (N = 4, 8, 16, 32, and so forth) In Plackett-Burman designs the number of runs N is a multiple of 4 (N = 4, 8, 12, 16, 20, 24, and so forth) Plackett-Burman designs fill in the gaps in the run sizes Fractional designs introduce confounding among the estimated effects (no such thing as a free lunch)

6. Some Design Principles Effects sparcity� in any experiment the great majority of effects are likely to be negligible Hierarchical ordering� main effects tend to be larger in magnitude and hence more important than 2-factor interactions, 2-factor interactions larger than 3-factor interactions, and so forth 3-factor and higher order interactions are very likely to be negligible and can be ignored

7. Design Resolution An index that denotes the nature of the confounding in fractional designs A resolution III design confounds main effects with 2-factor interactions A resolution IV design confounds main effects with 3-factor interactions and confounds 2-factor interactions with other 2-factor interactions A resolution V design confounds main effects with 4-factor interactions and 2-factor interactions with 3-factor interactions

8. Design Choices A 16-run fractional factorial design is resolution III (main effects confounded with 2-factor interactions) A resolution IV fractional factorial design (main effects confounded with 3-factor interactions) requires 32 runs A 12-run Plackett-Burman design is resolution III A 24-run Plackett-Burman design is resolution IV

9. A Resolution III Fractional Factorial Design with Generators E = ABC, F = BCD, G = ACD, H = ABD, I =ACD, J = AB

10. A 12-Run Resolution III Plackett-Burman Design for Testing Up to 11 Factors

12. Selecting and Pairing Test Stores We assigned two stores to each experimental run because the % change in sales is less variable for a 2-store unit than for a single store. This increased the statistical power of our test. We identified stores in the region with stable demand and chose the 48 with the highest weekly sales, pairing high demand stores with low demand stores.

14. Response Variable The baseline was the average weekly sales for each pair of stores over the 7 weeks prior to the 2-week experimental period For each experimental run we measured the percent change in sales for each pair of stores in week 1 of the experiment compared to the baseline and the percent change in sales in week 2 The response variable was the average of the week 1 and week 2 percent changes

15. Estimating Main Effects and Determining Statistical Significance Each estimated main effect is the average of the 12 responses when the factor is at the plus level minus the average of the 12 responses when the factor is at the minus level Determining statistical significance is equivalent to testing the difference in means for two independent samples, a consequence of the orthogonal design matrix

16. Estimated Effects

17. Significant Effects Rack on cooler in produce aisle increased sales by about 10 percent Sales dropped nearly 9 percent when the distribution of magazines was evened out across pockets An additional rack by snack foods increased sales by more than 5 percent

18. Conclusions and Future Directions Results show the potential benefits of additional product displays Ads on grocery dividers were the only advertising/promotion factor that showed some evidence of effectiveness But in-store promotions are inexpensive and easy to implement and their effectiveness warrants further study Fractional factorial designs dramatically reduce the number of experimental runs required and make it possible to test many factors simultaneously

19. Conclusions (continued) There are many opportunities to apply these methods in service settings Optimizing website design Direct mail campaigns Testing the effectiveness of educational programs --changes in class size, textbooks, computer software and so forth As researchers become more aware of these powerful and efficient approaches, their use is likely to become more widespread