Issues in Improving Responsible Retailing Practices

1. Issues in Improving Responsible Retailing Practices Joel W. GrubePrevention Research CenterPacific Institute for Research and Evaluation

2. Complex Factors Affect Sales Rates Broader Legal and Social Environment Corporate Characteristics Clerk Store Characteristics Context

3. Probability of at Least One SaleGiven True Compliance Rate ActualCompliance Rate Probability of at least one sale

4. Number of Observations Needed to Describe Prevalence of Sale for Individual Store 95%Confidence Interval Observations

5. Why Does it Matter? The complexity and variability create uncertainty when: • Evaluating program effects—How do we know what works? • Monitoring store performance—How do we know when a store has improved? • Comparing stores—How do we know one store is doing better or worse?

6. Voluntary Compliance Programs • Acting under the authority of Consumer Protection laws, state Attorneys General have engaged national retail chains to intensify their efforts to prevent underage sales of tobacco products • National drug store, discount and convenience store/gas station chains have executed Assurances of Voluntary Compliance (“AVC”) in which they stipulate to changes in hiring, training and point-of-sales protocol

7. Voluntary Compliance Programs Evaluation shows clerks and managers have: • High knowledge of laws and store policies • High perceived self-efficacy • Strong personal support for the goal of preventing underage tobacco sales Nevertheless: • Monitoring shows that clerks fail to perform age-verification protocols correctly in 20% - 30% of attempted tobacco purchases Source: Krevor, Prottas & Fournier, 2006

8. Voluntary Compliance Programs

9. Mystery Shopper Reports • Mystery Shopper (“MS”) reports are being considered by Attorneys General as an additional stipulation in AVCs • Mystery Shopper Reports provide detailed feedback on observed age-verification • There is no evidence at present whether frequent MS reports improve age-verification

10. Mystery Shopper Evaluation Project Brandeis Brad Krevor Stephen Fournier Jeffery Prottas PIRE Joel Grube William Ponicki

11. Mystery Shopper Evaluation Project The primary aims of the study are: • Do frequent (monthly) MS reports increase correct age-verification conduct? • Do the effects of intensive MS reporting persist? • How do managers use the reports? • Identify store, purchaser, and clerk characteristics related to age-verification conduct. Hypothesis: • Increased feedback will raise the salience of tobacco age-verification at the store and district level leading to higher rates of age-verification

12. Mystery Shopper Evaluation Project • Randomly assign 206 CORS to treatment and control • All CORs inspected by Mystery Shoppers twice a month for 13 months • Treatment condition: In Months 3 –10 every 2nd inspection by the Mystery Shopper will be reported • Control Condition: Every 6th inspection will be reported • Telephone interviews with store and district managers in months 6 and 9 • In months 11 - 13, the twice-monthly MS inspections will continue, but no MS reports

13. Mystery Shopper Evaluation Project • There are 206 CORS. For each COR there will be observations at 26 time points • A total of 5,356 MS observations • Sufficient observations for detecting a 5% treatment impact (75% to 80%) over the treatment period and a 10% (75% to 85%) impact within the shorter post-treatment period

14. Summary • Accurately measuring individual store performance is very difficult • Measures are highly variable • Performance is affected by a wide range of factors • Adequately capturing individual store performance requires a large number of measures at different times, days, and with different clerks • Performance at the aggregate level (e.g., chain, community) more easily measured • Despite difficulties, it is possible to measure performance and evaluate programs (e.g., MS)