Management field experiments

Management field experiments Nick Bloom (Stanford and NBER) www.stanford.edu/~nbloom AOM, August 3rd 2012

Management field experiments • Review of current experimental literature • Our projects in India and China • Thoughts on running field experiments

Surge in management experiments, but mainly in:(A) micro-firms (1 or 2 person) in developing countries (B) individual larger firms The reason is running management experiments is expensive

Developing countries, micro-firm experiments • Karlan & Valdivia in Peru; Bruhn, Karlan & Schoar in Mexico; Karlan & Udry in Ghana; McKenzie & Woodruff in Sri Lanka etc. • Provide limited (≈50 hours) of basic trainings to small firms: accounting, marketing, pricing, strategy etc. • Training is provided at random and data collected before & after • So far finding not-much impact. I see two potential explanations • management does not matter in tiny firms, or • intervention is very poor quality

Single firms in developed countries, (1/2) • Growing literature (surveyed in Bloom & Van Reenen, 2010, Handbook of Labour Economics) • Classic examples include: • Lazaer(2000, AER) on incentive pay at SafeliteGlass, • Shearer (2004, REStud) on tree planters and • Hamilton et al (2003, JPE) on group incentives in factories

Single firms in developed countries (2/2) • Recently Bandiera, Barankay and Rasul have an impressive set of papers. Run experiments on incentives for workers and managers, team selection, and task division on a fruit farm • Introduce changes ½ way through season (using last season as the control), finding for example • Worker incentive pay increases their performance, especially absolute (rather than relative) incentives • Manager incentive pay improves team selection (less favoritism) and the effort they put into monitoring workers

Management field experiments • Review of current experimental literature • Our projects in India and China • India • China • Thoughts on running field experiments

Does management matter?Evidence from India Nick Bloom (Stanford)Benn Eifert (Berkeley)Aprajit Mahajan (Stanford)David McKenzie (World Bank)John Roberts (Stanford GSB) http://www.stanford.edu/~nbloom/DMM.pdf

One motivation for looking at management is that country management scores are correlated with GDP US Japan Germany Sweden Canada Australia UK Italy France New Zealand Mexico Poland Republic of Ireland Portugal Chile Argentina Greece Brazil China India 2.6 2.8 3 3.2 3.4 Management score Source:www.worldmanagementsurvey.com

And firm management spreads look like TFP spreads US (N=695 firms) Density India (N=620 firms) Density Management score Source:www.worldmanagementsurvey.com

But does management cause any of these TFP differences between firms and countries? Massive literature of case-studies and surveys but no consensus Syverson (2011, JEL) “no potential driving factor of productivity has seen a higher ratio of speculation to empirical study”.

We ran an experiment on large firms to investigate the impact of modern management practices on TFP • Experiment on 20 plants in large multi-plant firms (average 300 employees and $7m sales) near Mumbai making cotton fabric • Randomized treatment plants got 5 months of management consulting intervention, controls got 1 month • Consulting was on 38 specific practices tied to factory operations, quality and inventory control • Collect weekly performance data from 2008 to August 2010, and long-run size and management data from 2008 to 2011

Exhibit 1: Plants are large compounds, often containing several buildings.

Exhibit 2: Plants operate continuously making cotton fabric from yarn Fabric weaving

Exhibit 3: Many parts of these Indian plants were dirty and unsafe Garbage outside the plant Garbage inside a plant Flammable garbage in a plant Chemicals without any covering

Exhibit 4: The plant floors were often disorganized and aisles blocked Instrument not removed after use, blocking hallway. Old warp beam, chairs and a desk obstructing the plant floor Dirty and poorly maintained machines Tools left on the floor after use

Exhibit 5: The inventory rooms had months of excess yarn, often without any formal storage system or protection from damp or crushing Yarn without labeling, order or damp protection Yarn piled up so high and deep that access to back sacks is almost impossible Different types and colors of yarn lying mixed A crushed yarn cone, which is unusable as it leads to irregular yarn tension

Intervention aimed to improve 38 core textile management practices in 6 areas – for example Targeted practices in 6 areas: operations, quality, inventory, HR and sales & orders

.6 .5 .4 .3 .2 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 Months after the diagnostic phase Adoption of the 38 management practices rose Treatment plants Control plants Share of 38 practices adopted Non-experimental plants in treatment firms Months after the start of the diagnostic phase

In terms of performance looked at four outcomes we have weekly data for Quality Inventory Output Productivity (defined as: Log(VA) – 0.42*log(K) – 0.58*log(L)) Use weekly data from March 2008 until August 2010 (after which some firms started upgrading to Jacquard looms)

Poor quality meant 19% of labor went on repairs Large room full of repair workers (the day shift) Workers spread cloth over lighted plates to spot defects Defects are repaired by hand or cut out from cloth Defects lead to about 5% of cloth being scrapped

Previously mending was recorded only to cross-check against customers’ claims for rebates

Now mending is recorded daily in a standard format, so it can analyzed by loom, shift, design & weaver

The quality data is now collated and analyzed as part of the new daily production meetings Plant managers meet with heads of departments for quality, inventory, weaving, maintenance, warping etc.

Quality improved significantly in treatment plants Control plants Quality defects index (higher score=lower quality) Treatment plants Weeks after the start of the experiment Note: solid lines are point estimates, dashed lines are 95% confidence intervals

Organizing and racking inventory enables firms to substantially reduce capital stock Stock is organized, labeled, and entered into the computer with details of the type, age and location.

Many treated firms have also introduced basic initiatives to organize the plant floor Marking out the area around the model machine Snag tagging to identify the abnormalities

Spare parts were also organized, reducing downtime (parts can be found quickly) Nuts & bolts Spare parts Tools

Production data is now collected in a standardized format, for discussion in the daily meetings After (standardized, so easy to enter daily into a computer) Before(not standardized, on loose pieces of paper)

TFP rose in treatment plants vs controls Treatment plants Total factor productivity Control plants Weeks after the start of the experiment Note: solid lines are point estimates, dashed lines are 95% confidence intervals

Why do badly managed firms exist? Competition heavily restricted by trade restrictions, the difficulty of new firms entering (finance is hard to raise), and the difficulty of good current firms expanding (limited by family size) Information is limited: firms either not aware of modern practices or simply do not believe they matter (“not worth the it”)

Does Working from Home Work?Evidence from a Chinese Experiment Nick Bloom (Stanford)James Liang (Ctrip & Stanford)John Roberts (Stanford)Jenny Ying (Stanford) http://www.stanford.edu/~nbloom/WFH.pdf

CTrip, China’s largest travel-agent (13,000 employees, and $5bn value on NASDAQ). James Liang is the co-founder, first CEO and Chairman CTrip, a large NASDAQ listed Chinese multinational, wondered about introducing working from home 34

The randomization into working from home was done publicly and also shown on the firm intranet Open lottery over even/odd treatment Working at Home Working at home Working at Home

Experiment yielded four learnings for the firm:(1) Working-from-home works (on average) Treatment Normalized calls per week Control Before the experiment During the experiment

Experiment yielded four learnings for the firm:(2) Better & worse workers both improve when WFH Normalized calls per week:difference between home and work Before experiment During experiment

Experiment yielded four learnings for the firm:(3) Selection: Worker choice increases WFH impact Normalized calls per week:difference between home and work Before theexperiment During the experiment After the experiment (roll-out)

Experiment yielded four learnings for the firm:4) Employees value WFH as attrition down 50% Note: average daily commute is 1.41 hours and cost $0.96

Experiment so successful that CTrip is rapidly rolling out WFH across the firm • Profit increase per employee WFH about $2,000 per year: • Rent: $1,200 per year • Retention: $400 per year • Labor costs: $300 per year

Thoughts on experiments • Expensive and hard, but worthwhile for the right question • Risky for junior faculty as can take many years • Think about both measure and intervention – both can be tough (in India measuring the control firms was tough) • Works best as a team effort – design, funding and execution all best as joint production • Running pilots and spending time on the ground invaluable for effective operation, analysis and presentation

Management field experiments Nick Bloom (Stanford and NBER) www.stanford.edu/~nbloom AOM, August 2012

Management field experiments