Responsible use of big data for public purposes: experiences from Sri Lanka

1. Responsible use of big data for public purposes: experiences from Sri Lanka Sriganesh LokanathanLIRNEasia Data Innovation for Policy Makers Nusa Dua, Bali 26 Nov 2012 This work was carried out with the aid of a grant from the International Development Research Centreof Canadaand the Department for International Development (DFID) of UK

2. About LIRNEasia (www.lirneasia.net) • LIRNEasia is a regional think tank based in Colombo, working across South Asia, South East Asia and the Pacific Small Island States • Our mission: • Catalyzing policy change through research to improve people’s lives in the emerging Asia Pacific by facilitating their use of hard and soft infrastructures through the use of knowledge, information and technology

3. LIRNEasia’s Big Data for Development (BD4D) Research • LIRNEasia has negotiated access to historical and anonymized telecom network big data from multiple operators in Sri Lanka • In the current research cycle we are • conducting exploratory research on answering several social science questions related to mobility and connectedness • developing self-regulatory guidelines for the collection, use and sharing of mobile phone data by third parties that may be used by mobile operators • http://lirneasia.net/projects/bd4d/

4. Why use mobile network big data? • Current data collection infrastructure is woefully inadequate for fast moving developing economies • Household surveys and censuses are infrequent and costly; analysis is slow • Sri Lanka Census 2011-12 results not out yet; only the 5% sample • Household Income & Expenditure Survey 2012-13: only 8 page summary • Low penetration of other data sources such as sensors, etc. for various policy domains • Low levels of ‘datafication’ in general • Very high coverage of the population by mobile phones in developing economies gives opportunities to leverage mobile network big data to produce development policy insights • Geographically granular • Of high temporal frequency • Includes data on mobility of users

5. Mobile network big data (+ other data)  rich, timely, policy-relevant insights Mobile network big data (CDRs, Internet access usage, airtime recharge records) Insights Construct behavioral variables Other data sources • Urban & transportation planning • Crisis management & DRR • Health monitoring & planning • Poverty mapping • Financial inclusion • Census data • HIES data • Survey maps • Transportation schedules • ++ • Mobility variables • Social variables • Consumption variables Analytics

6. Some examples of timely, policy relevant insights from mobile network big data

7. Mobile Network Big Data (MNBD) has high relevance for transportation planning • The image on the left depicts relative density of people in Colombo city and the surrounding regions at 1300 compared to 0000 (midnight the previous day) on a normal weekday. • The yellow to red colors depict areas whose density has increased relative to midnight. The blue color depicts areas whose density has decreased relative to midnight (the darker the blue, the greater the loss in density). The clear areas are those where the overall density has not changed.

8. Our findings closely match results from expensive & infrequent transportation surveys

9. MNBD based insights can inform urban policy 2020 UDA Plan 1985 Plan 2013 reality (from mobile data) Commercial areas Residential areas Mixed-use areas

10. We can derive new high-frequency measures of economic activity from MNBD Low High Economic Activity

11. So how did we get this data?

12. Mobile network big data (+ other data)  rich, timely, policy-relevant insights Private sector data Government data Mobile network big data (CDRs, Internet access usage, airtime recharge records) Insights Construct behavioral variables Other data sources • Urban & transportation planning • Crisis management & DRR • Health monitoring & planning • Poverty mapping • Financial inclusion • Census data • HIES data • Survey maps • Transportation schedules • ++ • Mobility variables • Social variables • Consumption variables Analytics

13. Obtaining data from mobile operators

14. Obtaining mobile network data • No established process exists, therefore prior relationships matter • Basic process: • Obtain in-principle agreement from CEO of companies (ideally atleast 2) where we had prior relationships • Negotiate specifics with 2nd and 3rd tier management • Approach other operators • Throughout we highlight reciprocity • methods for deriving public policy insights can also be adapted for commercial purposes

15. Answering operator concerns • Question: Will the regulator raise any concerns? • Answer: • The specific data requested did not contravene existing laws or principles • Data we obtain is anonymized with no linkages to original numbers • Question: Will this research reveal any proprietary business intelligence? • Answer: • We combine data from multiple operators • All researchers on an NDA with LIRNEasia • Operators sign off on our results before public release

16. Example: Establishing a data sharing agreement with an unnamed operator in Sri Lanka • 1 short initial meeting b/w LIRNEasia’s Founding Chair and operator CEO • Had sent a brief concept note on potential collaboration prior to meeting • Outcome: obtained in-principle agreement during this initial meeting • Negotiate specifics with 2nd and 3rd tier management: • 7 in-person meetings • Marketing, Business Intelligence, Network Engineering, Regulatory & Legal • ~7.5 hours in total • 4 conference calls • ~1 hour in total with operator; ~1 hour in total with LIRNEasia lawyers • 16 email exchanges • 11 with operator; 5 with LIRNEasia lawyers • 6 rounds of revisions of basic agreement • 11 people involved in total • 7 from operator; 3 from LIRNEasia; 1 from LIRNEasia’s lawyers • Sign agreements 6 months

17. How can access to data from operators for public purposes be made easier for others? • Reduce transaction costs: • Standardized agreement template(s) • Pro-actively deal with privacy concerns • Operators adopt self-regulatory guidelines for minimizing the potential harms from giving access to mobile network data • LIRNEasia working with operators in the region in this regard • Draft self-regulatory guidelines developed by LIRNEasia available at http://lirneasia.net/2014/08/what-does-big-data-say-about-sri-lanka/ • Perhaps there is a need for new operational models with 3rd party data guardians • E.g. Yale University Open Data Access (YODA) project acting as data guardians and providing researchers access to Johnson & Johnson clinical trial data

18. Obtaining government data

19. Obtaining government data • Some processes do exit for getting data, but not universally • Easier if requesting organization is officially affiliated to the government, which LIRNEasia isn’t • Our process for requesting government data varied/ varies using one or more of the following: • Submitting a data request proposal • Meeting(s) with senior level officials • Spending time talking to lower level officials and negotiating access • Building a compelling story from just the analyses of the mobile network data clearly outlining how insights can be improved with use of government data

20. Concluding thoughts • Negotiating data access from private sector operators should address their concerns and provide some reciprocal benefits • Getting government data is still not easy and requires not just policy enlightenment but also policy evangelism • Compelling data sharing (with privacy protections) is not really the answer in competitive industries, but there is a case for some form of open data policies in monopolistic sectors (e.g. electricity)