Open Data: Status and Directions

1. Open Data: Status and Directions HSCIC Health Information Supplier Forum 28 February 2013 Hugh Neffendorf Katalysis Limited APPSI member

2. Topics • Status of Open Data • Advisory Panel on Public Sector Information • Functions • Recent Activities • A National Information Framework • Open examples – OS and ONS • Open issues - where next for Open Data?

3. Open Data: Background and Status • Recent history……. • http://www.data.gov.uk • Drivers - transparency and accountability • Consultations • EU Directives • White Paper • Open acronyms…DSB, PDG, ODUG, ODI, etc. • Getting to grips with modern/practical issues Part of a worldwide ‘explosion’

4. Open – a government view

5. ODUG-centric view of the universe Government Open Data Users Central Government Media Campaigning Organisations Data Journalists Sector Transparency Boards Public Sector Transparency Board Departments e.g. BIS, DfT, CO etc. Hyper Local Media Open Rights Group Customer Groups e.g. PWSCG, GISCG Tech Trade Press Commercial Organisations OKFN TNA MySociety Data Strategy Board Web Services ONS Systems Integrators HMRC Consultancies Open Data User Group Agencies Big Data Insurers Public Data Group Market Research Transport and Logistics Trading Funds e.g. OS, MO, CH & LR Developers Credit Bureaux Telecoms GIS Organisations Local Government Environment Agency Incubators e.g. Tech City Academia Local Authorities ODI Health Services Citizens / Civil Society LGA Open Data researchers Education Providers Not Exhaustive Statisticians Groups e.g. Which? Consumer Focus etc. Libraries Rewired State UNCLASSIFIED

6. APPSI • Advisory Panel on Public Sector Information • Advice to Government - Ministerial appointments • Independent specialists - cross discipline • Lawyers, economists, statisticians, public sector, private sector, academics, subject specialists – supported by The National Archives We monitor PSI activity, review official initiatives, undertake independent developments

7. APPSI – recent activities • Extensive commentary on Open Data plans • National Information Framework • Discussion paper http://www.nationalarchives.gov.uk/appsi/appsi-discussion.htm • Glossary for Open Data and PSI terminology • Meeting on private sector and Open Data Comments and inputs on NIF and Glossary welcome

8. Private sector and Open Data • Met with Google, Sainsbury, Deloitte and GB Group (data reseller) • Six common features • Open Data not yet very useful • Awareness; relevance; key data not yet available • All want to see addresses made free • Easier licensing as important as price • Want bulk delivery as well as APIs • Happy to have data relatively ‘raw’ • Want consistency and continuity of supply

9. APPSI Glossary

10. APPSI Glossary • Approximately 70 terms defined with sources and star rating • Designed to reduce confusion and improve compatibility • Cabinet Office/APPSI consultation soon through www.data.gov.uk http://www.nationalarchives.gov.uk/appsi/open-data-psi-glossary-pilot.htm

11. National Information Framework

12. What is the NIF? • A broad approach to maximising the value of information to the nation and to society • A representation of a Government Information Strategy • An information parallel to the National (physical) Infrastructure • A structured approach to releasing the greatest benefit from our information • A manageable initiative to create, make readily available and publicise an entity and framework to promote availability and best practice in fostering openness and the information ecosystem

13. What topics are covered in the NIF? • Data, policies about its availability, pricing and licensing, standards, applications of the data, research, guidance, fostering, producers and users, tools, evidence and measurement, organisations involved, delivery, etc. • Potentially, all relevant data matters are in scope • Realistically, focussed on information and processes that government can most readily influence – and that are high priorities

14. What information is in the NIF? • Potentially a wide range of data but focussed first on PSI • Not just central government data - devolved governments, LAs, health, etc., near-government (e.g. utilities/transport), private sector involved in public services plus independent and voluntary sectors where they too provide publicly funded or (tax-) subsidised services • PSI is a wide definition and is sometimes blended with data from non-public sources

15. Privacy in the NIF • Can be personal or commercial confidential • Public confidence must be assured • Anonymity must be assured • There are proven good ways to release anonymised sensitive data – vital to do so • There are good precedents and current action e.g. Census SARs, OIC initiative, HSCIC data linkage, Scottish government consultation on data linkage http://www.scotland.gov.uk/Publications/2012/03/3260 An important element of PSI - needs special attention

16. What are the NIF applications? • A vast array – known and unknown • There is a need to better understand usage and applications for input to prioritisation and investment • The pace of change in electronic methods will greatly increase the scope and value of data (and add new challenges) The NIF is a living entity

17. Is the NIF the same as Open Data? • The NIF is a broader concept that embraces Open Data • Government plans for Open Data are a relevant and important component of the NIF • Recent initiatives and bodies provide a good starting point

18. Which public bodies have an NIF role? • Cabinet Office, BIS, other Departments, devolved administrations, ICO, UKSA, TNA/APPSI, DSB/ODI/PDG, LGA, NHS and many others • There is a need to rationalise roles and responsibilities for the NIF • Lead responsibility might be vested in an existing entity, at least initially The NIF should be UK-wide with logical devolved variants

19. Why is the NIF so important? • The information economy is already worth £billions and growing rapidly • The UK should be best placed to make the most of this • There is considerable waste from duplication of effort and data misuse • Better data availability in the widest sense will lead to better applications – quality, relevance, openness, public service, economic value

20. Key NIF topics: not adequately covered! • Establishing a centre of excellence • Seeking/demonstrating benefits from information • Stimulation of growth and openness • Regulation and inducements • Priorities • Research and skills development • Guidance to producers and users • Efficiency • Quality – promotion and review • Standards, protocols, terminology and metadata • Data intermediaries • Charging & licensing

21. APPSI recommendations (1) • Government should provide a clear and explicit strategic vision on its role in information collection, dissemination and exploitation • Policy should facilitate better public/private sector cooperation and innovation • Open Data should be recognised as a component and key stepping stone to NIF • Realising the wider NIF framework should be recognised as a key success factor

22. APPSI recommendations (2) • The Data Strategy Board should be considered as the initial leader of a NIF • Personal data needs special attention • Enhancing human capital is central to delivering Open Data/NIF success • NIF is a living thing so must anticipate new data needs, enforce standards, evaluate old data series for continuing value • Further discussion and investigation of the NIF is now required

23. Ordnance Survey and Open Data • Considerable OS support for Open Data • Some dozen products fully open • Some Linked Data • Mainly small scale - still very useful • OS provides advice and even hosting • Uses variation on Open Government Licence • Already many fine applications Main question – will there be more (and more detailed) open data, including Core Reference data such as addresses?

24. ONS and Open Data • ONS has fully embraced the Open agenda • Using the Open Government Licence • Own dissemination, not www.data.gov.uk • Census data and geography included • Experimenting with APIs and Linked Data

25. Open issues • Linked up activity – the NIF • Ensuring more Open Data – with metadata! • Improved accessibility of data • More focus on economic value of Open Data • Core reference data (e.g. addresses) free at the point of use • Meeting challenges – attitudes, usefulness, privacy, etc. • Maintaining a vigil on changing requirements

26. And finally…..

27. Outside view on HSCIC Perspective • A suitable balance of publications, data, services • A positive attitude to Open Data • A constructive approach to data linkage Is it as accessible, modern, ready for big data as necessary?