Presentation Transcript

1. Statistical Data Integration: What’s all the excitement about?

   August 2011

   Gemma Van HalderenStatistical Data Integration and Analysis BranchAustralian Bureau of Statistics
2. What’s all the excitement about? Information to address (solve?) those important issues of today Homelessness, climate change, social inclusion, ageing population Making better use of existing data ROI Filling data gaps Giving effect to legislation Maximising the use of data for official purposes A community of government officials demanding and creating change National statistical service in practice
3. Journey so far
4. Joint proposal from Australian Statistician and Secretary of Health and Ageing to put in place a safe and effective environment for integrating Commonwealth data for statistical and research purposes Portfolio Secretaries meeting (now Secretaries Board) April 2009 Cross Portfolio Statistical Data Integration Committee established Jointly chaired by ABS and Department of Health and Ageing Represented by portfolio agencies
5. Principles for Statistical Data Integration endorsed Portfolio Secretaries meeting (now Secretaries Board) February 2010 Developed by the Cross Portfolio Statistical Data Integration Committee Represented by portfolio agencies
6. Commonwealth High Level Principles
7. Principle 3: Integrator’s Accountability An IA must be identified for each statistical data integration proposal. This authority will be held responsible for the sound conduct of the statistical data integration proposal, in line with the agreed requirements of the responsible agencies Although the IA is the single organisation ultimately accountable for the statistical data integration project, it may work with a network of agencies to achieve the data integration. The IA will ensure appropriate governance is in place including An open approval process is followed Documentation of the proposal The impact on privacy Risks have been assessed, managed and mitigated The expected costs and benefits The outputs
8. Principle 3 cont. A family of data integration projects using the same source datasets, for similar purposes, with the same IA, may be treated as a single program for the purposes of the approval process The IA will be responsible for the ongoing management of the integrated data, ensuring it is kept secure, confidential, and fit for the purposes for which it was approved If it is an ongoing project, the IA will be responsible for initiating and managing its regular review, in consultation with source data agencies
9. Principle 3: Integrator’s Accountability – ABS experience For integration projects involving Census data, ABS is the nominated IA All data integration projects involving Census data are treated as a family of projects ABS is ultimately accountable for Census data integration projects An approval process was followed for the projects such as Indigenous Mortality and Enhancing Cancer statistics Proposals have been documented and are available on the ABS website
10. Principle 3: Integrator’s Accountability – ABS experience Privacy management included a Privacy Impact Assessment (in 2006), consultations with Privacy Commissioners, and only making use of name and address during Census processing period. Risks have been assessed, managed and mitigated. For example ABS is only developing a 5% Statistical Longitudinal Census Dataset, not a 100% dataset. Datasets are deleted once purpose of the project has been met Close consultation with other data custodians e.g. Registrars Expected benefits are clearly outlined in the ABS Information Paper Normal ABS governance processes are in place for outputs
11. Principle 2: Custodian’s Accountability Each responsible agency for source data: Must agree mechanisms to achieve adequate control and manage risk approach to their own situation. These mechanism may include the use of particular institutional arrangements with trusted institutions, the use of specified standards and audits against those standards, and the potential application of sanctions Will need to agree the nature of valid uses that can be made of the integrated datasets and the approval mechanism to be applied to applications to use the datasets, as well as any control mechanisms to be applied to such use
12. Principle 2 cont. Each responsible agency for source data: Will need to manage the potential increase in identifiability of data for which they are responsible when it is used in conjunctions with data from other sources. It will need to agree mechanisms by which it can assure itself that outputs from the data integration are not likely to enable the identification of individuals or businesses Will need to agree the final content of any new data integration proposal, or any material change to an existing data integration proposal as part of the approval process. They must be kept inform of, and agree, more minor proposed changes to existing proposals. When an agency does not agree to the use of its source data in a statistical data integration proposal, the data will not be included.
13. Principles for Statistical Data Integration endorsed Portfolio Secretaries meeting (now Secretaries Board) February 2010 Agreement to establish governance and institutional arrangements to give effect to the high level principles Developed by the Cross Portfolio Statistical Data Integration Committee Represented by portfolio agencies
14. Governance and Institutional Arrangements for Statistical Data Integration endorsed Secretaries Board October 2010 Cross Portfolio Statistical Data Integration Committee completed role New governance arrangements involving portfolio agencies established
15. Governance and Institutional Arrangements Cross Portfolio Data Integration Oversight Board Members: ABS, DHS, DoHA, FaHCSIA Secretariat Best Practice Guidelines Network of Survey Liaison Officers Register of data integration projects Integrating Authorities: Accreditation Process Education and Training
16. Oversight Board Terms of Reference Cross Portfolio Data Integration Oversight Board Members: ABS, DHS, DoHA, FaHCSIA Role provide strategic and collaborative leadership, support effective governance and help manage the risks of particular projects help manage the systemic risk associated with conducting multiple data integration projects endorse changes or additions to the overall environment e.g. the development of new general tools to support integration or safe access to integrated data.
17. Oversight Board Terms of Reference Cross Portfolio Data Integration Oversight Board Members: ABS, DHS, DoHA, FaHCSIA Activities review data integration projects which pose a significant level of systemic risk, advise on risk mitigation strategies, and review any adverse incidents of high public concern. approve a comprehensive set of guidelines describing best practice for data integration projects involving Commonwealth data establish an accreditation process to enable the endorsement of authorised and accredited Integrating Authorities with the capacity to deal with high risk data integration projects.
18. Integrating Authorities: Accreditation Process Authorisation HIGH RISK PROJECTS 1. Authorisation to receive identifiable data – by legislation or consent 2. Legal protections prohibiting the disclosure of identifiable data MEDIUM AND LOW RISK PROJECTS 1. Authorisation to receive identifiable data – by legislation or consent 2. Policy environment which provides support to ensure that no identifiable data is disclosed
19. Accreditation Process Integrating Authorities: Accreditation Process Self-Assessment IAs prepare report on criteria for accreditation Audit Independent party audits self-assessment Decision Oversight Board makes final decision on accreditation Publication of list of accredited agencies
20. Integrating Authorities: Accreditation Process
21. Interim Accreditation Criteria – ABS experience Availability of appropriate skills Appropriate technical capability Lack of conflict of interest Culture and values that ensure protection of confidential information and support the use of data as a strategic resource Transparency of operation Appropriate governance and administrative framework Ability to ensure secure data management IAs must demonstrate that information that is likely to enable identification of individuals or organisations is not disclosed to external users Adhere to separation principle? What expertise and experience does the agency have to undertake high risk data integration projects? What strategies are in place to acquire the necessary expertise? Does the organisation have secure IT infrastructure, including hardware and software systems, with the capacity to support the potentially large and/or complex files associated with high risk data integration projects? Does the agency have a compliance monitoring or regulatory function? Are data revision statements published? What are the institutional and project-specific governance arrangements for data linking? Program of audits to ensure continued security of data? Are there mechanisms in place to engage with stakeholders to maximise the usefulness of the statistics? Are details of governance arrangements published? How is safe access provided? Comply with Australian Government Protective Security Policy Framework? What framework is in place to conduct investigations and handle complaints? Are detail of data integration projects published? How does your agency demonstrate that it provides for valuable use of the data? What documentation and training is available to ensure staff have the appropriate skills and knowledge required in high risk data integration projects? Have staff undergone police checks prior to employment? Is the system designed so that access and changes to data can be audited by date and user identification? Does the system ‘footprint’ inspection of records and provide an audit trail? Is access to agency’s premises controlled? Is an appropriate culture and values embedded in a corporate plan/mission statement/policies etc? What other relevant material is published? Is the Internet gateway secured? Regularly reviewed by Defence Signals Directorate? Have staff been trained in requirements for protecting personal information and are they aware of policies regarding breaches of security or confidentiality? What IT support is in place for staff? Does the agency have an Information Security Policy and procedural plan?
22. Governance and Institutional Arrangements Cross Portfolio Data Integration Oversight Board Members: ABS, DHS, DoHA, FaHCSIA Secretariat Best Practice Guidelines Network of Survey Liaison Officers Register of data integration projects Integrating Authorities: Accreditation Process Education and Training
23. Best Practice Guidelines High Level Principles Confidentiality
24. Principle 6: Preserving Privacyand Confidentiality Operational, administrative and personal identifiers should be removed from datasets as soon as they are no longer required to meet the approved purposes of the statistical data integration project When identifiers need to be retained, eg for longitudinal studies, they should be kept separate from the integrated dataset The number of unit records and data variables to be included in an integrated dataset should be no more than required to support the approved purposes Eg 5% Statistical Longitudinal Census Dataset, not 100%
25. Principle 6 cont. The type of matching used should be chosen as the minimum needed to support the approved purposes, and the range of attributes used to establish a common identity should be the minimum necessary for the linking operation to succeed Eg probabilistic data linking rather than exact data linking Access to potentially identifiable data for statistical and research purposes, outside of secure and trusted institutional environments, should only occur where : legislation allows; it is necessary to achieve the approved purposes; and meets agreements with source data agencies Custodians need to advise on legislation
26. Principle 6 cont. Risks of indirect as direct identification should be carefully managed when data is disseminated outside secure and trusted environments, particularly of units with unusual characteristics. This management must take account of the potential increase in identifiability of one set of data when combined with another set. It might involve strict data use licensing conditions, reducing detail, perturbing data, or seeking the consent of the individual or business involvdto release potentially identifiable data, the last of these being most likely in the case of business data
27. Principle 6 cont. Once the approved purpose of the project is met, the related datasets should be destroyed, or if retained, the reasons for and necessity of retention documented, and a review process et up. If such a retention was not part of the initial approval process, re-approval of the decision to retain in required. Archiving of statistically integrated datasets should be restricted to confidentialised datasets.
28. High Level Principles www.nss.gov.au Roles and Responsibilities Scope Risks Authorisation Interim Accreditation
29. Where is the Journey going
30. Where is the Journey going Principle 1 : Treat data as a strategic asset
31. Where is the Journey going Principle 1 : Treat data as a strategic asset
32. Thank you