Evolving the Assessment of the Effectiveness of IAEA Safeguards Implementation INMM

1. Evolving the Assessment of the Effectiveness of IAEA Safeguards ImplementationINMM Bruce W. Moran, Head Section for Effectiveness Evaluation Department of Safeguards

2. Starting with the State-Level Concept A holistic approach to safeguards implementation Applicable to all States with SG agreements Based on a comprehensive and continual State evaluation and a State-level approach, including a specific combination of safeguards measures for an individual State Executed through an annual implementation plan Considering the State as a whole provides the opportunity to take State-specific factors into consideration during all stages of safeguards implementation Implementation of the State-level concept is responsive to changes in the analysis, ensuring that safeguards conclusions remain soundly based and up-to-date 2012-01-24

3. Implementing the State-level Concept Enhancement of the implementation of the Agency’s State-level conceptrequires: • An expanded use of State-specific factors and a structured acquisition path analysis to define and prioritize State-specific technical objectives • State-level approaches that specify and provide options for Headquarters and in-field activities required to meet the technical objectives • Identification and selection of activities in the annual implementation plan • Linkage of the State evaluation process with conduct of safeguards verification activities 2012-01-24

4. Implementing the State-level Concept 2012-01-24 Evolving the safeguards system to be more: • Objectives As opposed to criteria driven; allows for based: customized State-level approaches to meet State-specific objectives • Information Use of all information, including State driven: factors, to determine objectives and conduct safeguards activities • Focused: At the State level and on issues of concern; putting resources where the risks are • Adaptable: Responsive to changes in information and analysis for conducting safeguards

5. Assuring Continued Effectiveness • Each DIR-SGOis responsible and accountable for • the implementation of safeguards for each State assigned to the Division and • the conclusions arising therefrom • Each DIR-SGO must ensure that • the implementation of the activities is consistent with procedures or guidelines established for such purpose • any changes to the activities achieve the objectives 2012-01-24

6. Assuring Continued Effectiveness • State Evaluation Reports • Prepared by the relevant State Evaluation Group • Reviewedby a Departmental Committee • State-Level Approaches • Prepared by the relevant State Evaluation Group • Reviewedby Departmental Sub-Committee • Annual Implementation Plan • Preparedby the relevant State Evaluation Group • Reviewed by the relevant Operations Division Director (DIR-SGO) • Quality Management System • Evaluate departmental processes and procedures and their implementation • Oversee the corrective action programme 2012-01-24

7. Assuring Continued Effectiveness State Evaluation State-Level Safeguards Approach Safeguards Processes and Safeguards Approaches Annual Implementation Plan and Safeguards Implementation 2012-01-24

8. SEE Responsibilities State Evaluation are cross-cutting State-Level Safeguards Approach Safeguards Processes and Safeguards Approaches ) Annual Implementation Plan and Safeguards Implementation 2012-01-24

9. Section for Effectiveness Evaluation (SEE) • SEE will operate as the central quality control tool of the Department for implementation and evaluation issues • SEE will perform the quality control activities on statements, surveillance, and seals, among others • SEE will perform an independent and in-depth assessment for each State of • safeguards verification activities • goal or objective attainment • safeguards conclusions and recommendations • SEE will lead assessments of selected State evaluation reports to ensure that the conclusions drawn are sound 2012-01-24

10. SEE Responsibilities • SEE will • Conduct selective evaluations of the effectiveness of the State evaluation process • Conduct selective evaluations of specific safeguards verification activities across a selection of States, such as • inspections, complementary access, design information verification and other safeguards activities, such as information analysis • Conduct selective evaluations of the effectiveness of safeguards measures, such as • monitoring activities (e.g., seals and surveillance), measurement activities, and information collection activities • Conduct selective evaluations of annual implementation plans • Conduct technical reviews of anomalies • Prepare the Safeguards Implementation Report and Data Evaluation Report 2012-01-24

11. SEE Evaluation Activities • Selection process will be random and information-driven • Evaluations will be performed through • Document and information reviews • Quality control tests • Observations and interviews • Periodic reports to DDG and Management Committee on findings • SEE findings identify where process audits and corrective actions may need to be initiated 2012-01-24

12. Evolving the Safeguards Implementation Report • Purpose of the SIR is to provide safeguards conclusions and information to enhance confidence in those conclusions • SIR should provide more analysis and explanation of the data • SIR is to be aligned with the State-level concept for safeguards implementation. • The report should demonstrate how the Agency is using all relevant information to draw safeguards conclusions • SIR is to be focused on safeguards implementation • Activities formerly presented as “safeguards strengthening” are now routine safeguards tools 2012-01-24

13. Summary • Safeguards effectiveness evaluation is independently performed by Departmental entities for all aspects of safeguards implementation • SEE quality control activities will assure that • Verification activities were correctly performed • Data was correctly generated and analysed • All information was considered in the State evaluation • Assurance is provided to States that safeguards activities met their objectives and support the safeguards conclusions 2012-01-24