Enhancing Defence International Training: Measuring Success & Impact

1. Defence International Training Centre Brief to the Bureau of International Language Co-operation Conference - Tartu

2. Return on Investment: How do we know we have won the match?

3. Defence Co-operation Program – Context “An essential element of Defence’s international engagement is the provision of education and training in Australia. Provided by Defence to members of international military and other Government agencies, International Education and Training (IET) is a useful tool for engaging foreign countries and is an integral part of the development of relationships. It serves Defence’s International Engagement (IE) policy objective of developing personal and professional relationships between Defence members and their international counterparts”. Defence International Engagement Manual

4. Defence International Training Centre – Context&Mission Mission: To contribute to Defence’s international engagement objectives as described in the WP2016 by providing training and support through the Defence Cooperation Program Role: To provide training and support that enhances Defence cooperation and cross-cultural awareness between members of Australia’s Defence Organisation and our international colleagues. International engagement will become an integrated core function across the entire Defence portfolio, aligned with the Strategic Defence Objectives. The Defence Cooperation Program, currently providing defence assistance to 28 countries, will be enhanced to build the confidence and capacity of our important regional partners. The ADF will participate more regularly in multinational exercises and the overseas presence of Defence personnel will be gradually increased over time. p.22 2016 Defence White Paper

5. Defence International Training Centre – Command Intent  Increasing our outreach into the region  Improving the student experience of Australia and the ADF  Ensuring we have the right mix of people & resources to meet our objectives

6. Defence International Training Centre – So how do we measure success? • Putting together the intent behind the Defence Co-operation program, DITC’s mission and the CO’s command intent we have these areas of focus that can provide a starting point to ascertain return on investment: • Developing personal and professional relationships between Defence members and their international counterparts • International engagement will become an integrated core function across the entire Defence portfolio • The Defence Cooperation Program, currently providing defence assistance to 28 countries, will be enhanced • The ADF will participate more regularly in multinational exercises • The overseas presence of Defence personnel will be gradually increased over time. • Increasing our outreach into the region • Improving the student experience of Australia and the ADF • Ensuring we have the right mix of people & resources to meet our objectives • ……………………How do we convert these areas of into measurable metrics? Can we?

7. Defence International Training Centre – What do we have control over? • We should measure only those areas that DITC can reasonably influence: • Developing personal and professional relationships between Defence members and their international counterparts • International engagement will become an integrated core function across the entire Defence portfolio • The Defence Cooperation Program, currently providing defence assistance to 28 countries, will be enhanced • The ADF will participate more regularly in multinational exercises • The overseas presence of Defence personnel will be gradually increased over time. • Increasing our outreach into the region • Improving the student experience of Australia and the ADF • Ensuring we have the right mix of people & resources to meet our objectives            No! We can’t really influence this measure Maybe! We can influence parts of this measure Yes! We can definitely influence this measure

8. Defence International Training Centre – So what can we really measure?  • Included Measure - 1.0 Ensuring we have the right mix of people & resources to meet our objectives • 1.1 What percentage of our annual budget bid did we receive? • 1.2 What are our vacancy rates for positions across the school? • 1.3 How many programs or courses were cancelled due to lack of resources? • 1.4 What sort of feedback are we getting on the quality of our staff? • Included Measure - 2.0Improving the student experience of Australia and the ADF • 2.1 What sort of feedback are we getting on students’ experiences with staff? • 2.2 What sort of feedback are we getting on the programs we conduct? • 2.3 What sort of feedback are we getting on the quality of the food we serve? • 2.4 What sort of feedback are we getting on the quality of our accommodation?             No! We can’t really influence this measure Maybe! We can influence parts of this measure Yes! We can definitely influence this measure

9. Defence International Training Centre – So what can we really measure? • Included Measure - 3.0 Increasing our outreach into the region • 3.1 What were our student numbers at DITC for the previous 12 months? • 3.2 How many students attended Mobile Training Team delivered courses in the last 12 months? • 3.3 How many students attended online delivered courses in the last 12 months? • 3.4 How many students accessed our online portals? • 3.5 How many cultural awareness briefs did we deliver to ADF personnel in the last 12 months       • Excluded Measure - The overseas presence of Defence personnel will be gradually increased over time • Mostly out of DITC control • Aspects that are in our control subsumed into measure 3.2 above     No! We can’t really influence this measure Maybe! We can influence parts of this measure Yes! We can definitely influence this measure

10. Defence International Training Centre – So what can we really measure? • Excluded Measure - The ADF will participate more regularly in multinational exercises • ADF participation in multi-national exercises is beyond DITC control • Excluded Measure - The Defence Cooperation Program (DCP), currently providing defence assistance to 28 countries, will be enhanced • DITC does not set the strategic course for DCP • Aspects of the program that DITC does have control over can be measured at measures 1.0. 2.0 and 3.0 • Excluded Measure - International engagement will become an integrated core function across the entire Defence portfolio • DITC does not have strategic oversight of the Defence Portfolio • DITC can assist this to happen through measure 3.5       No! We can’t really influence this measure Maybe! We can influence parts of this measure Yes! We can definitely influence this measure

11. Defence International Training Centre – A recap: What we have so far • Included Measure - 1.0 Ensuring we have the right mix of people & resources to meet our objectives • 1.1 What percentage of our annual budget bid did we receive? • 1.2 What are our vacancy rates for positions across the school? • 1.3 How many programs or courses were cancelled due to lack of resources? • 1.4 What sort of feedback are we getting on the quality of our staff? • Included Measure - 2.0Improving the student experience of Australia and the ADF • 2.1 What sort of feedback are we getting on students’ experiences with staff? • 2.2 What sort of feedback are we getting on the programs we conduct? • 2.3 What sort of feedback are we getting on the quality of the food we serve? • 2.4 What sort of feedback are we getting on the quality of our accommodation? • Included Measure - 3.0 Increasing our outreach into the region • 3.1 What were our student numbers at DITC for the previous 12 months? • 3.2 How many students attended Mobile Training Team delivered courses in the last 12 months? • 3.3 How many students attended online delivered courses in the last 12 months? • 3.4 How many students accessed our online portals? • 3.5 How many cultural awareness briefs did we deliver to ADF personnel in the last 12 months

12. Defence International Training Centre – A recap: What we have so far 2.0 Student Satisfaction of: 1.0 People and Resources: 2.1 Standard of Staff - 100% 1.1 Budget allocation - 92% 1.2 Vacancy Rate – 11% 2.2 Courses and programs – 88% 1.3 Courses cancelled – 0% 2.3 Standard of accommodation – 89% 2.4 Standard of meals – 73% DITC Key Performance Indicators 3.0 – Website Outreach 3.0 Outreach 67 www.ditc.net.au 3.2 MTT Attendance 182 3.5 Attendance by ADF personnel at Cultural Awareness Briefs www.ditc.gov.au 3.0 - Outreach by Country

13. Defence International Training Centre – Are we missing the point? The measures may be well developed and the data may be accurate but remember the reason DITC exists: Developing personal and professional relationships between Defence members and their international counterparts And our mission statement……. To contribute to Defence’s international engagement objectives by providing training and support through the Defence Cooperation Program How do we measure personal and professional relationships? How do we measure the richness of International Engagement? Quantitative measures can never really measure these things. ….and what about language outcomes?

14. Defence International Training Centre – Stories as Measurement

15. Defence International Training Centre – One Last Thing ‘Traditional’ quantitative measures including survey results and measures of website hits * Australian Defence Force English Language Profiling System

16. Questions and Discussion