Whitney, Bradley, & Brown (WBB) Consulting, Reston, Virginia, 28-30 October

1. Military Operations Research Society (MORS)Cyber Analysis WorkshopOnline Plenary Session21 October 2008 Whitney, Bradley, & Brown (WBB) Consulting, Reston, Virginia, 28-30 October Government Senior Leader virtual review, 30 October Registration Information: MORS Office (703) 933-9070 or www.MORS.org

2. Defense Connect Online (DCO) • Connections for this meeting • DCO https://connect.dco.dod.mil/cyberanalysis with audio through computer speakers • Audio backup, call 877-206-5884 with code 547836 for teleconference • Anyone not connected? • If no audio, respond in chat pod • If no visual, speak up or call teleconference

3. DCO Rules of Engagement (ROE) • Many individuals online today • We want all of your inputs! • Most of you can not talk • DCO has the communication capabilities—this is our ROE • Chat: any time to everyone or an individual • Comment/Question pod: write any time, addressed at planned periods • Suggestion pod: write any time, not reviewed in this session • Attendee/status: indicate “have a question” to interrupt • Other DCO capabilities • Agenda pod • Polls will occur later in this session

4. Agenda • STRATCOM/J5 Address • AF/A9 Video • Review Workshop Approach • Start with M&S Requirements • Tracks develop challenges and recommendations • Discipline Groups improve and add to recommendations • Senior Leaders review and guide • Summary of M&S Requirements • Tracks Plans • Discipline Groups • Solicit intended participation • MORS Opportunities

5. Dr. Mark A. Gallagher Secretary, MORS Deputy Director for Resource Analysis, HQ USAF/A9R Work Shop Chair’s Welcome • Thanks!!! • Dr. Henningsen, Headquarters Air Force A9, our official workshop co-sponsor • Ms. Susan Shekmar, OSD NII, our official workshop co-sponsor • Mr. Moore, President of WBB Consulting for hosting our meeting 28-30 October at your wonderful facilities • Mr. Cares, Alidade Incorporated for providing SharePoint site for planning • New initiatives • Conducting both unclassified and classified tracks so uncleared individuals can contribute to solving these national security changes • Using wikis and these online sessions so you can guide the agenda and discussions • Vetting recommendations to senior government leaders during workshop • Request your active participation! • Questions?

6. Thoughts from USSTRATCOM • Ability to operate, defend and fight in and through cyberspace is analogous to where Air Power was during the interwar period • The cyber domain simultaneously intersects every other domain • Understanding the cyber threat and improving our analytic approaches and techniques for cyberspace are key challenges • I look forward to hearing the result of the workshop. Mr. Michael Elliot, SES, Deputy Director, Plans and Policy (J5A), U.S. Strategic Command

7. Video from our Workshop Co-Sponsor • Her views on the needs for cyber analysis in this video • Analytical techniques with capabilities similar to operations and acquisitions in other areas • Cyber offense is more difficult than the most challenging kinetic actions, combating terrorists • Cyber defense is more challenging than preventing crime • Cyber is crucial to our national security Dr. Jacqueline R. Henningsen, SES Director for Studies and Analyses, Assessments and Lessons Learned, Headquarters U.S. Air Force MORS Sponsor and Fellow Cyber Analysis Workshop Co-Sponsor

8. Workshop Goals and Objectives • Goal: Advance the analytical foundation for cyber actions for national security • Objectives • Understand the cyber threat • Improve analytical approaches that support cyberspace operations • Address cyber analysis including modeling & simulation requirements • Critique present and proposed analytical approaches and techniques • Prepare recommendations to improve cyber analysis • Out brief senior government leaders on recommendations • Write workshop report with recommendations and justifications

9. Workshop Leadership • Staff Functions • Security and Facilities (Greg Ehlers) • Virtual Collaboration (Scott Hamilton, Todd Hamill) • Physical Meeting (Jeff Cares) • Taxonomy (Bob Koury) • WBB Site Coordinators (Dennis Baer and Tim Hope) • Senior Leader Coordination (Greg Keethler) • Cyber Modeling and Simulation Requirements (Chris Jeffrey) • Workshop Bulldog (Mark Reid) • Matrix participation between tracks and discipline groups • Tracks – desire co-leads external to DoD for unclassified tracks • Cyber Environment (Greg Larsen) • Cyber networking for situation awareness and C2 (Len Popyack andPat Allen) • Cyber vulnerabilities, protection, defense (Bud Whiteman) • Cyber deterrence (Pat McKenna, Terry Pudas) • Cyber exploitation and offensive operations (Bob Morris, Jim Pickle, Linda Namikas) • DoD Web-Policy Impacts on Cyber Operations (Dennis Murphy, Jason Dechant) • Discipline Groups • Optimization (Lee LehmKuhl) • Decision Analysis (Hunter Marks, Rafael Matos) • Simulation (Sandy Thompson, Laura Nolan) • Computer Science (Jarret Rush) • Social Sciences (Deanna Caputo) • Keynote Speaker: Dr. Ronald C. Jost, Deputy Assistant Secretary of Defense

10. Workshop Organization • Tracks have lead on addressing modeling and simulation requirements • Discipline (academic specialty) groups suggest approaches to track challenges • Physical meeting is primary track sessions with discipline group at end of day • The workshop is “working” in participants will develop approaches to meet the three sets of Cyber Modeling and Simulation requirements

11. Physical Meeting Schedule 9 Ninety-Minute Sessions: 6 for Tracks and 3 for Discipline Groups

12. Track and Group Interface • Tracks must write summary for end-of-day • Any agreed challenges, recommendations, and actions • Issues to be addressed • Specify classification with unclassified version, if possible • Discipline Groups • Review tracks summaries • Write specific recommendations with justification • May develop own challenges, recommendations, and actions • Specify classification with unclassified version, if possible • Attendees participate in both a track and a discipline group • Track is in-depth focus within an area • Discipline group provides overview and a different perspective of the challenges

13. Recommendation Format • Challenge: Express current limitation or problem that analysis can address • Recommendation: • Describe actions to implement recommendation (samples types below) • Need for organization cooperation between … • Need funding for … • Improve testing by … • Characterize each recommendation • Priority (critical, important, needed, enhancing) • Urgency (immediate < 1 yr, near-term 1-3 yrs, long-term >4 yrs) • Resources (inexpensive < $1M, medium cost $1M-$10M, expensive > $10M) • Senior leaders will assess • Priority (critical, important, needed, enhancing, no value) • Urgency (immediate, near-term, long-term, not needed) • Feasible (likely, probably, challenging, not possible)

14. Planning Tool Improvement • Challenge: Need improved planning tools for cyber operations • Recommendation: require planning estimates • Require effectiveness estimates with indication of technique accreditation status • All approval packages • Cyber tests and experiments • If planning technique is not accredited, capability provider must submit it to IO JMEM for review • STRATCOM lead annual review of accredited planning models and report to OSD OT&E and NII • Characterization (Important, Long-Term, Challenging) Notional Recommendation Only

15. Cyber Tools Classification • Challenge: Many cyber tools may be over classified as SAR/SAP • Recommendation: Develop and implement a risk assessment decision aide to guide tool classifications • Commission team of analytical organizations to propose approaches and develop prototypes • Arrange independent analytical review of proposals, document strengths and weakness, and recommend classification decision aide • Mandate application of decision aide in classifying tools • Characterization (Important, Long-Term, Challenging) Notional Recommendation Only

16. Workshop Report • Workshop will produce a worthwhile written report • Makes current analysts aware of other initiatives • Brings new analysts up to current capability • Provides recommendations to senior leaders on how to proceed • Report Content • Summarizes of background • Identifies issues • Assesses current analysis approaches • Evaluates enhancements or alternative approaches • Recommends steps to develop or implement improved analytical approaches • Tracks and Discipline Groups need to write their good ideas!

17. Cyber M&S Requirements Sources • ASD(NII) “determine the M&S requirements for EBO in cyberspace” • 72 requirements (broad analytical tasks) • IO JMEM COCOM inputs • 20 requirements (more tactical requirements) • Air Force Agency for Modeling and Simulation (AFAMS) • 5 organizations brainstorming thoughts

18. Cyber M&S Requirements These requirements are mostly general analysis tasks.

19. Unclassified and Classified Tracks • Cyber Environment • Dr. Greg Larsen, IDA • Cyber Situational Awareness and Command and Control • Dr. Len Popyack, AINFOSEC, and Dr. Pat Allen, JHU/APL • Cyber Vulnerabilities, Protection, and Defense • Bud Whiteman, BAH, USSTRATCOM & IO JMEM • Cyber Deterrence • Pat McKenna, USSTRATCOM, and Terry Pudas, NDU • Cyber Exploitation and Offensive Operations • Col Jim Pickle, HQ AF GCIC; Col Bob Morris, 67 NWG/CC; Linda Namikas, ACC 346 Test Squadron • DoD Web-Policy Impacts on Cyber Operations • Prof. Dennis Murphy, Army War College; Jason Dechant, IDA Classified sessions will be limited to Secret No Forn

20. Cyber Environment Track • Lead: Dr. Greg Larsen, Institute for Defense Analyses (IDA) • Track classification will be Unclassified • Cyberspace is the emerging center of gravity for global interactions • Critical issues have many implications and include: • The space is “constructed” not natural • The effective use of cyber capabilities depends on agile adaptation to changes in the environment • The increasingly strong dependence of other capabilities operating in other environments on the cyber environment complicates the M&S challenges • Cyber warfare cannot and should not be equated to information warfare or computer networks warfare • Cyber M&S must incorporate human behavior into operations in, through, and from cyberspace • This track is focused on this wide array of issues and frameworks that determine the credibility, relevance and significance of cyber analyses. • Questions?

21. Cyber Situational Awareness (SA) and Command and Control (C2)Track • Leads Dr. Len Popyack, AINFOSEC, and Dr. Pat Allen, JHU/APL • Track classification is Unclassified • Purpose: Identify issues and recommend actions for analysis of cyber support to C2 and SA • Topics: • Broad issues • Scalability & Applicability, Analysis of Cyber Support • Domains • Allegiances and Sides, Instruments of National Power, Timeframes • Technical Topics • Connectivity, Content & Measures, Security, Visualization, Tools • Other topics not listed above • Questions?

22. Cyber Vulnerabilities, Protection, and Defense Track • Lead Bud Whiteman, BAH at USSTRATCOM, IO JMEM • Track classification is SECRET/No Foreign Nationals • Our nation, including forces contributing to national security, rely on cyber systems and services  • What are the vulnerabilities of these systems?  • How do we protect and defend them?   • This track focuses on analytical methods to address these questions • Describe the capabilities of current tools • Determine what is need to meet the requirements • Questions?

23. Cyber Deterrence Track • Leads Pat McKenna, USSTRATCOM, and Terry Pudas, NDU • Session classification is Unclassified • Track topics • How is deterring cyber similar/different from “traditional” deterrence? • Who is the actor (e.g., state, non-state, individual)? • Attribution vs. non-attribution vs. not attributable • Lack of precedents, red lines, and established declaratory policy • What analytic capabilities are required? • Across academic disciplines (Social sciences, OR, etc.) • What analytic tools exist? What are the analytic gaps? • War gaming deterring cyber issues • Is it a valuable approach? • What has been done in the past? • What are the “best practices”? • How do you assess actions to deter cyber? • What is the contribution of cyber defense to deterring cyber? • How are 2nd (nth) order implications represented? • Questions?

24. Cyber Exploitation and Offensive OperationsTrack • Leads Col Jim Pickle, HQ AF GCIC, Col Bob Morris, 67 NWG/CC, and Linda Namikas, ACC 346 Test Squadron • Session classification is SECRET/No Foreign Nationals • Big Questions: • How can the US use cyber capabilities? • How can we plan and assess the effectiveness of these techniques? • Focus questions: • How is cyber similar/different from “traditional” exploitation/offensive actions? • What analytic and M&S capabilities are required? What analytic/M&S tools exist? • War gaming cyber conflict: How ? Is it valuable? What are the “best practices”? • How do you assess cyber offensive actions? What are meaningful metrics? • What are appropriate Cyber CONOPs? • Planned approach: • Overview of real-world ops • CNA JMEM Successes (TVM/WCM and Models) • M&S support needs from community • CONOP Development process • Metrics to support COCOMs and OPLANS • Questions?

25. DoD Web-Policy Impacts on Cyber OperationsTrack • Leads Professor Dennis Murphy, Army War College; Jason Dechant, IDA • Track classification will be Unclassified • Current and future war consider battle of ideas on par with battle of arms • Internet is crucial • Routine business and communication • Message delivery in strategic communication • Defend or Attack in the Cyberspace? • Defending the network for our use • Use the network offensively to get out our message proactively • Managing risk and achieving balance • Current policy applies centralized control and execution to protect the networks • Decentralized execution allows for proactive and reactive speed to send the message • This track is focusing on analysis approaches that can help the government implement balanced policies in support of cyberpower. • Questions?

26. Unclassified and Classified Discipline Groups • Optimization • Dr. Lee Lehmkuhl, MITRE • Decision Analysis • Hunter Marks, USSTRATCOM • Rafael Matos, WBB • Simulation • Dr. Sandy Thompson, PNNL • Laura Nolan, JHU/APL • Computer Science • Jarret Rush, MITRE, • Social Sciences • Dr. Deanna Caputo Classified sessions will be limited to Secret No Forn

27. Optimization Discipline Group • Focus on identifying contributions of optimization techniques to determine best Courses of Actions (COAs), potential vulnerabilities effect points, and resource tradeoffs arising across all tracks.  • Questions: • What optimization techniques can provide insights? • How do we address the softer qualitative aspects? • How can we minimize limitations of optimization approaches? • What are the assumptions of approaches and the effects when those assumptions are violated? • How can post optimality analysis be used most effectively? • Discipline Group Lead: Dr. Lee Lehmkuhl One Group: Unclassified

28. Decision Analysis Discipline Group • Focus on identifying contributions of decision analysis to the analytical challenges arising across all tracks.  • Techniques: • Value-Focused Thinking • Decision Trees • Influence and Affinity Diagrams • Issues • Determine decision-maker and approach weights and ranks • When have conditions change sufficient to modify model weight? • Discipline Group Leads: • Unclassified: Rafael Matos, WBB • Classified: Hunter Marks, USSTRATCOM 2 Parallel Groups: Unclassified and SECRET/No Foreign Nationals

29. Simulation Discipline Group • Focus on identifying needs and contributions of simulation to the analytical challenges arising across all tracks.  • Questions: • What systems should be simulated? • What simulation research is required? • What groups (users) require simulations? • What types of simulation tools exist and what are good qualities? • Discipline Group Leads: • Dr. Sandy Thompson, PNNL • Laura Nolan, JHU/APL One Group: Unclassified

30. Computer Science Discipline Group • Focus on computer technologies affect our ability to conduct cyber operations in the areas for each track.  • Questions: • How are the technologies affecting our ability to analyze cyber operations? • Are the analytical approaches addressing the critical aspects of the technologies? • Discipline Group Lead: Jarret Rush, MITRE, supporting AFRL/XPC One Group: Unclassified

31. Social Science Discipline Group • Focus on social and human dimensions that affect our ability to conduct cyber operations across each of the track areas.  • Questions: • How do we address human impacts on effectiveness of cyber operations? Are the track approaches addressing or ignoring critical aspects? • Are the threats of hackers, terrorists, non-state actors, and states being adequately addressed? • How can behavioral influence be modeled for operational purposes vs. predictive vs. descriptive purposes – what is “good enough” in which situations? • How can we apply the findings and methodologies of research done in the social sciences (e.g., psychology, anthropology, sociology, behavioral economics, etc) to the cyber problem? • Discipline Group Lead: Dr. Deanna Caputo, MITRE One Group: Unclassified

32. Senior Leader Out brief • Senior Leaders to be Briefed Real-time on Workshop Recommendations • Approximately 10 senior government leaders from the analysis or cyber communities • Participate in person or via on-line DCO session • Review Recommendations from Tracks and Discipline Groups • Feedback solicited in four aspects: • Priority of the challenge/recommendation • Critical, important, needed, enhancing, no value • Time urgency of implementing recommendation • Immediate, near-term, long-term, not needed • Feasibility of the recommendation • Likely, probable, challenging, not possible • Additional insights on the challenge/recommendation • This real-time feedback will be incorporated into the workshop report • A distinctly new feature of a MORS Workshop Your senior leaders (SES, Generals, and Admirals) may participate! Contact gregory.keethler@lmco.com or (407) 356-3119

33. WBB Facilities • The physical meeting on 28-30 Oct • WBB Consulting facilities in Reston, Virginia • Sheraton Hotel is next door • The facilities are nice and spacious • Almost all rooms have internet capability • Senior Leader DCO session can be projected in the various rooms • Questions?

34. Workshop Security • Two levels of Classification • Unclassified with green badges • SECRET/NO FORN with red badges • Clear participants may transition from one classification to the other • All participants • Monitor discussions and stop individuals before they say to much • Check attendance in classified rooms • Be sure—protect our nation!

35. Audience Polling • Your workshop intentions? • Registered and attending • Planning on attending, however not yet registered • Still considering • Not attending • For potential attendees, what is your security clearance? • Uncleared • SECRET/NO FORN • For potential attendees, what is your preferred track? • List of six tracks • For potential attendees, what is your preferred discipline group? • List of discipline groups by classification

36. Audience Polling (continued) • Did the workshop use of sharepoint site affect planning? • Very beneficial, minor benefits, no significant impact, adverse impact, I was unaware of it • The workshop use of an unclassified wiki was? • Very beneficial, minor benefits, no significant impact, adverse impact, limited impact due to policy restrictions, I was unaware of it • The workshop use of SECRET wiki was? • Very beneficial, minor benefits, no significant impact, adverse impact, limited impact due to SIPRNET access, I was unaware of it • The workshop use of DCO sessions was? • Very beneficial, minor benefits, no significant impact, adverse impact, no opinion • I expect the workshop use of discipline groups, rather than a synthesis group, to be? • significant improvement, minor improvement, don’t care, probably, a detriment • I expect the workshop online feedback from senior leaders to be • significant improvement, minor improvement, don’t care, probably, a detriment

37. Join MORS • MORS has been supporting the Department of Defense (DoD) for over 40 years • Improving analysis • Networking experts • Enhancing professional development • MORS is expanding to national and international security • Added Department of Homeland Security as a sponsor • Initiated a dialog with NASA • View www.mors.org or call (703) 933-9070 for more details on the society, membership, and registration for this workshop • Workshop Chair: Dr. Mark Gallagher, (703) 588-6949 or Mark.Gallagher@pentagon.af.mil • Questions?

38. Wiki Sites • www.cyberanalysis.pbwiki.com • http://www.intelink.gov/wiki/MORS_Cyber_and_Networking_Workshop