John Hoven jhoven@gmail linkedin/in/johnhoven TRADOC Mad Scientist Speaker Series

1. Unknown unknowns:Intelligence without intelligence requirements, operations without operational objectives John Hoven jhoven@gmail.com www.linkedin.com/in/johnhoven TRADOC Mad Scientist Speaker Series October 2015

2. John Hoven brief bio I learned operational qualitative analysis investigating mergers in the Antitrust Division of the US Justice Department. Unknown unknowns are a constant feature of these investigations, because business relationships are as nuanced and baffling as any human relationship. We have three weeks to focus in on the key issues through industry interviews, and then dig deeply enough to decide whether to recommend a full-blown investigation.

3. Three Issues • The capability gap: unknown unknowns in one-of-a-kind situations (sample size of one) • The innovation: Operational qualitative analysis • More descriptive name: Operational Iterative Context-specific Causal Inference (O-ICCI) • Current activity: ”Fuzzy front end of discontinuous innovation”

4. Unknown unknowns Help wanted One-of-a-kind situations  Unique context  Human relationships  Innovation leaps Operational Iterative Context-specific Causal Inference (O-ICCI) • Our goal: Build on “Every Soldier a Sensor” with new capabilities for rapid, in-depth understanding of cause-and-effect in ill-defined, one-of-a-kind situations where unknown unknowns are likely to be present and important.

5. Operational Iterative Context-specific Causal Inference (O-ICCI) • Context-specific means that the operational setting is a one-of-a-kind situation where unknown unknowns are likely to be present and important. Query: We’re focusing on locally nuanced civil-military operations. What other specific contexts should we examine? Query: Is this a setting where human cognition clearly outperforms computer analysis?

6. Operational Iterative Context-specific Causal Inference (O-ICCI) • Causal Inference means that analysis is not just descriptive. Hypotheses evolve and are tested before, during, and after an operation. • Sample size of one => causal inference does not rely on generic models, statistical inference, or control groups

7. Operational Iterative Context-specific Causal Inference (O-ICCI) • Operational means the tempo and purpose is operational rather than for research

8. Operational Iterative Context-specific Causal Inference (O-ICCI) • Iterative means fast iteration between information-gathering, analysis, and action. • Someone in the soldier’s unit constantly analyzes what is being learned, and directs the search for new information. • Simple, fast operational experiments are designed for rapid learning. Operational objectives are discovered and evolve over the course of the operation.

9. Unknown unknowns Help wanted One-of-a-kind situations  Unique context  Human relationships  Innovation leaps Operational Iterative Context-specific Causal Inference: Basic framework Learning People 1a. Fast feedback 2. Build trusted relationships 1b. Fast focus 3. Hypothesis testing for one-of-a-kind situations Solution

10. Basic framework (1a,1b): Fast feedback, Fast focus Design Act Test Unknown unknowns Help wanted Train One-of-a-kind situations  Unique context  Human relationships  Innovation leaps Learning People • 1a. Fast feedback • Conversational interviewing • Simple, fast experiments Do these concurrently, not sequentially 1b. Fast focus 2. Build trusted relationships Then go deep, and expand out 3. Hypothesis testing for one-of-a-kind situations Solution

11. Unknown unknowns Help wanted One-of-a-kind situations  Unique context  Human relationships  Innovation leaps Basic framework (2): Build trusted relationships We build relationships to achieve our goals We investigate relationships as one-of-a-kind situations Learning People • 2. Build trusted relationships • Shared interests • Shared personal interests and values • First partner • First user Solution

12. Basic framework (3): Hypothesis testing A does X B does Y AND / OR Confirming Evidence Contributing Cause Z Disconfirming Evidence AND / OR Unknown unknowns Help wanted Effect One-of-a-kind situations  Unique context  Human relationships  Innovation leaps Learning People • 3. Hypothesis testing for one-of-a-kind situations • Based on what is known (evidence and theory), articulate a cause-and-effect hypothesis – a chain of causes leading to an effect. Do this for the problem diagnosis, and then for a candidate remedy. • Search for new evidence and theory that confirm or reject the hypothesis. Revise or replace it. • Keep doing that, again and again, as you iterate toward a solution. Solution

13. Fuzzy front end of discontinuous innovation • Breakthrough … innovations (also called 'radical', 'discontinuous‘, or 'disruptive' innovations) are innovations which overcome problems that have not been solved, or even not been recognized before. (Khanh. Radical Innovation and Open Innovation. 2010)

14. Fuzzy front end of discontinuous innovation • In the case of discontinuous innovations … individuals perform a boundary-spanning function by identifying and understanding emerging patterns in the environment, with little or no direction from the organization… Consequently for discontinuous innovations, information search and related problems/ opportunities are unstructured and at the individual level during the fuzzy front-end. (Reid and de Brentani, "The fuzzy front end of new product development for discontinuous innovations,” 2004)

15. Potential innovator • Justice & FBI • State • USAID • Defense • Contractors • Intelligence • Operations Innovation Broker / Boundary Spanner “Might this be of interest to you?” • Tactical-Level Army Operations Fuzzy front end of discontinuous innovation • “Outside the box” • Operational qualitative analysis • Business-school social science Query: How might the US military be proactive in searching for discontinuous innovation opportunities? How might it support networking efforts of innovation brokers/boundary spanners?

16. Fuzzy front end of discontinuous innovation Basic O-ICCI for one context – e.g. civ-mil ops to facilitate health clinic or local agrifood trading Enrich the concept with non-basic elements (next slide) Generalize to “types” of similar contexts More specific context Concept Development Concept Legitimization 1a. Fast feedback 1b. Fast focus 2. Build trusted relationships 3. Hypothesis testing for one-of-a-kind situations

17. Fuzzy front end of discontinuous innovation Non-basic elements yet to come: • Prior knowledge for a specific context • Probable problems and opportunities • Probable factors that contribute to a problem or solution • Probable sources of relevant information • Collaboration within a teamseparated by time or distance • Support softwarefor operational qualitative analysis • Database managementfor operational qualitative analysis • Prior knowledge from a similar context • Search strategies for relevant, easily-understood theory and evidence • Collaboration across levelsof military hierarchy

18. Call to action: Might you have an interest in a “community of practice” in … • Strategies to support the fuzzy front end of discontinuous innovation, or • Army project in Operational Iterative Context-specific Causal Inference (O-ICCI) “A community of practice is a unique combination of three fundamental elements: a domain of knowledge, which defines a set of issues; a community of people who care about this domain; and the shared practice that they are developing to be effective in their domain.” (Wenger et al. Cultivating Communities of Practice. 2002)

19. To learn more: • Hoven & Lawton (forthcoming 2015) "Locally Nuanced Actionable Intelligence: Operational Qualitative Analysis for a Volatile World” https://app.box.com/s/sl6a56fsgr3ywhvzkopt • Reid & de Brentani (2004) “The fuzzy front end of new product development for discontinuous innovations” http://eprints.ubishops.ca/54/1/Reid%20and%20de%20Brentani%20JPIM2004.doc • Rubin & Rubin (2011) Qualitative interviewing • http://books.google.com/books?id=T5RDmYuueJAC&printsec=frontcover#v=onepage&q&f=false • Stern et al. (2012) “Broadening the range of designs and methods for impact evaluations” https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67427/design-method-impact-eval.pdf • Vermaak (2012) "Facilitating local ownership through paradoxical interventions" http://hansvermaak.com/wp-content/uploads/hans-vermaak-facilitating-local-ownership-paradoxical-intervention.pdf • Wenger et al. (2002) Cultivating communities of practice • https://books.google.com/books?id=ewmCX8AUvtcC&printsec=frontcover#v=onepage&q&f=false