Martha: a next generation testable language

1. World’s worst title Martha: a next generationtestable language Not a “language” but support tools for “what else” queries Tim Menzies West Virginia Universitytim@menzies.us

2. Problem • The unknown knowns • Factors that have been modeled separately, • but never studied in conjunction • Leveson: • common cause of software failure • components that are known to fine in isolation but failing when combined due to an unknown interaction. • Brian O’Conner: • Columbia incident- pre-launch foam-strike studies • Did not study a critical combination of factors

3. Hypothesis The “enough” hypothesis: • Early in lifecycle we know enough to find enough problems. “Enough” : • models exist or can be created for enough of the components • operational constrains known • system goals understood. • goals can be converted into oracles • behavior can be sampled, fast enough The “funnel” hypothesis (a.k.a. “low hanging fruit”) • Within a system of decisions, there are a few “master” decision and many “slaves” • It is practical to find those master decisions Faster, better arguments: • Sample, learn, restrain • Address the master decisions first, • Makes the slave decisions superfluous Procedural encodings + range functions + memoing • More usable modeling langauge for most folks

4. Not: “what is” a = f(b,c,d,…) Not: “what if”: For a few scenarios.. jiggle “b,c,d,…” see how “a” changes Rather: “what else” Memoed functions (with rollback) Hedges (fuzzy and-ors-nots) Not data types, but range functions Massive what-ifs (generates lots of output) An oracle scores each run A data miner: finds ranges of “b,c,d…” that select for poor scores Repeat till happy :-) Approach use hedges models contain use are range definitions functions are exercise monitored by restrains make simulations memos treatments scored-by Aha! used-by oracle learns data miner used-by

5. Importance/ Benefits The trouble isn't what people don't know; it's what they do know that isn't so. -- Will Rogers My ideal computer never says “hello world”;rather, it says “hello, that’s strange”. -- Tim Menzies Q1: Cold causes o-ring incidents? A1: No, so launch Q2: Heat blocks o-ring incidents? A2: Yes, don’t launch

6. Relevance To NASA • Increasing use by NASA: • Rapid development of mission concepts • E.g. • JPL’s TeamX • GSFC’s IMDC • How to audit TeamX or IMDC sessions? • How ensure that, in the heat of the moment, critical features of a mission are not over-looked? • Or worse, accidently over-written by other decisions?

7. CHEM Accomplishments • Taught Martha to graduate students • “just a bunch of functions” • Case studies • FLEET: a fleet of autonomous agents • CHEM: chemical production facility • COST: software cost model • Tasks parameter, fault simulation, optimization, tuning, localization, diagnosis, configuration • CHEM • COST • FLEET • Lessons: • “treatments” support a wide range of tasks • Surprisingly fast learning of controllers • Automating a “principle-investigator-in-a-box” COST FLEET

8. Next Steps • Better incremental data mining • 100 to 1000 times faster • Anytime algorithm • Smaller memory footprint • Better sampling policies • ??? Use internals of data miner to define what to try next • Release of a tool kit • More case studies • SMART (small ARRT) • TEAM-X • ECS support • with James Kiper, Miami Uni; and Leila Meshkat, JPL • Other offers?