Gary Kratkiewicz BBN Technologies Geoffrey Knauth BAE Systems

1. Taming Development and Data Chaos in Military Logistics:The ICIS Experience With Lightweight Languages Gary Kratkiewicz BBN Technologies Geoffrey Knauth BAE Systems

2. ICIS Background • Web-based software system • Analyzes future combat and humanitarian operations • Measures ability of Defense Logistics Agency to satisfy wartime requirements for following items: • Fuel • Food • Replacement items (repair parts, clothing, medical items, etc.) • Identifies potential problems • When will I run out of stock? • What should I buy today for tomorrows needs? • Do I have adequate sources & transportation?

3. Accomplishments • Early Web-based DOD system • Saved over $100 million • By computing reduced inventory needs for fuel and food • Adopted as part of DOD doctrine • Months of calculations reduced to hours • What-if capability leads to better decisions • Short analyses expanded role to crisis action • Used in military operations to: • Assess readiness • Compute operational requirements

4. Development Notes • Started in 1994 • Grew from small prototype to large system in 4 phases • User interface moved to Web in stages from 1996 to 1999 • Approximately a third of code is in Lightweight Languages • Significant data issues

5. Data, Data, Data • Large amounts of data • 800-1000 input files; 200 different formats • Gigabytes of input and output per analysis • Hundreds of analyses “active” • Data management issues • Received from many different agencies • Regular and irregular updates • Meandering formats • Duplication between analyses: • Computations • Intermediate & output files

6. 1. Prototype ICIS – 1994/1995 • Original prototypes completely coded in Tcl/Tk • Vague, rapidly changing, and conflicting requirements • Less than 1 MB of data • Ease of developing/tailoring lightweight prototypes greatly contributed to initial success

7. 2. Production ICIS – 1995/1996 • Lots of data: approaching 1 GB • Addition of (mostly) well-specified computational modules • Real use with classified data in mid-1996 • Mix of languages: • Tcl/Tk for GUI and glue for module execution • Perl for data preprocessing • C/C++ for computational modules

8. 3. Web ICIS – 1996-1999 • Address remote access issues • Moved to Web in stages • On server: • Tcl/Tk – glue for module execution • Perl – data preprocessing • C/C++ and Java – computational modules • Perl and C++ – dynamic generation of HTML and JavaScript • In browser: • HTML – static GUI elements • JavaScript – form-based GUI elements • Java AWT applets – interactive GUI elements

9. 4. Redesigned Web ICIS – 2001/2002 • Improve GUI and better manage data • Data approaching 1 TB • On server: • JScheme – managing data & module execution (sophisticated “glue”) • Perl – data preprocessing • C/C++ and Java – computational modules • In browser: • HTML – static GUI elements • JavaScript (much less) – simple GUI elements • Java AWT applets – more complex GUI elements • Java Swing applets – most complex GUI elements • JScheme for understanding scenario

10. Source Code Evolution

11. Summary of Lightweight Language Use in ICIS • Rapid prototyping • LL replaced by HL: completely for computations, partially for GUI • Highly iterative, innovative design & development • LL used in Data Set Manager, patent pending • Processing of meandering data • Gluing together heavyweight modules • LL replaced with more sophisticated LL • Building little languages • Test code

12. JScheme — Application Flexibility • Interpreter for debugging • Scripts for component functionality • Can be a server for execution management of multiple tasks • Example implementation: BBN’s Data Set Manager

13. User Asks ICIS Questions Example: “How much Fuel do I need?” • User interaction with ICIS is web-based • User defines composite scenario • User specifies results of interest • ICIS runs the assessment • Numerous modules might be run • It might take a while… • Results appear in browser

14. An Example Small Assessment

15. Problems • Obvious approach of using shell scripts and files doesn’t cut it here • Too much data • Many computations recomputed needlessly

16. A Full ICIS AssessmentDoes Many Computations The readable version of this graph takes up a large wall.

17. Data Set Manager • Data is described by features, not file names • A module spec describes • Inputs and outputs of a module • Process relationships with other modules • Modules can be implemented in any language • A module spec is applied to a composite scenario to produce an execution script • The script is executed • All computations are memoized

18. Execution Management Needs Dynamic Capabilities Data Set Manager takes full advantage of Scheme

19. Implementing the Data Set Manager in JScheme was fairly easy because: • Quasiquote was used to go from module spec to execution script • A full language was required to write module specs • Executing the script was just walking a tree structure

20. Murphy Strikes Start Goal Mistrial!

21. MVC: JScheme = M+C, Java = V Draw graph from knowledge of assessment state Generate User Options Show Details Try Again

22. Data Set Manager — Conclusion • Core implementation is 3804 lines of JScheme code • Supported by 287 lines of Perl, 102 lines of SQL • ICIS Module specs add another 1853 lines of JScheme • Current APIs for other languages are: • 688 lines of C • 496 lines of Perl • 356 lines of Java (1 class) • 120 lines of JScheme • BBN has applied for two patents