The Complexity of Programming Models

1. The Complexity ofProgramming Models Grady Booch IBM Fellow

2. How much software exists in the world? • SLOC is a measure of labor (not of value) • Old code never dies (you have to kill it) • Some code is DOA • Some assumptions • 1 SLOC = 1 semicolon • Number of software professionals worldwide • %of software professionals who cut code • SLOC/developer/year • $100US/SLOC

3. Number of software professional worldwide

4. % of software professionals who cut code

5. SLOC/developer/year

6. New or modified SLOC/year and cumulative

7. Defense Weapon System Telecom Switch National Air Traffic Control System Commercial Compiler Embedded Automotive Software CASE Tool Large-Scale Organization/Entity Simulation Small Scientific Simulation IS Application Distributed Objects (Order Entry) Defense MIS System Enterprise IS (Family of IS Applications) IS Application GUI/RDB (Order Entry) Business Spreadsheet Dimensions of software complexity Higher technical complexity - Embedded, real-time, distributed, fault-tolerant - Custom, unprecedented, architecture reengineering - High performance An average software project - 5-10 people - 3-9 month duration - 3-5 external interfaces - Some unknowns & risks Lower management complexity - Small scale - Informal - Single stakeholder - “Products” Higher management complexity - Large scale - Contractual - Many stake holders - “Projects” Lower technical complexity - Mostly 4GL, or component-based - Application reengineering - Interactive performance Royce

8. Stakeholders and views • A given system is many things to many different stakeholders • End user • Customer • Sys admin • Project manager • System engineer • Developer • Architect • Maintainer • Tester • Other systems • Multiple realities, multiple views and multiple blueprints exist

9. Simplicity has different points of view • User • Simple metaphors and gestures • End-user programmer • Access to significant parts and flexible mechanisms for behavior • Architect • Common architectural patterns • Developer • Common design patterns and idioms • Tester • Access to significant parts

10. Simplicity has different points of view • Business analyst • Clear separation of rules • Database analyst • Purity of semantics • Security officer • Clear and perfectly executed policies • Administrator • Clear separation of components

11. In the presence of essential complexity, establishing simplicity in one part of a system requires trading off complexity in another

12. Creating the illusion of simplicity

13. Simplicity in languages • Tradeoff between primitiveness and convenience • Control structures • Tradeoff between explicitness and abstraction • Java garbage collection • Tradeoff between performance of development and performance of execution • VisualBasic • Smalltalk • Tradeoff between packaging for design versus packaging for development versus packaging for deployment • Beans • Services • Aspects

14. A programming model specifies the semantic universe within which the developer labors and is defined by the languages, platforms, tools, and best practices of that constellation

15. Web-centric programming model • Platforms • Linux • Apache • MySQL • J2EE • Best practices • Coding • Design patterns • Deployment • User interface • Accessibility • Internationalization • Security • Logging • Backup • Tools • Eclipse • Dreamweaver • Photoshop • ClearCase • ClearQuest • RSA • Portfolio Manager • RequisitePro • Tester • Languages • HTML • CSS • XSL • XML • SQL • RSS • Java • JavaScript • PHP • Flash • UML

16. Alternative programming models • Game developer • High performance computing • Command and control • Artificial intelligence • Domain-specific frameworks • … Handbook of Software Architecture, http://www.booch.com/architecture

17. A system is shaped by a myriad of design decisions by different stakeholders that work to balance the forces swirling around the system

18. Load Load Compression Tension Forces in civil architecture Kinds of loads - Dead loads - Live loads - Dynamic loads Avoiding failure - Safety factors - Redundancy - Equilibrium Any time you depart from established practice, make ten times the effort, ten times the investigation. Especially on a very large project. - LeMessuier

19. Forces on software

20. Why is software inherently complex? • Complexity of the problem domain • Difficulty of managing the development process • Fluidity of software • Fundamental challenges of discrete systems

21. Complexity of the problem domain • Volume of requirements • Presence of competing/contradictory requirements • Non-functional requirements that push the limits of software • Requirements churn • Difficulty of communicating requirements • Impedance mismatch among stakeholders • Unrestrained external complexity • Software drag

22. The limits of software

23. Difficulty of managing the development process • Software as a team sport • Presence of multiple languages, platforms, processes, architectures, and tools • Issues of scalability • Technology churn

24. Scalability • Size up • Increasing database size by a factor of x increases query response time by at most a factor of x. • Speed up • Increasing the capacity of your hardware configuration by a factor of x decreases your query response time by no less than a factor of x. • Scale up • Increasing the workload on your system by a factor of x while maintaining response time and/or throughput requires increasing your capacity by a factor of no more than x. • Scale out • Increasing workers by a factor of x requires replicating your capacity by a factor of at most x. http://www.intelligententerprise.com/db_area/archives/1999/991602/scalable.jhtml

25. Fluidity of software • Software springs from pure thought and is intrinsically malleable, yet it can be made manifest in our hardware systems, limited only by our vision (and certain immutable laws of physics and software)

26. Fundamental challenges of discrete systems • Non-continuous behavior of discrete systems • Combinatorial explosion of states • Corruption from unexpected external events • Lack of mathematical tools and intellectual capacity to model the behavior of large discrete systems

27. Essential complexity • “Einstein argued that there must be simplified explanations of nature, because God is not capricious or arbitrary. No such faith comforts the software engineer. Much of the complexity that he must master is arbitrary complexity.” [Brooks] • We may master essential complexity, but we can never make it go away.

28. Measuring complexity of biological systems (syntactic) • Kolmogorov • Entropy • Mean component size • Number of behaviors exhibited • Species richness, relative to tolerance to risk • Species guilds • Energy flow • Grammatical complexity • Number of feedback loops • Cyclomatic measures (arcs, vertices, and components) • Graph complexity • Hamming distance http://www.carleton.ca/~hmasum/complex.html

29. Measuring complexity of biological systems (semantic) • Wordcount of description • Minimal description length (Rissanen) • Measure of environmental knowledge • Evolvability • Eigenbasis/measure of survivable environmental states • Program complexity http://www.carleton.ca/~hmasum/complex.html

30. Measuring complexity of software-intensive systems • Kolmogorov • Relative size of a program capable of generating a given string • Entropy • Enumeration of states and transitions http://cscs.umich.edu/~crshalizi/notebooks/complexity-measures.html

31. Measuring simplicity • If we don’t know how to measure complexity, it is reasonable to suggest that we don’t know how to measure simplicity • “I can’t define it, but I know it when I see it.” [Supreme Court Justice Brennan]

32. Beauty • Elegance is not an approach to finding a solution to a problem, it is the label we stick on the optimum solution • Elegance is doing the most with the least • Elegance means simplicity and less new code. • An elegant solution solves the whole problem." [Fisher, p. 37] Fisher & Gipson, “In Search of Elegance”

33. Triggers of complexity • Significant interactions • High number of parts and degrees of freedom • Nonlinearity • Broken symmetry • Nonholonomic constraints • Localized transient anarchy Flood, et al, Dealing with Complexity

34. Attributes of a complex system • “Frequently, complexity takes the form of a hierarchy, whereby a complex system is composed of interrelated subsystems that have in turn their own subsystems, and so on, until some lowest level of elementary components is reached.” [Courtois] • Hierarchic systems are decomposable if they can be divided into identifiable parts; they are nearly decomposable if their parts are not completely independent. [Simon]

35. Attributes of a complex system • The choice of what components in a system are primitive is relative arbitrary and is largely up to the discretion of the observer of the system. • As systems evolve, objects that we once considered complex become the primitive objects upon which more complex systems are built.

36. Attributes of a complex system • Intracomponent linkages are generally stronger than intercomponent linkages. This fact has the effect of separating the high-frequency dynamics of the components – involving the internal structure of the components – from the low-frequency dynamics - involving interaction among components. [Simon]

37. Attributes of a complex system • Hierarchic systems are usually composed of only a few different kinds of subsystems in various combinations and arrangements. [Simon]

38. Decomposible and nearly-decomposible systems • Vertically, the components of a complex system tend to be organized in increasing layers of abstraction • Horizontally, the components of a complex system tend to be clustered according to frequency • Both vertically and horizontally, the most resilient systems tend to exhibit loose coupling and tight cohesion among components Simon, The Organization of Complex Systems

39. Components • Loosely-coupled components adapt more easily to change • Loosely-coupled components permit time- and space-separation of processing • Overall flexibility can be enhanced by limiting the number of different kinds of components in the system (the system’s alphabet) • Alphabets are necessary but insufficient • Complex systems also require common languages, defining semantics of structural organization of alphabetic elements and interactional behavior among structures Simon, The Organization of Complex Systems

40. Languages • Must have sufficient variety in its primitive processes so that no meaning is absolutely excluded from expression • Must have sufficient flexibility in its rules of combination so that any nuance can be expressed by building up composite structures Simon, The Organization of Complex Systems

41. Attributes of complex systems • Complex systems will evolve from simple systems much more rapidly if there are stable intermediate forms than if there are not. [Simon] • A complex system that works is invariable found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system. [Gall]

42. Complex adaptive systems • Emergent behavior • Attributes • Classification of components • Identity of objects • Non-linearity of behavior • Flow of objects • Diversity • Use of internal models • Clustering Holland, Hidden Order

43. Characteristics of self-organizing systems • Dynamic, requiring continual interactions among lower-level components to produce and maintain structure • Exhibit bifurcation leading to multistable systems • Strange attractors Sante Fe Institute

44. Self-organization in biological systems • Pattern formation in slime molds • Feeding aggregation of bark beetles • Synchronized flashing among fireflies • Fish schooling • Nectar source selection by honey bees • Trail formation in ants • Swarm raids of army ants • Colony thermoregulation in honey bees • Comb patterns in honey bee colonies • Wall building by ants • Termite mount building • Construction Aagorithms by wasps • Dominance hierarchies in paper wasps Camazine, et al, Self-Organization in Biological Systems

45. Creating order in biological systems • Self-organization • Emergence of patterns at the global level solely from numerous interactions among lower-level components of the system, the rules for which are executed using only local information • Imposed organization • Direction from a supervisory leader • Organic blueprints or recipes • Patterns in the environment Camazine, et al, Self-Organization in Biological Systems

46. Complexity, Functionality, and Understandability Understandability Functionality Complexity

47. Fundamentals never go out of style

48. Space plan Services Stuff Structure Skin Site Shearing layers of change Brand, How Buildings Learn

49. Fundamentals of well-engineered software-intensive systems • Crisp abstractions • Clear separation of concerns • Balanced distribution of responsibilities • Simplicity via common abstractions and mechanisms

50. Abstraction • All abstractions are context-dependent • All non-trivial abstractions are, to some degree, leaky (and leaky abstractions are problematic). [Joel on Software] • There is no such thing as a perfect abstraction • Perfect is the enemy of good enough