Managing complexity in a Software plus Services World

2. Session Code: ARC206 Managing complexity in a Software plus Services World Amanda Jackson Fronde Systems Group Ltd

3. Presentation Overview Introduction Redefining Enterprise Architecture Understanding complexity Partitioning Mathematics of complexity SIP Summary

4. Amanda Jackson Consultant/Developer with Fronde Systems Ltd Started NZ Girl Geek Dinners in 2007 In the past has covered most roles in IT including: QA, Network Admin, Source Control Manager, Build Management, Games Development, Change Management Working towards EA & SA certification

5. What is this talk about? Simplifying architecture through: Modeling complexity Partitioning complex systems By simplifying architecture we can more easily: Add business value Benefit from trends like Software plus Services

6. Software plus Services • Combines Internet services with client and server software • Increasingly complex, globally distributed systems • More choices in the ways to access and manage software • A new way of thinking is required at the EA level!

7. Before we can simplify and redefine we need to understand: What Enterprise Architecture is Why we need Enterprise Architecture What can go wrong How things fail Simplify & Redefine

8. What is Enterprise Architecture? Documentation or processes that describe: • Structure • Organization • Behaviours • Planning Changes

9. Enterprise Architecture “An enterprise architecture is a description of the goals of an organization, how these goals are realized by business processes, and how these business processes can be better served through technology.” Roger Sessions - Simple Architectures for Complex Organizations

10. Benefits of a good Architecture Adds value to an organization Is concerned with all aspects of an organization Ensures a good ROI Helps ensure successful projects

11. Why we need EA Based on a number of studies done in the last 10 years (such as The KPMG Canada Study, The Chaos Report and others), it has been established that more than 50% of IT projects fail outright! Andy Blumenthal – Director of Enterprise Architecture & IT Governance for the US Coast Guard

12. What can go wrong? Failed projects Loss of income

13. Why EA fails Excessive complexity Processes are: Expensive Time consuming Poorly defined Results are: Not testable Not reproducible Not directly usable Value is not delivered

14. EA Failures Four years ago the government announced to a grateful NHS a national IT program to become the world’s largest civil computer scheme. But after a breathless start, delivery dates for key software were missed, the full costs of implementation have always been unclear, and experts are divided over whether the scheme is too ambitious to ever work as originally planned. Questions that need to be answered on NHS IT plan by Tony Collins, April 2006 in ComputerWeekly,com

15. In October 2007, Gartner predicted that 40% of all existing Enterprise Architecture programs will be shut down by 2010. Fewer than 5% of firms use Enterprise Architecture effectively. Ross, Weill & Robertson – Enterprise Architecture as a Strategy. What does this mean for EA?

16. What is the problem?

17. The new EA Focus on complexity at the enterprise level Understand the mathematics of complexity Create a model for complexity Define a process for controlling complexity Test solutions against that model

18. What is Complexity? A function of the number of states that system can find itself in!

19. Software System Complexity values (heads, tails) penny; penny = read (penny_sensor); if (penny == heads) message (“Penny is heads”); if (penny == tails) message (“Penny is tails”); end; Complexity = States Per VariableVariables

20. Business System Complexity Prepare Penny/Heads Memo Penny = Heads Memo RE: Penny Outcome: Heads Read Penny Prepare Penny/Tails Memo Penny = Tails Memo RE: Penny Outcome: Tails Complexity = Branches per Decision Point Decision Point

21. Software & Business Comparison When considering complexity we can see that: Variables are like decision points States per variable are like branches per point

22. Mathematical Foundations

23. Partitioning The most important complexity control strategy Divide and conquer to reduce complexity Reduced complexity = improved efficiency

24. The Power of Partitioning Number of autonomous systems: 1 Number of states per variable: 6 Number of variables: 12 Number of states: 2,176,782,336 Number of autonomous systems: 2 Number of states per variable: 6 Number of variables: 6 Number of states: 93,312 Reduction in complexity: 99.57%

25. Looking at it another way… 1 Bucket / 2 Dice a.k.a. 1 autonomous system 2 Buckets / 1 Die Each a.k.a. 2 autonomous systems 1 Die = 6 states 1 Die = 6 states 1 Die = 6 states 2 Die = 6 6 states Total states this bucket: 36 Total states for 2 buckets: 12

26. And now with 12 variables… 1 Bucket / 12 Dice 2 Buckets / 6 Die Each 3 Buckets / 4 Die Each 1 Die = 6 states 12 Die = 612 Total states: 2,176,782,336 66 = 46,656 states per bucket 64 = 1,296 states per bucket Total states: 93,312 Total states: 3,888

27. What is a partition? • A set of subsets that divide a larger set • All elements of the larger set live in one, and only one, of the subsets

28. Five Laws of Partitions Must be true partitions Definitions must be appropriate to the problem at hand The number of subsets must be appropriate The size of the subsets must be roughly equal The interactions between subsets must be minimal and well defined

29. Which are partitions? Set 1 Set 2 Set 3 Set 4 Set 5 Unpartitioned

30. What do we partition? Consider same-category-as

31. Same-category-as Food Stationery Reading Kitchenware

32. Our new partitions Food cereal soft drink flour lollies Stationery pen pencil notebook Reading newspaper Kitchenware knife cup

33. SIP (very brief) Overview SIP describes the main approaches used for: Complexity control Partitioning Simplification Iterative Delivery Overall goal is to align the IT and Business processes so they work together effectively

34. Partition Identification • Start at the highest possible view of the enterprise then attempt to partition • If partition is successful – repeat • Keep repeating until you cannot partition any more • At each stage of partitioning, assign types and deployment information

35. Start at the highest possible view Retail Organization Operations Retail Sales Catalogue Sales Internet Sales Brand Awareness Product Awareness Planning

36. Identify Synergies Retail Organization Operations Retail Sales Catalogue Sales Internet Sales Brand Awareness Product Awareness Planning

37. Create simpler partitions Retail Organization Operations Sales Marketing Planning

38. Keep following your path Retail Organization Operations Sales Marketing Planning Sales Retail Distribution Retail Retail Sales Inventory Reporting

39. Partition Simplification Construct sub-classes of a partition. Any functionality that can be removed from an element should be removed

40. Software plus Services Hugely flexible Integrate traditional software with distributed services Put services in a cloud for direct user access A dizzying, and very impressive array of S+S services The ability for the architecture to get hugely bloated and complex if not planned well – a.k.a. Bloatware!

41. How do we partition S+S? Keep It Simple Create an overall high level architecture umbrella – start at the highest possible view Architect and partition each solution that is relevant to your current objective Don’t repeat elements in multiple subsets

42. Case Study in Complexity • Complexity can creep into even the most extensively planned project • Unchecked complexity leads to project failure • Using Simple Iterative Partitions might have saved this project, even when it was well into failure mode

43. NPfIT Launched in June 2002 Automate and centralize NHS record keeping Automation of all patient care information Access to any patient record by any authorized health care professional in the UK Ability to book appointments with any health care facility in the UK Automation of prescription services

44. NPfIT Functionality Regional Clinical Information Systems Initial cost - approximately NZ$14 billion Infrastructure Systems Initial cost – approximately NZ$3.2 billion Shared Applications Initial cost – approximately NZ$425 million

45. NPfIT Overview Multi-billion dollar project Contracts split between at least a dozen vendors Covers a geographic area of close to 100,000 square miles Offers services to 60 million people Expected to process 300 transactions per second

46. Current status of NPfIT In crisis almost from the first day One year into the project most vendors were having problems relating to each other The worst off, the Accenture/iSoft partnership.

47. Accenture/iSoft By Sept 2006 Accenture had enough and abandoned the project They walked away from over $5 billion in revenues They wrote off in excess of $700 million they had already spent They agreed to pay over $100 million to settle legal obligations.

48. Overall Cost of NPfIT? Estimated cost range is $68 to $142 billion

49. Simple Iterative Partitions Phase 1 – audit of organization readiness Phase 2 – working on the partitioning Phase 3 – simplify the partitions Phase 4 – prioritize subsets in the partitions Phase 5 - iteration

50. Phase 1 Highlights deep distrust between NHS factions Delivers extensive training in the nature of complexity Focuses on managing complexity as the absolute highest priority