Being agile in the real world Experiences from an IBM’er

1. Being agile in the real worldExperiences from an IBM’er Ole Rasmussen Senior IT Architect Business Consulting Services ora@dk.ibm.com

2. A few words on me and the organisation I’m working in • Ms. Sc. in Computer Science at Aalborg University (Center), 1993 • Bs. Sc. in Mathematics at Aalborg University (Center), 1990 • Working for IBM since 1997 • Business Consulting Services • Senior IT Architect • Teaching at Niels Brock since 1996 • My context is “one of a kind” and “custom application” (partly) and for given customers • Roles • Lead Architect, Solution architect, Reviewer, Designer, Method consultant, Teacher and mentor • Areas • Service oriented architecture, e-Business, Functional architecture, Object-oriented analysis and design, Methodologies, Java, J2EE • Industries • Public, Insurance, Distribution, Travel & Transportation, Communication

3. IBM’s is indeed based on matrixes

4. $96.3 $1.2 $2.6 $85.1 $81.2 $77.5 $15.1 $1.4 $3.5 $1.1 $3.2 $1.9 $2.9 $12.6 $13.1 $11.9 $31.2 $27.5 $34.5 $32.0 $46.2 $36.4 $28.9 $33.2 1998 2000 2002 2004 • Services become increasingly important • In $ billions (as reported) • Enterprise Investments/Other • Global Financing • Software • Hardware • Services

5. IBM Global Services Client IBM Sales & Distribution IBM Products & Technology e-bHS BCS IBM Research US EMEA AP SO BCS IBM Global Finansing AMS ITS Channel Partners Business Partners IBM’s envision a lot of the business is driven by a BCS – the consulting business line within IGS

6. Agenda Lightweight versus heavyweight methodologies Experiences with customers on agile engagements My world of methodologies Businesses are requiring agility – within the business Patterns improve my “agility”

7. Agenda Lightweight versus heavyweight methodologies Experiences with customers on agile engagements My world of methodologies Businesses are requiring agility – within the business Patterns improve my “agility”

8. Principles Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan Characteristics Adaptive rather than predictive lightweight not heavyweight can deal with requirements drift People- rather than process-oriented less document-oriented Iterative development Development team empowerment of development team multi-skilled members in small teams (<= 6) Customer Relations contract price difficult to negotiate close business partnership & user contact required Process Adaptation product & process needs to be reviewed after each iteration (and further adaptations made if needed) what went well? what can be improved? what have we learned? what still puzzles us? Agile Methods: Principles and Characteristics

9. Agile Methods: Process Adaptation process adapt apply development customer team project project iteration review

10. Applicability Project environment Unstable requirements able to be delivered incrementally Suggests business systems rather than e.g. RT Customer profile Willing to engage in the development process Different relationship with developers Development teams Smaller, motivated, highly skilled & empowered to make technical decisions Limitations Distributed development environments Subcontracting Building reusable artefacts Large team development Building safety-critical software Developing large, complex software (Turk, D., France, R, & Rumpe, B., 2002) Agile Methods: Applicability and Limitations

11. Example Heavyweight methodology - Unified Process Overview • Unified Process is component based • system is built using software components interconnected using well defined interfaces • Unified Process uses Unified Modelling Language (UML) • Unified Process is distinguished by being • use-case driven • architecture-centric • iterative and incremental • Based around the 4Ps - People, Project, Product, Process • Provides disciplined approach to assigning tasks and responsibilities • Guide for how to use Unified Modelling Language (UML) effectively • Activities create and maintain (UML) models • Is a configurable process

12. Use Case Driven • “A description of a set of sequence of actions, including variants, that a system performs that yields an observable result of value to a particular actor.” • (Jacobson et.al. 1999) • i.e. a piece of functionality that gives a user a result of value • Development process follows a flow • proceeds through a series of workflows derived from the use cases • use cases are specified, designed and are the source for test cases • they drive system architecture which in turn influences use case selection • both mature as the development lifecycle continues • Architecture Centric • Software architecture shows different views of the system being built and embodies the static & dynamic aspects of the system (design framework) • Also influenced by the computer architecture, operating system, DBMS, network protocols etc. • Related as function (use case) and form (architecture) • The form must allow the system to evolve from initial development through future requirements (i.e. the design needs to be flexible) • Key use cases influence the design of the architecture which may in turn influence development of other use cases • Iterative and Incremental • Systems development is frequently a large undertaking - may be divided into several “mini-projects” each of which is an iteration resulting in incremental development of the system • Iterations must be selected & developed in a planned way i.e. in a logical order - early iterations must offer utility to the users • iteration based on a group of use cases extending the usability of the system developed so far • iterations deal with the most important risks first • not all iterations are additive - some replace earlier “superficial” developments with a more sophisticated and detailed one. Heavyweight methodologies - main distinguishing characteristics Concepts of use case driven, architecture centric and iterative & incremental are of equal importance

13. Agenda Lightweight versus heavyweight methodologies Experiences with customers on agile engagements My world of methodologies Businesses are requiring agility – within the business Patterns improve my “agility”

14. The typical customer expects a contract with … • Huge scope • Fixed deliverables • Fixed price Does this align with agile approaches? Applicability • Project environment • Unstable requirements able to be delivered incrementally • Suggests business systems rather than e.g. RT • Customer profile • Willing to engage in the development process • Different relationship with developers • Development teams • Smaller, motivated, highly skilled & empowered to make technical decisions Limitations • Distributed development environments • Subcontracting • Building reusable artefacts • Large team development • Building safety-critical software • Developing large, complex software (Turk, D., France, R, & Rumpe, B., 2002)

15. Closing the “gaps” during the project Design your project to handle the nature of and uncertainties within the requirements • Requirements appearance • Part of tender => Not an agile approach • Pre-analysis => mainly “paper work”; may include a proof-of-concept • Elaborated during project => may become an agile approach – depends on customer willingness • Is your customer able to sign-off (an analysed version of) requirements? • The customer is uncertain about the correctness and completeness • The customer can not envision the solution • The customer do not want to be hindered in changing mind

16. XXXWorld Central MCO Local (Country) SP Non-Core Presentation layer Service layer Service layer Service layer Service layer SP Core MCO Presentation PIM (calendar, Addresses) Chat Instant messaging Unified messaging Presentation layer Music Currency conversion Content filtering Multi-access functionality Syncron- isation Content filtering Play Arcade Caching Caching Self-service Language E-mail Application layer SMS/EMS MMS MCO Services Personal- isation Content management Commerce Security OIF adaptations OIF adaptations 3rd party Specific service 3rd party specific service 3rd party specific service 3rd party Specific service 3rd party Specific service 3rd party Specific service MCO Specific service MCO Specific service MCO Specific service MCO Specific service Access right management Push notifications CRM Location XXX Integration Framework Streaming ASP 3rd Parties 3rd Parties Back-end PLMN CC&B Presentation layer Application layer Application layer Network Element(s) Data Warehouse Charging Settlement Reporting Service layer My only project using an agile approach XXXWorld Service Platform – a logical overview

17. The development approach was XP inspired We faced two major problems • The project was very large • The customers customer (the countries) was not represented in the development project

18. In most projects some agile “best practices” are applied • Proof-of-concepts (properly not originally agile;-) • Iterative development (not originally agile;-) • Automated unit testing • (Bi-) Nightly build, deployment and unit test

19. Agenda Lightweight versus heavyweight methodologies Experiences with customers on agile engagements My world of methodologies Businesses are requiring agility – within the business Patterns improve my “agility”

20. Worldwide Project Management Method I employ multiple methods to perform my work IGSM defines how to design and deliver the solution SSM defines how we sell products and services WWQA/MD defines IBM business controls during the proposal and project WWPMM defines how we manage a project BCS Risk Management

21. The methodologies are integration throughout the sub-processes of the overall CRM process

22. IBM has a very comprehensive methodology for projects 1 1.1 1.1.1 1.1.2 1.1.3 1.2. 1.2.1 WP WP WP WP WP WP WP WP 2 5 5 1 2 2 15 3 5 1 1 1 Engagement Model Process Guidance Engagement Family Capability Pattern Solution Startup Release Solution Outline Micro Design Build Cycle Deployment Macro Design Reference Architecture Fit/Gap Analysis Architecture Overview Diagram Non-functional Requirements Solution Close Architectural Template Phase Current I/T Infrastructure Current I/T Standards Component Model SLC Analysis Use Case Model Deployment Units Operational Model Technical Prototype Class Diagram Software Distribution Plan Deployment Unit Matrices UI Conceptual Model UI Design Guidelines Viability Assessment System Management Plan dependencies to and from most other WPs Architectural Decisions Dependency Diagram Initiate Build Cycle Implement User Experience Perform Programming Cycle Develop Support Materials Perform System Testing Work Product Descriptions Plan Deployment Prepare for Testing Perform Development Testing Engagement Confirm Build Cycle Build Event Driven Business Activity Organization Application Architecture Operations Domain Outline Architecture Model Outline Architecture Model • Architecture Overview Diagram [architecture] • Architectural Decisions [architecture] • Architectural Templates [architecture] • Operational Model [architecture] • Component Model [architecture] • Reference Architecture Fit Gap Analysis [architecture] • Candidate Asset List[project management] • Viability Assessment [architecture] • Develop Architecture Overview • Survey Available Assets • Identify Sterotypical Interactions • Develop High Level Component Model • Develop High Level Operations Model • Refine Viability Assessment Technique Papers Task Roles

23. Work products define the pieces of work to be done in a project. • Tangible artifacts produced during the project • Models, reports, diagrams, plans, code and other documents which are direct "stepping stones" to the final deliverable. • Have a specific purpose in the engagement and describes specific content using a predefined semantics and syntax. • Produced as a result of performing one or more tasks. • Some tasks produce less tangible outputs (pass/fail, trained students) and are called "outcomes". • Not necessarily the same as a deliverable, it may be an intermediate product and not delivered to the customer. • All deliverables consist of work products. • Not all work products are deliverables. • Are the basis of intellectual capital reuse on engagements. • Examples of WPs from other engagements are an important form of IC • Can structure some IC around WPDs

24. Work product types are the common building blocks of engagements • Work products form the basis of • Work plan development (project tailoring) • Deliverables management • Intellectual capital reuse • Quality management • WP's enable resource sharing • People from different segments can work together effectively on the same engagement. • Resources can move to new segments without significant retraining.

25. Work product types have a 3-25+ page Work Product Description • A Work Product Description describes a type of work product. • Some WPDs have one instance per project e.g. Release Plan. • Some have many instances per project e.g. Design Interaction Diagram • Work product description elements • Description • Purpose • Impact of NOT having the work product • Reason for not needing the work product • Notation • Example • Development approach • Validation and verification • Advice and guidance • References • Estimation considerations

26. Within my domain (IT architecture) an description standard forms the basis for work products

27. Common work product descriptions provided by the methodology forms an excellent communication vehicle - UML eases communication Reference Architecture Fit/Gap Analysis Non-functional Requirements Architecture Overview Diagram Architectural Template Standards Current IT Environment Service Level Char. Analysis Component Model Use Case Model Deployment Units Operational Model Technical Prototype Class Diagram Software Distribution Plan UI Design Guidelines UI Conceptual Model Viability Assessment System Context IT Services Strategy Performance Model Parametric Costs Architectural Decisions Change Cases dependencies to and from most other WPs Technical Transaction Map

28. Phases, activities and tasks structure the project – the engagement models provide “common” structures

29. The large amount of engagement families and models reflects the broad spectrum of project that IBM engage

30. A methodology always has to be designed for the specific situation. A Method Adoption Workshop (MAW) is used to bring the project and the method together • The Method • WW SDD Procedures • Tooling Expected MAW results • Trained participants • List of work products to be produced • Idea of project's process • Initial project plan • Project estimates • Risk Assessment • Resource Plan for the Team Engagement participants • Project manager • Architect • Designer • Other selected participants MAW Method Exponent(s)

31. Agenda Lightweight versus heavyweight methodologies Experiences with customers on agile engagements My world of methodologies Businesses are requiring agility – within the business Patterns improve my “agility”

32. A Time of Change ‘On Demand Business is our way of describing a fundamental shift in computing architecture and how it is applied to business — a shift toward integrated solutions and quantifiable business value, not just technology features and functions.’

33. The New Agenda An On Demand Business is an enterprise whose business processes — integrated end-to-end across the company and with key partners, suppliers and customers — can respond with flexibility and speed to any customer demand, market opportunity or external threat.

34. Clients’ Needs and Expectations Are Changing Continuous Change Impact on Services Industry To Client Pain Points From Rigorous Competition Fusion of Business and IT Responsive Predictive Unrelenting Financial Pressures Focused Selective sourcing Diffuse Unpredictable Threats Standardization, commoditization Variable Fixed Demand for best-in-class Resilient Vulnerable

35. Enterprise Architecture “the city plan” • System Architecture • functional aspects • operational aspects • “the infrastructure and single • building design” ”a shift toward integrated solutions and quantifiable business value” mandates for business and IT alignment – Enterprise Architecture is key Strategy Business Opportunity Technology Availability Business Strategy Information Technology Strategy Strategy and vision “Which city do we want to build?” TheGap Enterprise Architecture Enterprise wide focus Business Architecture IT Architecture Planning • Processes • Information • People • Locations • Applications • Data • Technology The Great Divide Transition Plan Business Operating Environment and IT Infrastructure Design and Delivery Project focus IT Solutions

36. The pain points implies business challenges facing enterprises today – these challenges demand the fusion of business and IT Increase revenue create new routes to market, create new value from existing systems Provide a flexible business model react to market changes more quickly Each represents a SOA value proposition Integrate across the enterprise integrate historically separate systems, facilitate mergers and acquisitions of enterprises Reduce cycle times and cost for external business partners move from manual to automated transactions, facilitate flexible dealings with business partners Drive down cost eliminate duplicate systems, build once and leverage, improve time to market Reduce risk and exposure improve visibility into business operations

37. What is Service-Oriented Architecture ? “SOA in context …” • a set of services that a business wants to expose to their customers and partners, or other portions of the organization • an architectural style which requires a service provider, requestor and a service description • a set of architectural principles, patterns and criteria which address characteristics such as modularity, encapsulation, loose coupling, separation of concerns, reuse, composability and single implementation • a programming model complete with standards, tools and technologies such as Web Services Business Architecture Implementation

38. SOA and Services Defined A Service-Oriented Architecture is an enterprise-scale ITarchitecture for linking resources on demand. These resources are represented as business-aligned services which can participate and be composed in a value-net, enterprise, or line of business to fulfill business needs. The primary structuring element for SOA applications is a service as opposed to subsystems, systems, or components. A Service is a discoverable software resource which has a service description. The service description is available for searching, binding and invocation by a service consumer. The service description implementation is realized through a service provider who delivers quality of service requirements for the service consumer.

39. Is Web services part of the answer? • Web Services can be a part of the answer ... but mostly we'll get to that later • Service Oriented Architecture (SOA) is another part • The two are not the same thing: • Most of today's production Web Services systems aren't service oriented architectures - they're simple remote procedure calls or point-to-point messaging via SOAP or well structured integration architectures • Most of today's production service oriented architectures don't primarily use Web Services - they use ftp, batch files, asynchronous messaging etc. - mature technologies • Achieving the promoted benefits of SOA and Web Services, requires both SOA and Web services

40. Service Consumer 5 6 7 8 JService Portlet WSRP B2B Other consumers 4 business processes process choreography service modeling 3 services atomic and composite Integration (Enterprise Service Bus approach) Data Architecture & Business Intelligence QoS, Security, Management & Monitoring Infrastructure Service Service Provider 2 service components 1 Custom Application OO Application Packaged Application operational systems Packaged Application Custom Application Composite service Atomic service Registry An SOA is composed of multiple layers that decouple the provider and consumer views

41. Service Consumer 5 6 7 8 JService Portlet WSRP B2B Other consumers 4 business processes process choreography 3 Integration (Enterprise Service Bus approach) Data Architecture & Business Intelligence QoS, Security, Management & Monitoring Infrastructure Service services atomic and composite Service Provider 2 enterprise components 1 Custom Application OO Application Packaged Application Packaged Application Custom Application operational systems Composite service Atomic service Registry The role of Service Oriented Modeling and Architecture (SOMA) in SOA development is to provide a prescriptive technique for modeling (analysis and design) necessary to create a Service-Oriented Architecture with composable services At the heart of SOMA is the identification and specification of services, components and flows << Input from: Business Componentization/Analysis >> Identification of candidate Services, Components, and Flows Specification of Services, Components, Flows Realization Decisions << Output to: SOA Implementation >>

42. Domain Decomposition Goal-Service Modeling Existing Asset Analysis component flow specification service flow specification Subsystem Analysis Service Specification Component Specification information specification message & event specification Realization Decisions service allocation to components technical feasibility exploration component layering SOMA activities are grouped into three major steps: Identification, Specification and Realization What we do Identification of candidate Services, Components, and Flows Identification How we do it Specification of Services, Components, Flows Specification Realization Realization Decisions • The first major step in the SOMA process identify candidate services and enterprise components. • The second major step selects and specifies the services and enterprise components that will be exposed • The third major step captures realization decisions (concurrently with steps one and two)

43. The Service Model captures information related to services and is built incrementally as SOMA activities are iteratively carried out Service Model Service Portfolio Service Hierarchy Service Exposure Service Dependencies Service Composition Service NFR Service Messages Realization Decisions State Management

44. “Effective IT Governance is the single most important predictor of value an organization generates from IT.” MIT Sloan School of Mgmt. Consistent Service Model Strategy Planning Business Opportunity Business Strategy Information Technology Strategy Technology Availability Reconcile Multiple Viewpoints & Interests Enterprise-wide focus Business Architecture IT Architecture Model & Assemble Business Operating Environment and IT Infrastructure Deploy & Manage The governance model defines: • What has to be done? • How is it done? • Who has the authority to do it? • How is it measured? IT Solutions Governance is yet another key…IT and Operations align with Business

45. Facilitates communication between services ESB Apps & Info Assets SOA Reference ArchitectureSupporting your SOA Lifecycle Business Innovation & Optimization Services Facilitates better decision-making with real-time business information Interaction Services Process Services Information Services IT ServiceManagement DevelopmentServices Enables collaboration between people, processes & information Orchestrate and automate business processes Manages diverse data and content in a unified manner Integrated environment for design and creation of solution assets Manage and secure services, applications & resources Partner Services Business App Services Access Services Connect with trading partners Build on a robust, scaleable, and secure services environment Facilitates interactions with existing information and application assets Infrastructure Services Optimizes throughput, availability and performance

46. The developer roles are changing in the SOA era

47. Agenda Lightweight versus heavyweight methodologies Experiences with customers on agile engagements My world of methodologies Businesses are requiring agility – within the business Patterns improve my “agility”

48. Patterns prevail most of my efforts • Coding and design (the few times I get into these activities) • Analysis – industry analysis models • Business models • Architectural patterns • Reference architectures

49. The Service Component Compound Pattern – A Recommended Design Façade Mediator Rule Object Composite [Factory] [Adaptors] Source: Ali Arsanjani, “Enterprise Component Pattern”, Pattern Languages of Programming 2001

50. Business Administration New Business Development Relationship Management Servicing & Sales Product Fulfilment Financial Control and Accounting direct control Portfolio Planning Business Planning Sector Planning Account Planning Sales Planning Fulfilment Planning Business Unit Tracking Sector Management Relationship Management Sales Management Fulfilment Planning Compliance Reconciliation Product Management Credit Assessment Staff Appraisals execute Staff Administration Product Directory Credit Administration Sales Product Fulfillment Customer Accounts Marketing Campaigns Customer Dialogue Document Management General Ledger Production Administration Contact Routing Component Business Modelling is technique used to show the entire enterprise – and make analysis and decisions on the business Columns are Business Competencies, defined as large business areas with characteristic skills and capabilities, for example, product development or supply chain. A Business Component is a part of an enterprise that has the potential to operate independently, in the extreme as a separate company, or as part of another company. An Accountability Level characterises the scope and intent of activity and decision-making. The three levels used in CBM are Directing, Controlling and Executing. • Directing is about strategy, overall direction and policy. • Controlling is about monitoring, managing exceptions and tactical decision making • Executing is about doing the work