Systems Engineering for the Internet and the Web

1. Systems Engineering for the Internet and the Web Rob Oshana oshana@airmail.net 214-415-9690

2. My Background • Defense business experience • Internet/web experience • Commercial “shrink wrap” experience • SMU adjunct (CSE, EETS) • Consulting with telecom • E-Commerce certificate

3. So Why am I Talking About This ? • I learned a lot about system engineering in DoD environment • I saw a need for sound system engineering principles in the internet space (currently chaotic) • I believe there is an opportunity in the educational space

4. Introduction • The Internet has increased the scope and complexity of information technology systems, placing even greater importance on system planning and design • System engineering for rapid, iterative methodologies of the Internet world

5. System • A system can be defined as • an integrated composite of people, products, and processes that provide a capability to satisfy a need or objective [MIL-STD-499B] • a collection of components organized to accomplish a specific function or set of functions • an interacting combination of elements, viewed in relation to function [INCOSE 95]

6. System • A system may be a product that is hardware only, hardware/software, software only, or a service • the sum of the products being delivered to the customer(s) or user(s) of the products • achieve the overall cost, schedule, and performance objectives of the business entity developing the product

7. Systems engineering process • Systems engineering process is a comprehensive problem-solving process used to • transform customer needs and requirements into a life-cycle balanced solution set of system product and process designs • generate information for decision makers • provide information for the next product development or acquisition phase

8. SE-CMM Process Areas

9. Applicable Process Areas • Analyze Candidate Solutions • Derive and Allocate Requirements • Evolve System Architecture • Integrate Disciplines • Integrate System • Understand customer needs • Coordinate with suppliers, etc

10. Analyze Candidate Solutions • Identifies the characteristics of a process for choosing a solution from several alternatives • design decision • production decisions • life-cycle cost decisions • human factors decisions • risk reduction decisions

11. Derive and Allocate Requirements • Typical Work Products • operational concept • user interaction sequences • maintenance operational sequences • timelines • simulations • usability analysis

12. Understand Customer Needs and Expectations • Interface control working groups • Questionnaires, interviews, operational scenarios obtained from users • Prototypes and models • Brainstorming • Market surveys • Observation of existing systems, environments, and workflow patterns

13. Coordinate with Suppliers • Typical Work Products • make-vs.-buy trade study • list of system components • sub set of system components for outside organizations to address • list of potential suppliers • beginnings of criteria for completion of needed work

14. System Engineering Applied to Internet Infrastructure

15. Example - Campus Network http://www.cisco.com/cpress/cc/td/cpress/ccie/ndcs/ 01ccie.htm#35145

16. Determining Requirements • Understand requirements • Selecting capability and reliability options that meet these requirements • Solution must reflect the goals, characteristics, and policies of the organizations in which they operate

17. Determining Requirements • Two primary goals drive design and implementation • Application availability • Cost of ownership • IS budgets today often run in the millions of dollars as large organizations increasingly rely on electronic data for managing business activities • A well-designed solution can help to balance these objectives!!

18. The Design Problem: Optimizing Availability and Cost • Design problem consists of the following general elements • Environmental givens • location of hosts, servers, terminals, and other end nodes • the projected traffic for the environment • projected costs for delivering different service levels

19. Optimizing Availability and Cost • Performance constraints • network reliability • traffic throughput • host/client computer speeds (for example, network interface cards and hard drive access speeds). • Internetworking variables • network topology • line capacities • packet flow assignments

20. Optimizing Availability and Cost • Goal is to minimize cost based on these elements while delivering service that does not compromise established availability requirements • Primary concerns are availability and cost • essentially at odds • increase in availability must generally be reflected as an increase in cost

21. Assess needs and cost Select topologies and technologies to satisfy needs Model Network Workload Simulate behavior under expected load Perform sensitivity tests Rework design as needed General Network Design Process

22. Assessing User Requirements • Users primarily want application availability in their networks; • response time • interactive online services, such as automated tellers and point-of-sale machines • throughput • file- transfer activities (low response-time requirements) • always a tradeoff; think Size/Weight/Power!

23. Assessing User Requirements • reliability • Financial services, securities exchanges, and emergency/police/military operations • high level of hardware and topological redundancy • Determining cost of any downtime is essential in determining the relative importance of reliability

24. Assessing User Requirements • User community profiles • Interviews, focus groups, and surveys • Interviews with key user groups • Focus groups • Formal surveys can be used to get a statistically valid reading of user sentiment • Human factors tests

25. Assessing Proprietary and Nonproprietary Solutions • Compatibility, conformance, and interoperability are related to the problem of balancing proprietary functionality and open internetworking flexibility • Multivendor environment or specific, proprietary capability • Open routing protocol can potentially result in greater multiple-vendor configuration complexity

26. Assessing Proprietary and Nonproprietary Solutions • Gaining a measure of interoperability versus losing functionality • Previous internetworking (and networking) investments and expectations for future requirements have considerable influence over choice of implementations

27. Assessing Proprietary and Nonproprietary Solutions • Must consider • installed internetworking and networking equipment • applications running (or to be run) on the network • traffic patterns • physical location of sites, hosts, and users • rate of growth of the user community • physical and logical network layout

28. Assessing Costs • Internetwork is a strategic element in customer’s overall information system design • cost of internetwork is much more than the sum of your equipment purchase orders. • Must be viewed as a total cost-of-ownership issue • Must consider the entire life cycle of your internetworking environment

29. Costs to Consider • Equipment hardware and software costs • initial purchase and installation, maintenance, and projected upgrade costs • Performance tradeoff costs • cost of going from a five-second response time to a half-second response time

30. Costs to Consider • Installation costs • Installing a site's physical cable plant can be the most expensive element of a large network • installation labor • site modification • fees associated with local code conformance • costs incurred to ensure compliance with environmental restrictions (such as asbestos removal)

31. Costs to Consider • Expansion costs • cost of ripping out all thick Ethernet, adding additional functionality, or moving to a new location • Projecting future requirements and accounting for future needs saves time and money

32. Costs to Consider • Support costs • Complicated internetworks cost more to monitor, configure, and maintain • training • direct labor (network managers and administrators) • sparing • replacement costs • Also out-of-band management, SNMP management stations, and power

33. Costs to Consider • Cost of downtime • Evaluate the cost for every minute that a user is unable to access a file server or a centralized database • If the cost is high enough, fully redundant internetworks might be best option

34. Costs to Consider • Opportunity costs • Every choice made has an opposing alternative option • specific hardware platform • topology solution • level of redundancy • system integration alternative

35. Costs to Consider • Opportunity Costs • opportunity costs of not switching to newer technologies and topologies might be lost competitive advantage, lower productivity, and slower overall performance • Any effort to integrate opportunity costs into your analysis can help to make accurate comparisons at the beginning of the project

36. Costs to Consider • Sunken costs • Investment in existing cable plant, routers, concentrators, switches, hosts, and other equipment and software are sunken costs • If the sunken cost is high, might need to modify networks so that existing internetwork can continue to be utilized

37. Estimating Traffic: Work Load Modeling • Empirical work-load modeling • instrumenting a working internetwork • monitoring traffic for a given number of users, applications, and network topology • Characterize activity throughout a normal work day • type of traffic passed • level of traffic • response time of hosts • time to execute file transfers

38. Work Load Modeling • Extrapolating to the new internetwork's number of users, applications, and topology • Tools • Passive monitoring of an existing network • Measure activity and traffic generated by a known number of users

39. Work Load Modeling • Problem with modeling workloads on networks is that it is difficult to accurately pinpoint traffic load and network device performance as functions of the number of users, type of application, and geographical location

40. Work Load Modeling • Factors that influence the dynamics of the network • The time-dependent nature of network access • Differences associated with type of traffic • Routed and bridged traffic place different demands • The random (nondeterministic) nature of network traffic

41. Sensitivity Testing • Sensitivity testing involves breaking stable links and observing what happens • how traffic is rerouted • speed of convergence • whether any connectivity is lost • and whether problems arise in handling specific types of traffic

42. Sensitivity Testing • This empirical testing is a type of regression testing: • A series of specific modifications (tests) are repeated on different versions of network configurations • By monitoring the effects on the design variations, you can characterize the relative resilience of the design

43. System Engineering Techniques Applied to the Web

44. Quantitative Analysis Business model & measurable goals E-Business site architecture 1 2 Predict E-Business Site performance Measure E-Business Site 8 3 Forecast Workload Evolution 7 Characterize Customer Behavior 4 Obtain Performance Parameters Characterize Site Workload 6 5 Develop Performance Models

45. Metrics: - revenue/sec - response time - throughput Customer Model Workload Model Resource Model What-if questions regarding impacts of customer behavior What-if questions regarding impacts of workload, architecture, and configuration changes Customer, Workload, and Resource Models

46. What is a performance model? • A model of a system helps one understand some fundamental characteristics of the system • “All models are wrong, but some are useful!”

47. Zipfs Law • If one ranks the popularity of words in a given text (p) by their frequency (f) then f ~ 1/p • A few elements score very high and a very large number of elements score very low • Many phenomena on the web can be modeled by Zipfs law

48. Zipfs Law • P = k/r where P is the number of references to a document, r is the rank, k is a positive constant • Some documents are very popular while most documents receive just a few references • Can use Zipfs law to understand some asymptotic properties of web caching performance

49. Zipfs Law • Results obtained from Zipfs model are useful • to characterize WWW workloads • analyze document dissemination and replication strategies • model the behavior of caching and mirroring systems

50. Other Types of Models • CBMG • CSID • Resource model; represents the structure and the various components of an e-business site • Performance model; represents the way system’s resources are used by the workload and capture the main factors determining system performance