Distributed System: The Overall Architecture

1. Distributed System:The Overall Architecture

2. Previously… Internet---Topology and Reliability • Internet is a scale-free network • A small number of nodes have a large number of links while the majority of nodes only have a small number of links • Internet is robust to random failures, but vulnerable to targeted attacks

3. Previously… IT Architecture VS. IT Infrastructure • An IT architecture is a blueprint showing how the parts will interact and interrelate. • System, information, departments... • Multiplicity of structures and views • An IT infrastructure is the implementation of an architecture. • processors, software, databases, electronic links, data centers, standards, skills, electronic processes... • We now tend to divide computing into applications and infrastructures

4. 1 Distributed Computing The Importance of the IT Infrastructure 2

5. Host-Based Hierarchy • Master-slave relationship • A hierarchy in which the host is in control • Workload is shared among different levels of processors • Terminals are only access device • Data storage

6. Decentralized Stand-Alone Systems (1)

7. Decentralized Stand-Alone Systems (2) • They are decentralized, not distributed • Islands of computation problems for stand-alone systems • Ad hoc data flow to the corporate host • ERP

8. Peer-to-Peer LAN-Based Systems (1)

9. Peer-to-Peer LAN-Based Systems (2) • No hierarchy • Peer-to-peer communications • Interconnecting LANs rather than hierarchical communications through a central hub • No superior computer

10. Hybrid Enterprise-wide Systems (1)

11. Hybrid Enterprise-wide Systems (2) • The structure of choice for many years • Allows company to link "information islands" and automate business processes • Systems integration

12. Client-Server Systems (1) • Arose to take advantage of the processing capabilities of both host machines and PCs in the same system • The computational flow of pure client-server systems is asymmetric. • Constraints on the use of the client-server style • Limit the number of clients that can be connected to a server • Impose a restriction that servers cannot interact with other servers. • N-tiered client-server model • DB, application, presentation

13. Client-Server Systems (2) ---Distribution of Processing Distributed Presentation Remote Presentation Distributed Function Remote Data Management Distributed Database Data Management Data Management Data Management Data Management Data Management Application Function Application Function Application Function Data Management Server Presentation Network Application Function Application Function Application Function Client Presentation Presentation Presentation Presentation Presentation

14. Client-Server Systems (3) Remote Presentation • Most of the Web-based content systems today • DB sitting at the back • Web server(s) provide contents • Browsers display contents • Traditional GIS • Spatial data in DB • GIS server processing data and rendering image • Clients display maps Data Management Application Function Server Network Client Presentation

15. Client-Server Systems (4) Remote Data Management • Traditional C/S DB applications (fat client) • DBMS stores and retrieves data • Client applications process and display data • Client maintenance may be challenging Data Management Server Network Client Application Function Presentation

16. Client-Server Systems (5) Distributed Function • Some new Web-GIS systems • Image rendering moved to the client • Reduce network flow • Load balancing • Most of the online-gaming systems Data Management Application Function Server Network Client Application Function Presentation

17. Client-Server Systems (6) Distributed Database • Mobile computing • Less dynamic data stored locally (e.g. product category) • Data sysc. when connected • Some applications in retail chains • Local data copy to improve service availability • E.g. Microsoft access and SQL Server Data Management Server Network Client Data Management Application Function Presentation

18. Client-Server Systems (7) ---Three-tier Client-Server Style Server (usually DB server) connected to the network via one or more servers, and sometimes directly as well Multiple specialized servers, some possibly dedicated to middleware (application servers) Internet or LANs Clients, some of which may be portable

19. Client-Server Systems (8) ---Benefits and Drawbacks • Better access to information • Shift the focus of computing to user and empower employees • Increases organizational flexibility: • Allows new technology to be added more easily without affecting rest of system • Streamlines work flow between functional areas • Encourages people to work together via networks • Supports new organizational structures via its connectivity • Drawbacks • Not lower in cost • Easier for users, far more complex for IS • What looked like simple connections have turned into large, often fragile, complex systems

20. Internet-Based Computing (1) • In the late 1990s, the client-server trend was augmented by the Internet • The tenets of client-server remain • With Internet sitting at the heart • Ubiquitous computing • Various computing platforms: servers, PCs, handhelds • A new computing environment • Irregular request arrival patterns • Unpredictable user numbers • Thin client and fat client • Network computers & Java

21. Internet-Based Computing (2) ---Server-based Computing • Some services are by nature centralized • Web search • Corporate info query • Handheld device • Services reside on corporate servers • Applications can be securely accessed by any device, they can be updated directly on the server, and they do not have to be tailored to run on specific machines

22. P2P Computing---Definition (1) • Peer-to-peer systems are distributed systems consisting of interconnected nodes able to self-organize into network topologies with the purpose of sharing resources such as content, CPU cycles, storage and bandwidth, capable of adapting to failures and accommodating transient populations of nodes while maintaining acceptable connectivity and performance, without requiring the intermediation or support of a globally centralized server or authority. Androutsellis-Theotokis S,Spinellis D. A Survey of Peer-to-Peer Content Distribution Technologies, [J].ACM Computing Surveys,2004,36(4):335-371.

23. Peer-to-Peer Computing (2) • Components directly interact as peers by exchanging services. • Request/reply interaction • May involve complex bidirectional protocols of interactions • Two-way communication existing between two or more p2p components • P2P VS. C/S • Load distribution • The topology of the system changes at runtime as users enter the system or change their connections

24. Peer-to-Peer Computing (3) --- P2P Applications • Communication and Collaboration • Chat/Irc, Instant Messaging (MSN, Yahoo! Messenger), Chat/IRC • Distributed Computation • Seti@home [Sullivan III et al. 1997;SetiAtHome 2003] • genome@home [Larson et al. 2003; GenomeAtHome 2003] • Internet Service Support • Peer-to-peer multicast systems [VanRenesse et al. 2003], Internet indirection infrastructures [Stoica et al. 2002] • Database Systems • LRM [Bernstein et al. 2002] • Content Distribution • Sharing of digital media and other data between users; • Napster, Gnutella, BitTorrent, eMule... • P2P Video: PPLive, TVAnts

25. P2P Computing--- Content Distribution (4) • Creating a distributed storage medium that allows for the publishing, searching, and retrieval of files by members of its network. • May also include nonfunctional features like • Provisions for security, anonymity, fairness • Increased scalability and performance • Resource management and organization capabilities

26. Web Services (1) • Software modules invoked via URLs • permit widespread computer-to-computer use of the Internet. One computer program or Web Service makes a request of another Web Service to perform its task (or set of tasks) and pass back the answer • Hot topic about Web service • Next generation of distributed systems • Allows the building of large, complex systems by linking and integrating various modules • Makes the Internet the hub of computing

27. Web Services (2) • Permits flexible systems not possible before • Dynamic binding of Web services at execution time • Releases companies from building and maintaining systems in house • Will draw on existing systems • Wrapping – encapsulate functionality from an existing application in an XML envelope • Exposing – for use by others

28. Web Services (3) • WSDL • Service description, metadata describing the service • e.g. interface, location etc. • SOAP • A standard for XML-based information exchange between distributed applications • UDDI • A specification for distributed registries of web services Find Publish Bind

29. Web Services (4) Web Service Consumer Find a Service http://www.uddi.org Link to WSDL document UDDI Web Service How do we talk? (WSDL) http://yourserviice.com//?WSDL XML with service descriptions Let me talk to you (SOAP) http://yourservice.com/svc1 XML/SOAP BODY

30. Web Services (5) • Foundations for Web Services Service directory: UDDI Service description: WSDL Service interaction: SOAP Format description: XML Schema Data format: XML Communication Protocol: HTTP Communication Network: Internet

31. Web Services (6) • Significance of Web Services • Web Services offers an IT architecture based on the openness of the Internet. • This modularity permits handling a huge variety of possibilities by mixing and matching, and allows easier cross-company system linking • Companies thus only pay for the functionality they use when they use it, which reduces the number of IT assets companies need to house and maintain

32. Service-Oriented Architecture (1) • Move component interactions from hard-coded to dynamically discovered and invoked • By exposing the data and functions in a way that other systems can easily use • Service-Oriented Architecture (SOA) • An architecture is a formal description of a system, defining its components, their interrelationships, and the principles and guidelines governing their design and evolution over time • A service is a software component that can be accessed via a network to provide functionality to a service requester • SOA is a style of building reliable distributed systems that deliver functionality as services, with the additional emphasis on loose coupling between interacting services

33. Service-Oriented Architecture (2) • SOA refers to the design of a system, not to its implementation. • Service interaction in a service-oriented environment Registry Advertise ① ② Discover Service ③ Client Interact ④

34. Service-Oriented Architecture (3) • SOA features • Loosely-coupled • Coarse-grained • Standards-based • SOA benefits • Greater interoperability • Integration of legacy systems • Increased reuse • More agile business processes • Reduced maintenance costs

35. Service-Oriented Architecture (4) • Loose-coupling • The interacting software components minimize their in-built knowledge of each other • For example, having learned about a service’s existence, a client can discover its capabilities, its policies, its location, its interfaces and its supported protocols • The benefits of loose coupling include: • Flexibility • Scalability • Replaceability • Fault tolerance

36. Service-Oriented Architecture (5) • Achieving loose coupling between services • Web Service protocols • Publish-and-subscribe • Components interact via announced events • Messaging-oriented middleware (MOM) • Asynchronous message-passing with ability to store, route or transform messages in the process of delivery

37. Grid Computing (1) • A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities • Grid is a generalized network computing system that is supposed to scale to Internet levels and handle data and computation seamlessly

38. Grid Computing (2)

39. Grid Computing (3) • Resource management in Grid systems involves managing the basic elements • Grid elements • Processing elements • Compute servers (double each 18 months) • Handhelds • File servers • Networks (double each 9 months) • Playstations, cell phones etc… • Storage elements • Network elements

40. Grid Computing (4) • Grid systems can be classified depending on their usage:

41. Grid Computing (5) • Computational Grid: • Denotes a system that has a higher aggregate capacity than any of its constituent machine • Data Grid: • Systems that provide an infrastructure for synthesizing new information from data repositories such as digital libraries or data warehouses • Applications for these systems would be special purpose data mining that correlates information from multiple different high volume data sources • Service Grid: • Systems that provide services that are not provided by any single machine

42. Grid Computing (6) • Grid Computing Benefits • Exploit Underutilized resources • CPU Scavenging, Hotspot leveling • Resource Balancing • Virtualize resources across an enterprise • Data Grids, Compute Grids • Enable collaboration for virtual organizations

43. 1 Distributed Computing The Importance of the IT Infrastructure 2

44. IT Infrastructure---An Overview • The shared and reliable services that provide the foundation for the enterprise IT portfolio • Enabling business applications sitting above • Difficult to cost-justify • Technologists and business people have different views of infrastructure • Peter Weill et al presented a model that meshes different views together • Gives technologists and business users a common language

45. The Structure of IT Infrastructures (1)

46. The Structure of IT Infrastructures (2) • Four layers of IT infrastructure • IT components • Technology components, such as computers, DBMS packages, OS etc. • Generally not understood by business people • Human IT infrastructure • Translation of the IT components layer into services • Experts' knowledge, skills, experience and standards • Shared IT services • The infrastructure as a set of services that users can draw upon and share to conduct business. • Shared and standard applications • Stable applications such as accounting and HR

47. Public Infrastructure • Weill’s model in turn sits on top of the public infrastructure • E.g. the Internet, vendors, telecomm companies. • Similarity between IT infrastructure and public infrastructure such as roads, hospital, schools etc. • Everyone wants but no-one wants to pay • Provided by a "central authority" • Government or IT Department • Delicate and "difficult" investment balance

48. Three Views of Infrastructure • Economies of scale (Utility): • Infrastructure cost as an administrative expense • Minimize expense • Outsourcing may be viewed favorably because the IT infrastructure is not seen as strategic • Support for business programs (Dependent): • Infrastructure treated as business expense • Measured by short-term business benefits • Infrastructure planning in current business plan • Flexibility to meet changes in the marketplace (Enabling): • Primary benefit long-term flexibility • Intended to provide the foundation for changing direction in the future • IT cost seen as business investment