1 / 29

Distributed Computing Primer

Distributed Computing Primer. CMSC 491/691 Hadoop-Based Distributed Computing Spring 2014 Adam Shook. Some content adapted from Dr. Kalpakis’s CMSC 621 slides. Agenda. Evolution of Computing Infrastructure Networking Infrastructure Properties of Distributed Systems

vonda
Download Presentation

Distributed Computing Primer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Computing Primer CMSC 491/691 Hadoop-Based Distributed Computing Spring 2014 Adam Shook Some content adapted from Dr. Kalpakis’s CMSC 621 slides

  2. Agenda • Evolution of Computing Infrastructure • Networking Infrastructure • Properties of Distributed Systems • Example System Architectures

  3. Evolution of Computing Infrastructure

  4. Mainframe – 50s to 70s • Custom hardware • Custom low-level specialized code • Very expensive solutions

  5. Client/Server – 80s to 00s • IT-led architectures • More portable solutions • Scalable solutions based on demand • Reign of the Enterprise Data Warehouse

  6. Cloud – 00s to Today • Consumer-grade infrastructure • Growing IaaS and PaaS markets • Data revolution • Focus on applications and not infrastructure

  7. Where does Hadoop fit? • A piece of your data infrastructure • Can crunch data for analytics • Can expose data for web applications • Exploration of raw data • Augments today’s infrastruture • IMO, a big toolbox that can do a bit of everything

  8. Networking Infrastructure

  9. Single Server Server Scale Up HDD CPU RAM NIC HDD CPU RAM NIC Faster CPUs Bigger Storage Scale Out More Servers

  10. Local-Area Network (LAN) Rack Rack Server Server Server Server Server Server Server Server HDD HDD HDD HDD HDD HDD HDD HDD CPU CPU CPU CPU CPU CPU CPU CPU RAM RAM RAM RAM RAM RAM RAM RAM NIC NIC NIC NIC NIC NIC NIC NIC WAN Gateway HDD HDD HDD HDD HDD HDD HDD HDD CPU CPU CPU CPU CPU CPU CPU CPU RAM RAM RAM RAM RAM RAM RAM RAM NIC NIC NIC NIC NIC NIC NIC NIC

  11. Wide Area Network (WAN) London, England New York, NY Beijing, China

  12. Properties of Distributed Systems

  13. Distributed Systems • The development of low-cost powerful microprocessors, together with the invention of high speed networks, enable us to construct computer systems by connecting a large number of computers • A distributed system is a collection of independent computers that appears to its users as a single coherent system.

  14. Transparency Sometimes it make sense to expose distribution rather than hide it

  15. Properties of Distributed Systems • Reliability • Scalability • Availability • Efficiency • CAP Theorem

  16. Reliability • Can the system deliver services in face of several component failures?

  17. Scalability • Can the system scale to support a growing number of tasks?

  18. Availability • How much latency is imposed on the system when a failure occurs?

  19. Efficiency • How efficient is the system, in terms of latency and throughput?

  20. CAP Theorem • Consistent • Available • Partition Tolerant • Trade-off between Consistency and Availability

  21. Stateful vs. Stateless • Whether or not a distributed system saves their state on an attached device for recovery

  22. Distributed System Pitfalls • Peter Deutch identifies false assumptions made when building distributed systems • The network is reliable • The network is secure • The network is homogeneous • The topology does not change • Latency is zero • Bandwidth is infinite • Transport cost is zero • There is one administrator

  23. Example System Architectures

  24. Simple Client/Server

  25. Multi-Tiered Client/Server

  26. Round-Robin Client/Server

  27. MVC Example

  28. P2P Systems

  29. References • http://webdam.inria.fr/Jorge/html/wdmch15.html • Google Images

More Related