1 / 60

The Future of Distributed Systems .

™. The Future of Distributed Systems. Jim Gray Researcher Microsoft Corp. Gray@Microsoft.com. Outline. Global forces Moore’s, Metcalf’s, Bell’s, Bills, Andy’s laws Micro dollars per transaction Cyber-content is key value because distribution costs go to zero

prater
Download Presentation

The Future of Distributed Systems .

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ™ The Future of Distributed Systems. Jim Gray Researcher Microsoft Corp. Gray@Microsoft.com

  2. Outline • Global forces • Moore’s, Metcalf’s, Bell’s, Bills, Andy’s laws • Micro dollars per transaction • Cyber-content is key value because distribution costs go to zero • Distributed Systems Concepts and terms • Key software technologies • objects, transactions

  3. Metcalf’s LawNetwork Utility = Users2 • How many connections can it make? • 1 user: no utility • 100,000 users: a few contacts • 1 million users: many on Net • 1 billion users: everyone on Net • That is why the Internet is so “hot” • Exponential benefit

  4. Moore’s First Law • XXX doubles every 18 months 60% increase per year • Micro processor speeds • Chip density • Magnetic disk density • Communications bandwidthWAN bandwidth approaching LAN speeds • Exponential growth: • The past does not matter • 10x here, 10x there, soon you’re talking REAL change • PC costs decline faster than any other platform • Volume and learning curves • PCs will be the building bricks of all future systems 1GB 128MB 1 chip memory size ( 2 MB to 32 MB) 8MB 1MB 128KB 8KB 1980 1990 2000 1970 256M bits: 1K 4K 16K 64K 256K 1M 4M 16M 64M

  5. $/MB of DRAM 1000000 10000 100 1 1970 1980 1990 2000 $/MB of DISK 10,000 100 1 .01 1970 1980 1990 2000 Bumps In The Moore’s Law Road • DRAM: • 1988: United States anti-dumping rules • 1993-1995: ?price flat • Magnetic disk: • 1965-1989: 10x/decade • 1989-1996: 4x/3year! 100X/decade

  6. Gordon Bell’s Seven Price Tiers 10$: wrist watch computers 100$: pocket/ palm computers 1,000$: portable computers 10,000$: personal computers (desktop) 100,000$: departmental computers (closet) 1,000,000$: site computers (glass house) 10,000,000$: regional computers (glass castle) Super server: costs more than $100,000“Mainframe”: costs more than $1 million Must be an array of processors, disks, tapes, comm ports

  7. Bell’s Evolution Of Computer Classes Technology enables two evolutionary paths:1. constant performance, decreasing cost2. constant price, increasing performance Mainframes (central) Minis (dep’t.) Log price WSs PCs (personals) ?? Time 1.26 = 2x/3 yrs -- 10x/decade; 1/1.26 = .8 1.6 = 4x/3 yrs --100x/decade; 1/1.6 = .62

  8. R&D 8% Tax 5% SG&A 22% Software Economics Microsoft: $9 billion • An engineer costs about$150,000/year • R&D gets [5%…15%]of budget • Need [$3 million…$1 million] revenue per engineer Profit 24% R&D 16% SG&A 34% Tax 13% Productand Service 13% Intel: $16 billion IBM: $72 billion Oracle: $3 billion Profit 15% Profit 6% R&D 9% R&D 8% Profit 22% Tax 7% SG&A 11% Tax SG&A 12% P&S 59% 43% P&S 47% P&S 26%

  9. Software Economics: Bill’s Law Fixed_ Cost • Bill Joy’s law (Sun): don’t write software for less than 100,000 platforms @$10 million engineering expense, $1,000 price • Bill Gate’s law:don’t write software for less than 1,000,000 platforms @$10 engineering expense, $100 price • Examples: • UNIX versus Windows NT: $3,500 versus $500 • Oracle versus SQL-Server: $100,000 versus $6,000 • No spreadsheet or presentation pack on UNIX/VMS/... • Commoditization of base software and hardware Price Marginal _Cost = + Units

  10. Gordon Bell’s Platform Economics 100000 10000 Traditional computers: custom or semi-custom, high-tech and high-touch New computers: high-tech and no-touch Price (K$) 1000 Volume (K) Applicationprice 100 10 1 0.1 0.01 WS Browser Mainframe Computer type

  11. Grove’s LawThe New Computer Industry • Horizontal integrationis new structure • Each layer picks best from lower layer • Desktop (C/S) market • 1991: 50% • 1995: 75% Example Function Operation AT&T Integration EDS Applications SAP Middleware Oracle Baseware Microsoft Systems Compaq Intel & Seagate Silicon & Oxide

  12. Outline • Global forces • Moore’s, Metcalf’s, Bell’s, Bills, Andy’s laws • Micro dollars per transaction • Cyber-content is key value because distribution costs go to zero • Distributed Systems Concepts and terms • Key software technologies • objects, transactions

  13. 1987: 256 tps Benchmark • 14 M$ computer (Tandem) • A dozen people • False floor, 2 rooms of machines Admin expert Hardware experts A 32 node processor array Auditor Network expert Simulate 25,600 clients Manager Performance expert OS expert DB expert A 40 GB disk array (80 drives)

  14. 1997: 10 years later1 Person and 1 box = 1250 tps • 1 Breadbox ~ 5x 1987 machine room • 23 GB is hand-held • One person does all the work • Cost/tps is 1,000x less1 micro dollar per transaction 4x200 Mhz cpu 1/2 GB DRAM 12 x 4GB disk Hardware expert OS expert Net expert DB expert App expert 3 x7 x 4GB disk arrays

  15. mainframe mini price micro time What Happened? • Moore’s law: Things get 4x better every 3 years(applies to computers, storage, and networks) • New Economics: Commodityclass price/mips software $/mips k$/yearmainframe 10,000 100 minicomputer 100 10microcomputer 10 1 • GUI: Human - computer tradeoffoptimize for people, not computers

  16. ? performance 1985 1995 2005 What Happens Next • Last 10 years: 1000x improvement • Next 10 years: ???? • Today: text and image servers are free 1 m$/hit cost 70,000m$/hit advertising revenue • Advertising pays for them • Content is only “real” expense • “You ain’t seen nothing yet!”

  17. Kinds Of Information Processing Point-to-point Broadcast Lecture Concert Conversation Money Network Immediate Book Newspaper Mail Time-shifted Database It’s ALL going electronic Immediate is being stored for analysis (so ALL database) Analysis and automatic processing are being added

  18. Why Put EverythingIn Cyberspace? Point-to-point OR broadcast Low rent - min $/byte Shrinks time - now or later Shrinks space - here or there Automate processing - knowbots Network Immediate OR time-delayed Locate Process Analyze Summarize Database

  19. Billions Of Clients • Every device will be “intelligent” • Doors, rooms, cars… • Computing will be ubiquitous

  20. Billions Of ClientsNeed Millions Of Servers • All clients networked to servers • May be nomadicor on-demand • Fast clients wantfaster servers • Servers provide • Shared Data • Control • Coordination • Communication Clients Mobileclients Fixedclients Servers Server Super server

  21. 3 1 MM 10 nano-second ram 10 microsecond ram 10 millisecond disc 10 second tape archive ThesisMany little beat few big $1 million $10 K $100 K Pico Processor Micro Nano 10 pico-second ram 1 MB Mini Mainframe 10 0 MB 1 0 GB 1 TB 1 00 TB 1.8" 2.5" 3.5" 5.25" 1 M SPECmarks, 1TFLOP 106 clocks to bulk ram Event-horizon on chip VM reincarnated Multiprogram cache, On-Chip SMP 9" 14" • Smoking, hairy golf ball • How to connect the many little parts? • How to program the many little parts? • Fault tolerance?

  22. CPU 50 GB Disc 5 GB RAM Future Super Server:4T Machine • Array of 1,000 4B machines • 1 bps processors • 1 BB DRAM • 10 BB disks • 1 Bbps comm lines • 1 TB tape robot • A few megabucks • Challenge: • Manageability • Programmability • Security • Availability • Scaleability • Affordability • As easy as a single system Cyber Brick a 4B machine Future servers are CLUSTERS of processors, discs Distributed database techniques make clusters work

  23. The Hardware Is In Place…And then a miracle occurs ? • SNAP: scaleable networkand platforms • Commodity-distributedOS built on: • Commodity platforms • Commodity networkinterconnect • Enables parallel applications

  24. Outline • Global forces • Moore’s, Metcalf’s, Bell’s, Bills, Andy’s laws • Micro dollars per transaction • Cyber-content is key value because distribution costs go to zero • Distributed Systems Concepts and terms • Key software technologies • objects, transactions

  25. OutlineConcepts and Terminology • Why Distributed • Distributed data & objects • Distributed execution • Three tier architectures • Transaction concepts

  26. What’s a Distributed System? • Centralized: • everything in one place • stand-alone PC or Mainframe • Distributed: • some parts remote • distributed users • distributed execution • distributed data

  27. Why Distribute? • No best organization • Companies constantly swing between • Centralized: focus, control, economy • Decentralized: adaptive, responsive, competitive • Why distribute? • reflect organization or application structure • empower users / producers • improve service (response / availability) • distributed load • use PC technology (economics)

  28. What Should Be Distributed? • Users and User Interface • Thin client • Processing • Trim client • Data • Fat client • Will discuss tradeoffs later Presentation workflow Business Objects Database

  29. Transparency in Distributed Systems • Make distributed system as easy to use and manage as a centralized system • Give a Single-System Image • Location transparency: • hide fact that object is remote • hide fact that object has moved • hide fact that object is partitioned or replicated • Name doesn’t change if object is replicated, partitioned or moved.

  30. OutlineConcepts and Terminology • Why Distribute • Distributed data & objects • Partitioned • Replicated • Distributed execution • remote procedure call • queues • Three tier architectures • Transaction concepts

  31. threads shared memory Distributed ExecutionThreads and Messages • Thread is Execution unit(software analog of cpu+memory) • Threads execute at a node • Threads communicate via • Shared memory (local) • Messages (local and remote) messages

  32. Peer-to-Peer or Client-Server • Peer-to-Peer is symmetric: • Either side can send • Client-server • client sends requests • server sends responses • simple subset of peer-to-peer request response

  33. y = pObj->f(x); x f() return val; y = val; val Remote Procedure Call: The key to transparency • Object may be local or remote • Methods on object work wherever it is. • Local invocation

  34. proxy Obj Local? x x stub marshal un marshal x un marshal pObj->f(x) marshal Obj Local? x f() f() return val; return val; val val y = val; val val Remote Procedure Call: The key to transparency • Remote invocation y = pObj->f(x);

  35. Transaction Object Request Broker (ORB)Orchestrates RPC • Registers Servers • Manages pools of servers • Connects clients to servers • Does Naming, request-level authorization, • Provides transaction coordination (new feature) • Old names: • Transaction Processing Monitor, • Web server, • NetWare Object-Request Broker

  36. Microsoft DCOM based on OSF-DCE Technology DCOM and ActiveX extend it UNIX International Open software Foundation (OSF) ODBC XA / TX Object Management Group (OMG) NT OSF DCE DCE RPC GUIDs IDL DNS Kerberos Solaris COM CORBA History and Alphabet Soup 1985 X/Open 1990 1995 Open Group

  37. ActiveX and COM • COM is Microsoft model, engine inside OLE ALL Microsoft software is based on COM (ActiveX) • CORBA + OpenDoc is equivalent • Heated debate over which is best • Both share same key goals: • Encapsulation: hide implementation • Polymorphism: generic operationskey to GUI and reuse • Versioning: allow upgrades • Transparency: local/remote • Security: invocation can be remote • Shrink-wrap: minimal inheritance • Automation: easy • COM now managed by the Open Group

  38. Linking And EmbeddingObjects are data modules;transactions are execution modules • Link: pointer to object somewhere else • Think URL in Internet • Embed: bytesare here • Objects may be active; can callback to subscribers

  39. Transaction Bottom Line Re ORBs • Microsoft Promises Cairo distributed objects, secure, transparent, fast invocation • Netscape promises the CORBA • Both will deliver • Customers can pick the best one Object-Request Broker

  40. OutlineConcepts and Terminology • Why Distributed • Distributed data & objects • Distributed execution • remote procedure call • queues • Three tier architectures • what • why • Transaction concepts

  41. Fat Thin Fat Thin Work Distribution Spectrum • Presentation and plug-ins • Workflow manages session & invokes objects • Business objects • Database Presentation workflow Business Objects Database

  42. Server green screen 3270 Active Transaction Processing Evolution to Three TierIntelligence migrated to clients • Mainframe Batch processing (centralized) • Dumb terminals & Remote Job Entry • Intelligent terminals database backends • Workflow SystemsObject Request BrokersApplication Generators Mainframe cards TP Monitor ORB

  43. Mosaic NS & IE Active Web Evolution to Three TierIntelligence migrated to clients (like TP) Web Server WAIS • Character-mode clients, smart servers • GUI Browsers - Web file servers • GUI Plugins - Web dispatchers - CGI • Smart clients - Web dispatcher (ORB)pools of app servers (ISAPI, Viper)workflow scripts at client & server archie ghopher green screen

  44. PC Evolution to Three TierIntelligence migrated to server • Stand-alone PC (centralized) • PC + File & print servermessage per I/O • PC + Database server message per SQL statement • PC + App server message per transaction • ActiveX Client, ORB ActiveX server, Xscript IO request reply disk I/O SQL Statement Transaction

  45. The Pattern: Three Tier Computing • Clients do presentation, gather input • Clients do some workflow (Xscript) • Clients send high-level requests to ORB (Object Request Broker) • ORB dispatches workflows and business objects -- proxies for client, orchestrate flows & queues • Server-side workflow scripts call on distributed business objects to execute task Presentation workflow Business Objects Database

  46. Web Client HTML VB Java plug-ins VBscritpt JavaScrpt Middleware ORB TP Monitor Web Server... Object server Pool VB or Java Script Engine VB or Java Virt Machine HTTP+ DCOM ORB Internet DCOM (oleDB, ODBC,...) LU6.2 Legacy Gateways IBM The Three Tiers Object & Data server.

  47. Why Did Everyone Go To Three-Tier? • Manageability • Business rules must be with data • Middleware operations tools • Performance (scaleability) • Server resources are precious • ORB dispatches requests to server pools • Technology & Physics • Put UI processing near user • Put shared data processing near shared data Presentation workflow Business Objects Database

  48. MOM’s Business Objects DAD’sRaw Data Customer comes to store with list Gives list to clerk Clerk gets goods, makes invoice Customer pays clerk, gets goods Customer comes to store Takes what he wants Fills out invoice Leaves money for goods Easy to manage Clerks controls access Encapsulation Easy to build No clerks Why Put Business Objects at Server?

  49. What Middleware Does ORB, TP Monitor, Workflow Mgr, Web Server • Registers transaction programs workflow and business objects (DLLs) • Pre-allocates server pools • Provides server execution environment • Dynamically checks authority(request-level security) • Does parameter binding • Dispatches requests to servers • parameter binding • load balancing • Provides Queues • Operator interface

  50. Server Side ObjectsEasy Server-Side Execution • ORB gives simple execution environment • Object gets • start • invoke • shutdown • Everything else is automatic • Drag & Drop Business Objects A Server Network Receiver Queue Management Connections Security Context Configuration Thread Pool Service logic Synchronization Shared Data

More Related