Scalable Computing: The Future of Servers

1. ™ Scaleable ComputingJim GrayMicrosoft CorporationGray@Microsoft.com

2. Thesis: Scaleable Servers • Scaleable Servers • Commodity hardware allows new applications • New applications need huge servers • Clients and servers are built of the same “stuff” • Commodity software and • Commodity hardware • Servers should be able to • Scale up (grow node by adding CPUs, disks, networks) • Scale out (grow by adding nodes) • Scale down (can start small) • Key software technologies • Objects, Transactions, Clusters, Parallelism

3. 1987: 256 tps Benchmark • 14 M$ computer (Tandem) • A dozen people • False floor, 2 rooms of machines Admin expert Hardware experts A 32 node processor array Auditor Network expert Simulate 25,600 clients Manager Performance expert OS expert DB expert A 40 GB disk array (80 drives)

4. 1988: DB2 + CICS Mainframe65 tps • IBM 4391 • Simulated network of 800 clients • 2m$ computer • Staff of 6 to do benchmark 2 x 3725 network controllers Refrigerator-sized CPU 16 GB disk farm 4 x 8 x .5GB

5. 1997: 10 years later1 Person and 1 box = 1250 tps • 1 Breadbox ~ 5x 1987 machine room • 23 GB is hand-held • One person does all the work • Cost/tps is 1,000x less25 micro dollars per transaction 4x200 Mhz cpu 1/2 GB DRAM 12 x 4GB disk Hardware expert OS expert Net expert DB expert App expert 3 x7 x 4GB disk arrays

6. mainframe mini price micro time What Happened? • Moore’s law: Things get 4x better every 3 years(applies to computers, storage, and networks) • New Economics: Commodityclass price/mips software $/mips k$/yearmainframe 10,000 100 minicomputer 100 10microcomputer 10 1 • GUI: Human - computer tradeoffoptimize for people, not computers

7. ? performance 1985 1995 2005 What Happens Next • Last 10 years: 1000x improvement • Next 10 years: ???? • Today: text and image servers are free 25 m$/hit => advertising pays for them • Future:video, audio, … servers are free“You ain’t seen nothing yet!”

8. Kinds Of Information Processing Point-to-point Broadcast Lecture Concert Conversation Money Network Immediate Book Newspaper Mail Time-shifted Database It’s ALL going electronic Immediate is being stored for analysis (so ALL database) Analysis and automatic processing are being added

9. Why Put EverythingIn Cyberspace? Point-to-point OR broadcast Low rent - min $/byte Shrinks time - now or later Shrinks space - here or there Automate processing - knowbots Network Immediate OR time-delayed Locate Process Analyze Summarize Database

10. Magnetic Storage Cheaper Than Paper • File cabinet: cabinet (four drawer) 250$ paper (24,000 sheets) 250$ space (2x3 @ 10$/ft2) 180$ total 700$ 3¢/sheet • Disk: disk (4 GB =) 800$ ASCII: 2 mil pages 0.04¢/sheet (80x cheaper) • Image: 200,000 pages 0.4¢/sheet (8x cheaper) • Store everything on disk

11. DatabasesInformation at Your Fingertips™ Information Network™Knowledge Navigator™ • All information will be in anonline database (somewhere) • You might record everything you • Read: 10MB/day, 400 GB/lifetime(eight tapes today) • Hear: 400MB/day, 16 TB/lifetime(three tapes/year today) • See: 1MB/s, 40GB/day, 1.6 PB/lifetime (maybe someday)

12. People Name Address David NY Mike Berk Won Austin Database StoreALL Data Types • The old world: • Millions of objects • 100-byte objects • The new world: • Billions of objects • Big objects (1 MB) • Objects have behavior (methods) • Paperless office • Library of Congress online • All information online • Entertainment • Publishing • Business • WWW and Internet People Name Voice Address Papers Picture NY David Mike Berk Won Austin

13. Billions Of Clients • Every device will be “intelligent” • Doors, rooms, cars… • Computing will be ubiquitous

14. Billions Of ClientsNeed Millions Of Servers • All clients networked to servers • May be nomadicor on-demand • Fast clients wantfaster servers • Servers provide • Shared Data • Control • Coordination • Communication Clients Mobileclients Fixedclients Servers Server Super server

15. 3 1 MM 10 nano-second ram 10 microsecond ram 10 millisecond disc 10 second tape archive ThesisMany little beat few big $1 million $10 K $100 K Pico Processor Micro Nano 10 pico-second ram 1 MB Mini Mainframe 10 0 MB 1 0 GB 1 TB 1 00 TB 1.8" 2.5" 3.5" 5.25" 1 M SPECmarks, 1TFLOP 106 clocks to bulk ram Event-horizon on chip VM reincarnated Multiprogram cache, On-Chip SMP 9" 14" • Smoking, hairy golf ball • How to connect the many little parts? • How to program the many little parts? • Fault tolerance?

16. CPU 50 GB Disc 5 GB RAM Future Super Server:4T Machine • Array of 1,000 4B machines • 1 bps processors • 1 BB DRAM • 10 BB disks • 1 Bbps comm lines • 1 TB tape robot • A few megabucks • Challenge: • Manageability • Programmability • Security • Availability • Scaleability • Affordability • As easy as a single system Cyber Brick a 4B machine Future servers are CLUSTERS of processors, discs Distributed database techniques make clusters work

17. Performance = Storage Accesses not Instructions Executed • In the “old days” we counted instructions and IO’s • Now we count memory references • Processors wait most of the time Where the time goes: clock ticks used by AlphaSort Components 70 MIPS “real” apps have worse Icache misses so run at 60 MIPS if well tuned, 20 MIPS if not Sort Disc Wait Sort OS Disc Wait Memory Wait I-Cache Miss B-Cache D-Cache Data Miss Miss

18. Sacramento Storage Latency: How Far Away is the Data? Andromeda 9 10 Tape /Optical 2,000 Years Robot 6 Pluto Disk 2 Years 10 Clock Ticks 1.5 hr 100 Memory This Campus 10 10 min On Board Cache 2 On Chip Cache This Room 1 Registers My Head 1 min

19. The Hardware Is In Place…And then a miracle occurs ? • SNAP: scaleable networkand platforms • Commodity-distributedOS built on: • Commodity platforms • Commodity networkinterconnect • Enables parallel applications

20. Thesis: Scaleable Servers • Scaleable Servers • Commodity hardware allows new applications • New applications need huge servers • Clients and servers are built of the same “stuff” • Commodity software and • Commodity hardware • Servers should be able to • Scale up (grow node by adding CPUs, disks, networks) • Scale out (grow by adding nodes) • Scale down (can start small) • Key software technologies • Objects, Transactions, Clusters, Parallelism

21. Scaleable ServersBOTH SMP And Cluster Grow up with SMP; 4xP6is now standard Grow out with cluster Cluster has inexpensive parts SMP superserver Departmentalserver Personalsystem Clusterof PCs

22. SMPs Have Advantages • Single system image easier to manage, easier to program threads in shared memory, disk, Net • 4x SMP is commodity • Software capable of 16x • Problems: • >4 not commodity • Scale-down problem (starter systems expensive) • There is a BIGGEST one SMP superserver Departmentalserver Personalsystem

23. 1-TB home page www.SQL.1TB.com Todo loo da loo-rah, ta da ta-la la la Todo loo da loo-rah, ta da ta-la la la Todo loo da loo-rah, ta da ta-la la la Todo loo da loo-rah, ta da ta-la la la Todo loo da loo-rah, ta da ta-la la la Todo loo da loo-rah, ta da ta-la la la Todo loo da loo-rah, ta da ta-la la la TM 1-TB SQL Server DBSatellite and aerial photos Supportfiles Building the Largest Node • There is a biggest node (size grows over time) • Today, with NT, it is probably 1TB • We are building it(with help from DEC and SPIN2) • 1 TB GeoSpatial SQL Server database • (1.4 TB of disks = 320 drives). • 30K BTU, 8 KVA, 1.5 metric tons. • Will put it on the Web as a demo app. • 10 meter image of the ENTIRE PLANET. • 2 meter image of interesting parts (2% of land)One pixel per meter = 500 TB uncompressed. • Better resolution in US (courtesy of USGS).

24. What’s TeraByte? • 1 Terabyte: 1,000,000,000 business letters 150 miles of book shelf 100,000,000 book pages 15 miles of book shelf 50,000,000 FAX images 7 miles of book shelf 10,000,000 TV pictures (mpeg) 10 days of video 4,000 LandSat images 16 earth images (100m) 100,000,000 web page 10 copies of the web HTML • Library of Congress (in ASCII) is 25 TB 1980: $200 million of disc 10,000 discs $5 million of tape silo 10,000 tapes 1997: $200 k$ of magnetic disc 48 discs $30 k$ nearline tape 20 tapes Terror Byte !

25. TB DB User Interface Next

26. Tpc-C Web-Based Benchmarks • Client is a Web browser (7,500 of them!) • Submits • Order • Invoice • Query to server via Web page interface • Web server translates to DB • SQL does DB work • Net: • easy to implement • performance is GREAT! HTTP IIS = Web ODBC SQL

27. TPC-C Shows How Far SMPs have come • Performance is amazing: • 2,000 users is the min! • 30,000 users on a 4x12 alpha cluster (Oracle) • Peak Performance: 30,390 tpmC @ $305/tpmC (Oracle/DEC) • Best Price/Perf: 6,712 tpmC @ $65/tpmC (MS SQL/DEC/Intel) • graphs show UNIX high price & diseconomy of scaleup

28. TPC C SMP Performance • SMPs do offer speedup • but 4x P6 is better than some 18x MIPSco

29. The TPC-C RevolutionShows How Far NT and SQL Server have Come • Economy of scale on Windows NT • Recent Microsoft SQL Server benchmarks are Web-based tpmC and $/tpmC MS SQL Server: Economy of Scale & Low Price $250 DB2 $200 Informix $150 Better Price $/TPM-C Microsoft $100 Oracle $50 Sybase $0 0 1000 2000 3000 4000 5000 6000 7000 8000 Performance tpmC

30. What Happens To Prices? • No expensive UNIX front end (20$/tpmC) • No expensive TP monitor software (10$/tpmC) • => 65$/tpmC

31. SMP superserver Departmentalserver Personalsystem Grow UP and OUT 1 Terabyte DB • Cluster: • a collection of nodes • as easy to program and manage as a single node 1 billion transactions per day

32. Clusters Have Advantages • Clients and servers made from the same stuff • Inexpensive: • Built with commodity components • Fault tolerance: • Spare modules mask failures • Modular growth • Grow by adding small modules • Unlimited growth: no biggest one

33. Key goals: Easy: to install, manage, program Reliable: better than a single node Scaleable: added parts add power Microsoft & 60 vendors defining NT clusters Almost all big hardware and software vendors involved No special hardware needed - but it may help Enables Commodity fault-tolerance Commodity parallelism (data mining, virtual reality…) Also great for workgroups! Initial: two-node failover Beta testing since December96 SAP, Microsoft, Oracle giving demos. File, print, Internet, mail, DB, other services Easy to manage Each node can be 4x (or more) SMP Next (NT5) “Wolfpack” is modest size cluster About 16 nodes (so 64 to 128 CPUs) No hard limit, algorithms designedto go further Windows NT clusters

34. Private Private Shared SCSI disk strings disks disks B A Clients SQL Server™ Failover Using “Wolfpack” Windows NT Clusters • Each server “owns” half the database • When one fails… • The other server takes over the shared disks • Recovers the database and serves it

35. Billion Transactions per DayProject • Building a 20-node Windows NT Cluster (with help from Intel)> 800 disks • All commodity parts • Using SQL Server & DTC distributed transactions • Each node has 1/20 th of the DB • Each node does 1/20 th of the work • 15% of the transactions are “distributed”

36. How Much Is 1 Billion Transactions Per Day? • 1 Btpd = 11,574 tps (transactions per second)~ 700,000 tpm (transactions/minute) • AT&T • 185 million calls (peak day worldwide) • Visa ~20 M tpd • 400 M customers • 250,000 ATMs worldwide • 7 billion transactions / year (card+cheque) in 1994 Millions of transactions per day 1,000. 100. 10. Mtpd 1. 0.1 AT&T Visa BofA NYSE 1 Btpd

37. ParallelismThe OTHER aspect of clusters • Clusters of machines allow two kinds of parallelism • Many little jobs: online transaction processing • TPC-A, B, C… • A few big jobs: data search and analysis • TPC-D, DSS, OLAP • Both give automatic parallelism

38. Kinds of Parallel Execution Any Any Sequential Sequential Pipeline Program Program Partition outputs split N ways inputs merge M ways Any Any Sequential Sequential Program Program Jim Gray & Gordon Bell: VLDB 95 Parallel Database Systems Survey

39. Data RiversSplit + Merge Streams N X M Data Streams M Consumers N producers River • Producers add records to the river, • Consumers consume records from the river • Purely sequential programming. • River does flow control and buffering • does partition and merge of data records • River = Split/Merge in Gamma = Exchange operator in Volcano. Jim Gray & Gordon Bell: VLDB 95 Parallel Database Systems Survey

40. Partitioned Execution Spreads computation and IO among processors Partitioned data gives NATURAL parallelism Jim Gray & Gordon Bell: VLDB 95 Parallel Database Systems Survey

41. N x M way Parallelism N inputs, M outputs, no bottlenecks. Partitioned Data Partitioned and Pipelined Data Flows Jim Gray & Gordon Bell: VLDB 95 Parallel Database Systems Survey

42. 1,000 MIPS 32 $ 1 MIPS 1 $ .03$/MIPS The Parallel Law Of Computing • Grosch's Law: • Parallel Law: • Needs: • Linear speedup and linear scale-up • Not always possible 2x $ is 4x performance 2x $ is2x performance 1,000 MIPS 1,000 $ 1 MIPS 1 $

43. Thesis: Scaleable Servers • Scaleable Servers • Commodity hardware allows new applications • New applications need huge servers • Clients and servers are built of the same “stuff” • Commodity software and • Commodity hardware • Servers should be able to • Scale up (grow node by adding CPUs, disks, networks) • Scale out (grow by adding nodes) • Scale down (can start small) • Key software technologies • Objects, Transactions, Clusters, Parallelism

44. The BIG PictureComponents and transactions • Software modules are objects • Object Request Broker (a.k.a., Transaction Processing Monitor) connects objects(clients to servers) • Standard interfaces allow software plug-ins • Transaction ties execution of a “job” into an atomic unit: all-or-nothing, durable, isolated Object Request Broker

45. ActiveX and COM • COM is Microsoft model, engine inside OLE ALL Microsoft software is based on COM (ActiveX) • CORBA + OpenDoc is equivalent • Heated debate over which is best • Both share same key goals: • Encapsulation: hide implementation • Polymorphism: generic operationskey to GUI and reuse • Versioning: allow upgrades • Transparency: local/remote • Security: invocation can be remote • Shrink-wrap: minimal inheritance • Automation: easy • COM now managed by the Open Group

46. Linking And EmbeddingObjects are data modules;transactions are execution modules • Link: pointer to object somewhere else • Think URL in Internet • Embed: bytesare here • Objects may be active; can callback to subscribers

47. Database Spreadsheet Photos Mail Map Document Objects Meet DatabasesThe basis for universaldata servers, access, & integration • object-oriented (COM oriented) programming interface to data • Breaks DBMS into components • Anything can be a data source • Optimization/navigation “on top of” other data sources • A way to componentized a DBMS • Makes an RDBMS and O-RDBMS (assumes optimizer understands objects) DBMS engine

48. The Pattern: Three Tier Computing • Clients do presentation, gather input • Clients do some workflow (Xscript) • Clients send high-level requests to ORB (Object Request Broker) • ORB dispatches workflows and business objects -- proxies for client, orchestrate flows & queues • Server-side workflow scripts call on distributed business objects to execute task Presentation workflow Business Objects Database

49. Web Client HTML VB Java plug-ins VBscritpt JavaScrpt Middleware ORB TP Monitor Web Server... Object server Pool VB or Java Script Engine VB or Java Virt Machine HTTP+ DCOM ORB Internet DCOM (oleDB, ODBC,...) LU6.2 Legacy Gateways IBM The Three Tiers Object & Data server.

50. Why Did Everyone Go To Three-Tier? • Manageability • Business rules must be with data • Middleware operations tools • Performance (scaleability) • Server resources are precious • ORB dispatches requests to server pools • Technology & Physics • Put UI processing near user • Put shared data processing near shared data Presentation workflow Business Objects Database