230 likes | 257 Views
Explore the future of disk controllers transforming data storage with advanced technology drivers and innovative system architectures. Discover the shift towards centralizing processing units and memory within disk controllers for enhanced efficiency.
E N D
Put Everything in Future (Disk) Controllers(it’s not “if”, it’s “when?”)Jim Grayhttp://www.research.Microsoft.com/~GrayAcknowledgements:Dave Patterson explained this to me a year agoKim KeetonErik RiedelCatharine Van Ingen Helped me sharpen these arguments
Kilo Mega Giga Tera Peta Exa Zetta Yotta Technology Drivers: Disks • Disks on track • 100x in 10 years 2 TB 3.5” drive • Shrink to 1” is 200GB • Disk replaces tape? • Disk is super computer!
Data GravityProcessing Moves to Transducers • Move Processing to data sources • Move to where the power (and sheet metal) is • Processor in • Modem • Display • Microphones (speech recognition) & cameras (vision) • Storage: Data storage and analysis
It’s Already True of PrintersPeripheral = CyberBrick • You buy a printer • You get a • several network interfaces • A Postscript engine • cpu, • memory, • software, • a spooler (soon) • and… a print engine.
All Device Controllers will be Cray 1’s Central Processor & Memory • TODAY • Disk controller is 10 mips risc engine with 2MB DRAM • NIC is similar power • SOON • Will become 100 mips systems with 100 MB DRAM. • They are nodes in a federation(can run Oracle on NT in disk controller). • Advantages • Uniform programming model • Great tools • Security • economics (CyberBricks) • Move computation to data (minimize traffic) Tera Byte Backplane
Basic Argument for x-Disks • Future disk controller is a super-computer. • 1 bips processor • 128 MB dram • 100 GB disk plus one arm • Connects to SAN via high-level protocols • RPC, HTTP, DCOM, Kerberos, Directory Services,…. • Commands are RPCs • management, security,…. • Services file/web/db/… requests • Managed by general-purpose OS with good dev environment • Move apps to disk to save data movement • need programming environment in controller
The Slippery Slope Nothing = Sector Server • If you add function to server • Then you add more function to server • Function gravitates to data. Something = Fixed App Server Everything = App Server
Why Not a Sector Server?(let’s get physical!) • Good idea, that’s what we have today. • But • cache added for performance • Sector remap added for fault tolerance • error reporting and diagnostics added • SCSI commends (reserve,.. are growing) • Sharing problematic (space mgmt, security,…) • Slipping down the slope to a 2-D block server
Why Not a 1-D Block Server?Put A LITTLE on the Disk Server • Tried and true design • HSC - VAX cluster • EMC • IBM Sysplex (3980?) • But look inside • Has a cache • Has space management • Has error reporting & management • Has RAID 0, 1, 2, 3, 4, 5, 10, 50,… • Has locking • Has remote replication • Has an OS • Security is problematic • Low-level interface moves too many bytes
Why Not a 2-D Block Server?Put A LITTLE on the Disk Server • Tried and true design • Cedar -> NFS • file server, cache, space,.. • Open file is many fewer msgs • Grows to have • Directories + Naming • Authentication + access control • RAID 0, 1, 2, 3, 4, 5, 10, 50,… • Locking • Backup/restore/admin • Cooperative caching with client • File Servers are a BIG hit: NetWare™ • SNAP! is my favorite today
Why Not a File Server?Put a Little on the Disk Server • Tried and true design • Auspex, NetApp, ... • Netware • Yes, but look at NetWare • File interface gives you app invocation interface • Became an app server • Mail, DB, Web,…. • Netware had a primitive OS • Hard to program, so optimized wrong thing
Why Not Everything?Allow Everything on Disk Server(thin client’s) • Tried and true design • Mainframes, Minis, ... • Web servers,… • Encapsulates data • Minimizes data moves • Scaleable • It is where everyone ends up. • All the arguments against are short-term.
The Slippery Slope Nothing = Sector Server • If you add function to server • Then you add more function to server • Function gravitates to data. Something = Fixed App Server Everything = App Server
Disk = Node • has magnetic storage (100 GB?) • has processor & DRAM • has SAN attachment • has execution environment Applications Services DBMS RPC, ... File System SAN driver Disk driver OS Kernel
Technology Drivers: System on a Chip • Integrate Processing with memory on one chip • chip is 75% memory now • 1MB cache >> 1960 supercomputers • 256 Mb memory chip is 32 MB! • IRAM, CRAM, PIM,… projects abound • Integrate Networking with processing on one chip • system bus is a kind of network • ATM, FiberChannel, Ethernet,.. Logic on chip. • Direct IO (no intermediate bus) • Functionally specialized cards shrink to a chip.
TCP/IP Unix/NT 100% cpu @ 40MBps Disk Unix/NT 8% cpu @ 40MBps Why the Difference? Host does TCP/IP packetizing, checksum,… flow control small buffers Host Bus Adapter does SCSI packetizing, checksum,… flow control DMA Technology Drivers: What if Networking Was as Cheap As Disk IO?
Technology Drivers: The Promise of SAN/VIA:10x in 2 yearshttp://www.viarch.org/ • Today: • wires are 10 MBps (100 Mbps Ethernet) • ~20 MBps tcp/ip saturates 2 cpus • round-trip latency is ~300 us • In the lab • Wires are 10x faster Myrinet, Gbps Ethernet, ServerNet,… • Fast user-level communication • tcp/ip ~ 100 MBps 10% of each processor • round-trip latency is 15 us
RIP FDDI RIP ATM RIP FC RIP SCI RIP ? RIP SCSI SAN: Standard Interconnect Gbps Ethernet: 110 MBps • LAN faster than memory bus? • 1 GBps links in lab. • 100$ port cost soon • Port is computer PCI: 70 MBps UW Scsi: 40 MBps FW scsi: 20 MBps scsi: 5 MBps
Technology Drivers:100 GBps Ethernet replaces SCSI • Why I love SCSI • Its fast (40MBps) • The protocol uses little processor power • Why I hate SCSI • Wires must be short • Cables are pricy • pins bend
Functionally Specialized Cards P mips processor Today: P=50 mips M= 2 MB ASIC • Storage • Network • Display M MB DRAM In a few years P= 200 mips M= 64 MB ASIC ASIC
Technology DriversPlug & Play Software • RPC is standardizing: (DCOM, IIOP, HTTP) • Gives huge TOOL LEVERAGE • Solves the hard problems for you: • naming, • security, • directory service, • operations,... • Commoditized programming environments • FreeBSD, Linix, Solaris,…+ tools • NetWare + tools • WinCE, WinNT,…+ tools • JavaOS + tools • Apps gravitate to data. • General purpose OS on dedicated ctlr can run apps.
Basic Argument for x-Disks • Future disk controller is a super-computer. • 1 bips processor • 128 MB dram • 100 GB disk plus one arm • Connects to SAN via high-level protocols • RPC, HTTP, DCOM, Kerberos, Directory Services,…. • Commands are RPCs • management, security,…. • Services file/web/db/… requests • Managed by general-purpose OS with good dev environment • Move apps to disk to save data movement • need programming environment in controller