Storage Systems CSE 598d, Spring 2007

1. Storage SystemsCSE 598d, Spring 2007 Lecture 3: Disk drive trends, modeling (contd.) Feb 1, 2007

2. Topics • Disk drive modeling • SCSI vs ATA • Rules of thumb in data engineering

3. Disk Drive Modeling • Problems because • Non-linear • State Dependent • Not easy to model analytically • Pitfalls • Seek time linear w.r.t distance • Uniform distribution for rotational latency • Constant transfer times • Ignoring bus contention

4. Comparing four different models • (i) Constant fixed time for each I/O • (ii) Simple model which is • Seek time is linear with distance • No head settle/switch costs • Uniform rotational delay • Fixed controller costs • Linear transfer costs • (iii) Better seek and positioning model • (a) 3.45+0.597*sqrt(d) ms (for < 616 cylinders) • (b) 10.8+0.012d ms (for >= 616 cylinders) • 2.5 ms for head/track switch • Keeps track of rotational position • (iv) All of (iii) + Cache model + Read ahead + Bus speed + Controller overheads • Chosen metric for comparison: relative demerit

5. Results (i) (ii) (iv) (iii) This disk does not have a cache!

6. Disk with a cache (iii) (iv)

7. Conclusions • The following aspects are important • Disk cache/buffer (112) • Data transfer model (20) • Overlaps with bus transfer, seek-time, head-switching • Rotational position, data layout (2) • While these are not (that important)

8. How do we get the drive parameters for such modeling? • Manuals/Data sheets • Not everything is publicized • Things can still vary • Interrogative extraction • Though extensive SCSI interface, not all may be supported • Several more parameters may be needed • Empirical/experimental extraction • This is hard

9. Complications of empirical extraction • Overlapping controller overheads, bus transfers, mechanical delays, etc. • Contention for shared resources • Cache segmentation • Prefetching • Non-uniformity in performance (e.g. seeks) • Large seemingly non-deterministic delays (e.g. thermal recalibration) • Fluctuations in timing.

10. Parameters needed for modeling • Data layout • Seek, rotational latency and transfer costs • Bus, controller and host processing costs • Caching and prefetching parameters

11. Data Layout Parameters • Where does a block actually reside on disk? • May need to be re-acquired upon each formatting (since a re-allocated defect may be converted to slipped defects for better efficiency) • SEND/RECEIVE diagnostic of SCSI interface can be used to query the actual location of a block. • Doing this for each block would be very time-consuming

12. Storage SystemsCSE 598d, Spring 2007 Lecture 4: Disk drive trends, modeling (contd.) Feb 8, 2007

13. Empirical Extraction • Send commands to disk and measure Mean Time Between Request Completions – MTBRC(a,b) – of 2 requests iteratively. • Rotational distance between request pairs is varied until a minimum is reached.

14. Extracting Head switch time • MTBRC1 = MTBRC(1-sector write, 1-sector read on the same track) = Host1+Cmd+Media+Bus+Comp • MTBRC2 = MTBRC(1-sector write, 1-sector read on a diff. track of same cylinder) = Host2+Cmd+HdSw+Media+Bus+Comp HdSw = (MTBRC2–Host2) – (MTBRC1–Host1)

15. Extracting Seek Times • For each seek distance, select 5 points evenly spaced. From each of these points, perform 10 inward and 10 outward seeks of this distance. Get the average of these. (ii) Measure MTBRC(1-sector write, 1-sector read on same track), and MTBRC(1-sector write, 1-sector read on next cylinder). Difference between these is mechanical time for 1-cylinder seek. (iii) Subtract (ii) from the 1-cylinder distance value of (i). The diff. represents the non-mechanical overheads of seek. Subtract (iii) from each of the values obtained in (i)

16. Typical Seek time profile

17. Extracting Rotation Speed • Perform a series of 1-sector writes to the same location and calculate the mean time between completions.

18. Extracting Cache Segments, Size … • Say the # of segments is N. • Perform 1-sector reads of the first logical blocks of the first N-1 cylinders • Perform a 1-sector read of the first logical block of the last data cylinder • Perform a 1-sector read of the first logical block of the first cylinder. If that is a hit (measured by response time), then # of segments is N or greater.

19. Extraction techniques for • Segment size • Do prefetched data replace requested data in the current segment? • Are all requested data always thrown away? • Does prefetching stop on track/cylinder boundaries? • Is the prefetching size proportional to request size? • Does it implement read-on-arrival? Write-on-arrival? • Is cache space allocated on a track or sector basis? • Can READs hit on data placed in the cache by WRITEs? • What is the segment replacement algorithm?

20. The physical I/O path from CPU to the disk

21. CPUs RAMs System bus Bridge chip Host I/O bus (PCI, Infiniband) SCSI HBA FC HBA iSCSI HBA Graphics Card Ethernet NIC Fibre Channel SCSI IP LAN I/O buses

22. System bus • Rapid data transfer between CPU and memory • Host I/O bus • Common: PCI, emerging: Infiniband • Device drivers responsible for control of and communication with peripheral devices of all types • Part of the device driver for storage device almost always realized by firmware that is processed by special processors (ASICs) • ASICs are partially integrated into the main curcuit board, such as on-board SCSI controllers, or connected to the main board via add-on cards (PCI cards) • Storage devices connected to the server via the host bus adapter (HBA) • Communication connection between the HBA and the peripheral device is called the I/O bus • Similar I/O path/techniques used within a disk subsystem

23. I/O bus technologies • SCSI • ATA/IDE, Serial-ATA (SATA) • SCSI over IP (iSCSI) • Fibre Channel • USB • … many more

24. SCSI basics • Small Computer System Interface • First version released in 1986 • Many versions since • The dominant technology for UNIX and PC servers • Assignment: find out what your laptop/desktop uses • A communication protocol as well as bus • Parallel bus for data and additional lines for control of communication

25. SCSI basics (more) • A daisy-chain can connect upto 16 devices together • SCSI protocol defines • How devices reserve the bus • In what format data is transferred • Initial versions: message then ACK then next message • Latest versions: asynchronous issuance, multiple messages in transit together, increased data rate

26. SCSI vs ATA:Motivating Factors • Cost (Market Demands) • Form factor • Configuration in groups • Reliability • Access Patterns

27. Leading to differences in … • Mechanics • Materials • Electronics • Firmware • Performance (RPM and Seeks) • Reliability • Power Consumption • …

28. Differences in Mechanics • ES Head/Disc Assembly • Sustain higher disturbance • Higher rigidity • More mass • Higher bandwidth servos • Avoiding through holes • Filter for particles, desiccant for humidity, carbon absorbent for organic materials • Better air flow hardware • O-ring seals for spindle • Higher quality sealing

29. Mechanics (contd.) • Actuator • Larger magnets for faster seeks • Lower resistance (thicker and fewer windings) actuator coils • Latch (to hold actuator when off) can affect seek performance. ES compensates for this with a bi-stable latch. • Spindle • Higher RPM => Windage and Vibrations • PS Drives use a cantilever design to hold a motor (captured only at base), while ES drives capture the motor at both ends.

30. Differences in Electronics • Needs to take and process commands from host, perform head positioning, servo processing, data transfers, cache management, etc. • PS drives may not have separate servo processor (to handle repeatable on non-repeatable runouts). • ES ASIC gate count 2X PS gate count • ES firmware code 2X PS firmware code size (to handle more concurrency) • ES Cache space 10X PS Cache space

31. Differences in Magnetics • More or less similar (since there is no reason why latest advancements may not be used in both). • Main differences are in electronics needed to provide a Signal-to-Noise (SNR) ratio for the higher RPM of ES drives.

32. Differences in Performance • Capacity • Areal density is similar since they use same magnetics • Differences due to # of platters and their size • Size of Platters • Power is nearly cubic to platter size. • To sustain higher RPM, ES drives use smaller platters (2.5” and lower) -> also helps seeking • # of platters • Trend is towards de-populated drives since you can use more drives to meet the capacity demands in ES environments

33. Performance (contd.) • Data Rates • Though higher RPM favors ES, PS benefits from larger platter size, and more frequent introductions of newer models.

34. Performance (contd.) • Seeks • Mechanical improvements and smaller platters favors ES. • ES also allows larger queue depths of outstanding requests to benefit from smarter scheduling.

35. Rotational Vibration • Environment/nearby drives can excite the drives to throw the actuator off-track. • Note that this causes performance loss. • Need to understand how much vibration (in radians/square-sec) is present and design for it. • Some recent drives even have a vibration sensor for compensation in servo processing.

36. Reliability • Described based on power-on hours (8 hrs/day for PS and 24 hrs for ES). • Depends on • Duty cycle (40% for ES vs. 75% for PS due to shorter seeks) • Temperature • Particles inside • Head crashes

37. Serial ATA (SATA) • Serial implementation of ATA • Higher data rates • 133-150 Mbps compared to 320 Mbps for SCSI • Easier to configure, cheaper, less reliable (?)