1 / 56

MegaWare Firmware Plan Update

MegaWare Firmware Plan Update. March 10, 2010. Agenda . MR/3ware Convergence Background Process Key 3ware Firmware Features/Technologies MegaWare FW Boxcars for ThunderBolt /Tugaloo Boxcar 1.1 Boxcar 1.2 MR FW Technical Summit Update Topics for discussion FW Requirements

rafe
Download Presentation

MegaWare Firmware Plan Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MegaWareFirmwarePlan Update March 10, 2010

  2. Agenda • MR/3ware Convergence • Background • Process • Key 3ware Firmware Features/Technologies • MegaWare FW Boxcars for ThunderBolt/Tugaloo • Boxcar 1.1 • Boxcar 1.2 • MR FW Technical Summit Update • Topics for discussion • FW Requirements • FW Enablers for MegaWare FW • Preliminary MegaWare FW boxcars beyond Tugaloo • Tentative Plan

  3. MegaWare Vision – A single RAID Stack MegaRAID Stack Stack Vertical OEMs Large Integrators Channel Market OEM Market One Single Stack: MegaWare LSI Confidential

  4. “MegaWare” Overview

  5. 3ware Key Success Factors for Channel Market MegaWare Firmware

  6. Convergence Process • Steering Committee: • Tom Kodet (PdM) • Brian Highers (PdM) • Michael Giraudo (MR/3w FAE Mgmt) • Jim Rizzo (MR Engineer) • Adam Weiner (MR Engineer) • Anant Baderdini (MR Engineer) • David Solina (MR Engineer) • Senthil Thangaraj (MR FW Mgmt) • Basu Hallyal (MR Architect) • Naman Nair (MR Architect) • Bob Sheffield (MR Architect) • Mark Ish (MR Architect) • Alex Tang (3ware Architect) • Horia Simionescu (3ware Architect) • Chris Therene (MR Engineering Mgmt)

  7. Firmware Convergence Process • - Enablers • Preliminary Scoping • Boxcars Content identification

  8. “MegaWare” Content • Critical • Must Have • Desirable • Not Needed 3ware Features Criticality

  9. Key 3ware Firmware Technologies/Features

  10. 3ware FW “Critical”/”Must Have” Features Deployment

  11. MegaWare FirmwareBoxcars for ThunderBolt/Tugaloo

  12. 2011 2010 2009 MegaRAID MegaRAID Firmware Roadmap 2008 2008 2008 3G MegaRAID EF Maintenance (3.6) 6G MegaRAID Skidaway (4.1) Amicalola (4.2) Tallulah (4.3) 10M03 (4.4) 10M06 (4.5) 6G DC MegaRAID Tugaloo (5.1) Panola (5.2) Phase #1 Phase #2 12G DC MegaRAID ‘MegaWare’ Vogel (6.1) Phase #3 LSI Confidential LSI Proprietary

  13. MegaRAID Firmware Roadmap Q1-2010 Q2-2010 Q3-2010 Q4-2010 2006 2007 Feature Complete Feature Complete MegaWare Boxcar #2 MegaWare Boxcar #1 Tugaloo(5.0) (6Gb/s SAS) • Added Platforms: • Alcor • Procyon • Overview: • Dual Core Thunderbolt • Performance Enhancements • Added Features: • DIF* • Cache-Offload • Split DPL Architecture • “MegaWare” Phase #1 * Premium Feature LSI Confidential

  14. “MegaWare” – Boxcars Plan for Tugaloo (5.1) • Boxcar #1 Content (LSIP200052525) – CY10Q1 (Training Boxcar/Enablement) • Features: • Phy Info (Direct Attached) – Priority #3 • Phy Settings (Direct Attached) – Priority #3 • Lock Request from Host – Priority #3 • Development Tools: • Green Hills Software – Debug Dual Core • Test Tools: • Twrc/File2Lba • SafetyNet Automation Framework • Error Injection • Boxcar #2 Content (LSIP200052526) – CY10Q2 (More meaningful P1/P2 features) • Features: • Advanced Event Capture – Priority #1 • Drive swap/POR check after link bounce (no degrade) – Priority #1 • Drive Performance Monitoring - Priority #2 • Cold Boot Automatic Unit Exporting - Priority #2 • SCSI Mode Page controls – Priority #2 • Development Tools: • Coverity ? • Test Tools: • Tw_analyze • SafetyNet Automation Framework • Soft Reset, BBU

  15. “MegaWare” – FW Critical Features Deployment

  16. MR FW Technical Summit Update

  17. MR FW Technical Summit – Sessions Coverage

  18. Technical Summit Goals - Accomplishments • Knowledge Sharing: Complete • Bring-up up to speed the MR team on key technologies for a successful channel play, so MR team can understand what features are important and why. • Bring-up to speed the MR team on 3ware I/O path/caching implementation, so the team can assess better what concepts could be reused and what changes could be achieved to match the channel market requirements, and in what timeframe. • Bring-up the entire group up to speed on latest technologies and understand how they may impact the MR stack in the future (Multi-Core, SSD-SSC WB/Snapshot, 3ware) and drive new requirements. • Design Direction for “MegaWare”: Complete • Define and agree on a technical direction (strategy and preliminary design) together so we can understand the scope, fully plan the project, be aligned with the DAS strategy and prepare for this key migration. • Architecture Scalability: Complete • Understand how the underlying platform (ThunderBolt, Invader or Avenger) may affect the architecture of the solution, so we can design something that will scale in the future. • Validate Requirements for MegaWare: Complete

  19. “MegaWare” – FW Critical Features Deployment Technical Summit

  20. MegaWare Key Requirements • Seamless Caching Configuration • Hides controller/array state • Configuration Roll-up • Meet performance demand of vertical streaming applications (D2D Backup, Video Surveillance, V.O.D, etc…) • Improved multi read/write streaming support • Streaming QOS (Balancing) • Minimum guaranteed throughput • Reduced maximum I/O latency • Scalable performance (scale with more memory, etc…) • Drive Write Cache protection • Need to identify performance metrics (i.e. X% performance drop of optimal mode) • No/minimum impact to FPI/Os and LD I/Os

  21. MegaWare Enablers MegaWare Key Requirements MegaWare Features Enablers New Modified Existing

  22. MegaWare: Enablers • Multi-Read Streams Performance Enhancements • Buffer Allocation Improvements • Read Ahead Improvements • Read/Write stream balancing • Buffer provisioning • DM Improvements • Write Streaming Improvements • Guaranteed minimum bandwidth and maximum latency • Read-ahead improvements and read/write balancing help toward this end • Background/Foreground I/O pacing improvements • Device write cache protection • FUA Support • WXOR Support for RMW operations

  23. MegaWare FirmwarePreliminary boxcars beyond Tugaloo

  24. MegaWare FW Panola/Vogel Tentative Boxcars • Panola Release: MegaWare Boxcar 2.0 • Multi-Read Streams Performance Enhancements • Write Streaming Improvements • Read/Write stream balancing • Vogel Release: MegaWare Boxcar 3.0 • Guaranteed minimum bandwidth and maximum latency • Device write cache protection • FUA Support • WXOR Support for RMW operations • Features not mapped yet: • StorSave Profile

  25. MegaWare FirmwareDevelopment Update

  26. MegaWare Firmware – Development Update • MegaWare 1.1: • Completed wet thumb scoping for PRs in this boxcar • Working with MR architects on defining new FW APIs • GHS Support POC completed • Completed repository preparation, adding GHS support. • Development will be done on Liberator boards • MegaWare 1.2: • Working with MR architects on defining new FW APIs and mapping functionality. • MR FW Training: • Group #1: Only 1 engineer full time on MegaWare, 2 part time • Hands-On training • Difficult to assemble Group #2, due to resource needs for existing 3ware programs/DEO • Learning DAS development process, CQ/CC tools LSI Confidential

  27. 2011 2010 2009 MegaRAID MegaRAID Firmware Roadmap 2008 2008 2008 3G MegaRAID EF Maintenance (3.6) 6G MegaRAID Skidaway (4.1) Amicalola (4.2) Tallulah (4.3) 10M03 (4.4) 10M06 (4.5) 6G DC MegaRAID MW 1.1 MW 1.2 Tugaloo (5.1) MW 2.0 Panola (5.2) Phase #1 Phase #2 12G DC MegaRAID ‘MegaWare’ MW 3.0 Vogel (6.1) Phase #3 LSI Confidential LSI Proprietary

  28. 3ware FW “Critical”/”Must Have” Features Deployment

  29. MegaWare FirmwareNext Steps

  30. MegaWare FW – Next Steps • Complete planning for 1.1 and 1.2 boxcars • Complete mapping out FW changes • Start wet thumb scoping • Discuss Management SW for these boxcars • Complete preliminary FW design for highly complex P1 features. • Complete mapping out FW changes • Start wet thumb scoping • Complete Requirements/FW design for remaining P1 features (ie. StorSave profile, etc…). Need strong help from PdM here… • Continue review of P2/P3 features with PdM • Rapid RAID Recovery • Background Task Activity Policy (BTAP) • Etc… LSI Confidential

  31. Backup Slides

  32. Critical Features for Vertical OEMs and Large Integrators Critical Features • Full Backwards compatibility – All Management Tools support the 7xxx, 8xxx, and 9xxx product families. Firmware support is carried over to the next Hardware release. • Customers: Silicon Mechanics, Amazon, Bluehost, Dolby Labs, Overland Data, Cisco, Limelight Networks, LogLogic and many integrators. • Linux support for all 2.6 kernels. Driver source for 2.6.8 to 2.6.32 kernels. Driver diskettes for distribution Operating Systems. • Customers: Pelco, Bluehost, Unitrends, Solera Networks, McAfee, AMAX, SilverPeak and many integrators. • StorSave – The ability to save I/O data in controller cache and disk cache. Replaying of I/Os for all unit cases (Ok, degraded, rebuild, etc), even without BBU. • Customers: Pelco, Quantum, and Silicon Mechanics. LSI Proprietary

  33. Critical Features for Vertical OEMs and Large Integrators Critical Features • Multi-Stream Write/Read Performance: Provide Quality of Service to both read and write streams while maintaining high overall performance. • Customers: Pelco, Quantum, LodgeNet, Solera Networks, Adtech, MaxVision, Overland Data, etc… • Long Command Time (LCT) handling. JBOD configuration for Video Editing. An issue with one disk doesn’t effect the other disks such that all disks respond within 250msec for each I/O request. • Customer: Avid, Amazon • Performance Monitoring. Ability to gather statistics on slow drives. Capture I/O characteristics of applications. • Customers: Pelco, LogLogic, Avid, and BSCii, etc… LSI Proprietary

  34. Must Have Features for Vertical OEMs and Integrators Must Have Features • Easy to use CLI. Intuitive interface which enables better scripting. • Customers: Bluehost, Equus, Unitrends, PSSC Laboratory, LogLogic, GlobalStore, and SilverPeak. • Background Task Activity Policy . Allow optimal application performance while background tasks are running. (9.5.2) • Customers: SonicWall, Dolby Labs, and WSI. • Rapid RAID Recovery. Fast rebuild of degraded drives. • Customers: Dedicated Computing and many System Integrators. LSI Proprietary LSI Proprietary

  35. Must Have Features for Vertical OEMs and Integrators Must Have Features • Console Redirection in 3BM. Allows to run as headless system. Allows customers to enter 3BM using a different key sequence. Includes authentication challenge prior to entering 3BM. • Customer: Siemens Nixdorf, etc… • Chassis ManagementSupport. Many chassis are AMI-based (SES over I2C). • Customers: Dedicated Computing, NEI, Adtech, and AMS. • ATA/SCSI Pass-Through support. The capability to download drive FW through the controller as well as gather information directly from the drives. An API is provided to both customers and drive partners. • Customers: Avid, Rackable, Blackwave, McAfee, LimeLight Networks, and Netscout. LSI Proprietary LSI Proprietary

  36. Performance Considerations

  37. Vertical Markets: Customer Specific Applications Video Editing NAS Security and surveillance Bulk Storage 3ware ACS Disk to Disk Backup Workstations Videoon-demand Gaming Supercomputing Streaming/Low Latency Environments

  38. 3ware RAID Firmware Stack – Performance Features History Scalable Streaming Design 9.5.2 9.1.5 9.2.0 9.3.0 9.3.0.4 9.3.1 9.4.0 9.4.2 9.5.0 9.5.0.1 9.5.1 InitialStreamingSupport Streaming Extension Streaming Extension SAS SAS/SATA Perf Impr. Supermicro RAID6 Stripe Engine MPIO/CIFS Perf. Improvmnt 9690SA Adv. Content Streaming Accel. Time to Optimal Mode Fast Array Initialization Intelligent Read-Ahead Back. Task Activity Policy 9650SE User-LUN Carving Enclosure Services (CCU) Drive Insertion/Removal StorSave Profile Auto-Rebuild Native SATA / NCQ R/W Load Balancing Mac OS/SideCar BBU / StorSave Stream Fusion Multi-LUN Array Naming OCE/RLM 9590SE 9550SX 9500S Feb ‘08 May ‘09 Sep ‘05 Feb ‘05 Apr ‘06 Aug ‘08 Aug ‘04 Sep ‘06 Oct ‘06 Oct ‘07 Sept ‘07 Time

  39. Performance Measurements Tier #1 - Synthetic Benchmarks Raw Device Iometer (Seq/Rnd, Single/Multiple Stream), Atto DiskBench, dd, etc… Tier #2 - Local Applications File Systems (ext3, xfs, NTFS…) Bonnie++, Iozone, etc.. Tier #3 - Network Applications Network Protocols and File Systems (SMB, CIFS, NFS) Netbench, MPIO, Custom Apps Tier #4 – Customer-Specific Applications Disk-To-Disk Backup, Video On Demand, Video Surveillance, Digital Cinema, iSCSI SAN, etc…

  40. Attempts to maximize sustained write throughput in very complex environments while minimizing maximum latency Initially targeted as the new high performance I/O path engine for next generation product (PCI 5Gbs/SAS 6Gbs), it has been retrofitted into existing products (9650SE/9690SA) Advanced Content Streaming (ACS) with IRP - Overview Intelligent Read Prefetch (IRP) • 9.5.1: ACS - Multi-Writes, Multi Reads/Writes • Write intensive environments (D2D • Backup, Video Surveillance, etc…) • 9.5.2: IRP - Multi-Reads, Multi Reads/Writes • Extends lead in read intensive • environments (Video Servers, Digital • Cinema, Video On Demand, etc…) • Maintain performance in worst case conditions

  41. IRP adds better management of read command execution to improve performance. Leverages and fits into the ACS design. Complex ADAPTIVE management of read command handling Read Ahead Caching Stream Management Detection (strength, direction, etc…) Monitoring Read data pre-fetching (Buffered vs Un-buffered streams) Recycling Balancing (Reads vs Reads, Reads vs Writes) Stream Policies High queue depth threshold Read data allowance Prefetch parameters (trigger point, amount) Combination of static and dynamic parameters based on number of streams and types (Read Only, Low Write Streams, High Write Streams), queue depth, RAID mode, RAID type, etc… Intelligent Read Prefetch (IRP) - Overview

  42. Write Caching Layer for all RAID Levels Configurable Write Cache Policy on Array Degrade - Better performance in degraded mode Advanced Content Streaming (ACS)/Intelligent Read Prefetch (IRP) RAID5 Parity Caching (w/ Write Journaling) RAID5 Peer Reads Write Method for 4 drives and less – Better random performance Read/Write Load Balancing Drive LBA Reordering for improved IOPS at High Queue Depth Starvation Protection Mechanism Low-Latency Commands Priority (Host Reads, FUA Writes) Complement NCQ drives – Better visibility of commands AV Mode Optimized I/O latency between foreground/background I/Os Drive Performance Monitoring Others Performance Features - Overview

  43. Storsave Data Protection Modes • 3ware products provide to end-user a simple method for selecting the desired degree of performance and protection: storsave • Storsave has three modes that define behavior with or without BBU • Perform mode provides the highest level of performance • FUA commands are not honored • Write cache is kept ON even when RAID goes degraded • Balance mode provides a higher level of data protection • FUA commands are honored • Write cache is turned OFF when no BBU and RAID goes degraded • Protect mode provides the highest level of data protection • When no BBU FUA are honored • Write cache is turned OFF when RAID goes degraded • 3ware drive data protection is implemented LSI Confidential

  44. Performance Improvements with ACS

  45. Streaming Performance Single/Multi Streams Not strict sequential (out of order, mixed with random, raw/FS) Key Performance Metrics Maximum Raw Throughput Scaling “Minimum Guaranteed” Performance Stream Balancing Key Performance Goals

  46. MPIO Performance Improvements • Performance improved by 10x for small I/Os !! • RAID5/9.5.1 is slightly faster than RAID0/9.4.2

  47. Netbench Performance Improvements • 9.5.1 is more than 2 times faster than 9.4.2 for 12/16 clients

  48. Market: Video Servers, V.O.D. , Digital Cinema • Minimum guaranteed read performance • An array can be in any mode at any point in time • Normal, Degraded, Double-Degraded (R6) • Application requirements: • Number of streams to be supported • Bandwidth requirements per stream • Care about performance in Worst Mode but not performance in best mode • Drive the need for a storage device to deliver a “Minimum Guaranteed throughput” (performance in worst mode, which may not be necessarily degraded mode) • => The minimum read throughput dictates how many streams can be supported for a video application

  49. Multi-Write Streams – All Modes – QD=16 Above 1 GB/s for 64 streams Up to 1.7X faster

  50. Multi-Read Streams – All Modes – QD=16 Above 500 MB/s for 64 streams in DD Up to 2.5X faster

More Related