1 / 19

Case Study - Storage Consolidation

Explore Yahoo's storage consolidation journey, from legacy architecture challenges to new solutions that boost reliability and scalability while reducing management overhead. Learn about the benefits, drawbacks, and outcomes of adopting NAS and SAN for improved data management.

fuhr
Download Presentation

Case Study - Storage Consolidation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Case Study - Storage Consolidation Steve Curry Yahoo Inc.

  2. About Yahoo! Quick Stats 300+ million registered users 2 billion page request per day 25 countries, 14 languages 500TB data on disk 1PB data on tape

  3. Yahoo! Storage Operations Responsibilities All US storage administration Data archiving / backups US/Global storage architecture / standards 2nd tier support for global operations Tool development 24/7 global issue/outage response Reporting

  4. Case Study #1 – Y! Photos/Briefcase • Online photo album Case Study #1 • Online file storage

  5. Case Study #1 – Y! Photos/Briefcase Legacy Architecture Cheap… *repeat* cheap JBOD’s Single host support JBOD array A/B mirror for redundancy FreeBSD OS 150TB of content Custom apps

  6. Case Study #1 – Y! Photos/Briefcase …Legacy Architecture Advantages Low cost hardware Extremely distributed Disadvantages Not very scalable Management headache No longer meets reliability requirements

  7. Case Study #1 – Y! Photos/Briefcase …Legacy Architecture Management Issues Management is per host (over 160 storage hosts) Synchronous mirror between A/B pair No “Hot-Swap” support Single spindle performance

  8. Case Study #1 – Y! Photos/Briefcase This… X 12!  Single tier, single spindle performance.

  9. Case Study #1 – Y! Photos/Briefcase Consolidation Plan NAS or SAN? Requirements Reliability Scalability Reduce management overhead Considerations Current hardware investment Application support

  10. Case Study #1 – Y! Photos/Briefcase Network Attached Storage Solution Management Filers are heavily deployed Smart appliance Suite of tools already developed for filers Advantages RAID redundancy Multi-spindle performance Takes advantage of existing hardware Ease of application port

  11. Case Study #1 – Y! Photos/Briefcase …Network Attached Storage Solution Disadvantages Initial cost of deployment (cutover, SCSI –vs- IDE) Lot’s of JBOD’s to get rid of! ;-)

  12. Case Study #1 – Y! Photos/Briefcase New Architecture NAS solution FreeBSD app servers Load balanced 10 storage hosts Point in time snapshots Dedicated SAN backup fabric Distributed-farm model

  13. Case Study #1 – Y! Photos/Briefcase Simple 2 tier model. Scalable, redundant, multi-spindle RAID performance, hot-swap support.

  14. Case Study #1 – Y! Photos/Briefcase Consolidation Wins! Cost considerations Performance Backups Management High availability Hot swap

  15. Case Study #2 - Data Mining Case Study #2 Global data mining Global log collection

  16. Case Study #2 - Data Mining Current Architecture DAS attached arrays Custom scripts Stacker type tape libraries Single-tier disk storage

  17. Case Study #2 - Data Mining Management Issues Large storage host count Many small tape libraries No redundancy Does scale for future requirements

  18. Case Study #2 - Data Mining Storage Requirements High write performance Data growth 2TB per day!! Store data on disk for 30 days Archive to tape Consolidation Considerations Reduce host management Create a multi-tier storage architecture Consolidate to one large tape library Increase write performance

  19. Case Study #2 - Data Mining • Common Y! model • Multi-tier storage • Scalable

More Related