1 / 23

Storage Solutions The use case at the National Library of the Netherlands (KB)

Storage Solutions The use case at the National Library of the Netherlands (KB) Jeffrey van der Hoeven APARSEN webinar, April 14 th , 2014. Outline of talk. About the National Library of the Netherlands (KB) Storage challenges: creating digital collections Storage solution Cost

archer
Download Presentation

Storage Solutions The use case at the National Library of the Netherlands (KB)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Storage Solutions The use case at the National Library of the Netherlands (KB) Jeffrey van der Hoeven APARSEN webinar, April 14th, 2014

  2. Outline of talk • About the National Library of the Netherlands (KB) • Storage challenges: creating digital collections • Storage solution • Cost • Future perspective • Cloud storage: hot or not…

  3. Since 1798 / 248 FTE / 53M euro budget • We preserve & give access to everything published in and about the Netherlands • Central role in Dutch information infrastructure • Kept safe: 6M physical publications / 18M digital publications • Goal: everything digital in 2035

  4. We give open access to: What we do 8million 4,6million 2,1million Newspaper pages online Online visits Parlementary pages online

  5. Storage challenges: Creating digital collections

  6. Storage share of digital collections (in GB)

  7. Storage prospect at KB 1800m 1 PB & 1000M files Burj Khalifa Dubai 0,5 PB 1.5 million CD-ROM’s 828m & 500M files Empire State Building 443m 324m Eiffel tour 2010 2011 2012 2018

  8. Challenges in (long-term) storage • Volume (size and number of files) • Type of data (structured / unstructured) • Growth rate • Availability vs preservation • Cost per TB

  9. Storage solution

  10. IT & Storage at KB Two locations: • In-house = data centreforprimary storage and computing • Off-site = for data back-up & archiving • Hosting 230 servers (80 physical / 150 virtual) • Managing 550 TB of data • Managing +/- 500 million files: • PDF, TIFF, JPEG2000, JPEG, XML

  11. Storage Management Storage tiers Veryfast, veryexpensive Usedfor : indexing, databases HW : SAN withHiPerf SAS disks, near-future: SSD Gold Fast, expensive Usedfor : web hosting, processing HW : SAN withHiCap SAS disks Silver Slow (45 sec), sustainable Used : long-term archiving HW : Disk-based NAS with WORM Steel Very slow (> 45 sec) Usedfor : back-up & restore, archiving HW : LTO4/5 tape Bronze

  12. Storage process & strategy Selection Digital processing Access Stage 1 Stage 2 Stage 3 Stage 4 Stage 5 Shared file system(s) / API DB File system Storage management Storage on-site Off-site Bronze Bronze Steel Silver Gold Platinum Back-up

  13. Storage cost Source: http://www.brightsideofnews.com/2011/12/07/your-storage-blog-make-storage-cheaper-and-more-energy-efficient/

  14. TCO storage • Cost per Terabyte (TB) per year per storage tier • TCO composed of several cost components, based on whitepaper Four Principles for Reducing Total Cost of Ownership(2011 Hitachi) • In total 14 cost components included • In 2014 model was approved by PWC accounting office Referenced article: http://www.hds.com/assets/pdf/four-principles-for-reducing-total-cost-of-ownership.pdf

  15. Hardware & software Support Maintenance Power & cooling Floor space Monitoring Waste & duplication Off-site locations Network

  16. KB TCO storage 2014 per TB per year € 4,858.- € 1,036.- € 1,046.- € 387.- Bronze Steel Silver Gold

  17. KB TCO storage cost over years

  18. KB vs storage providers (cloud) KB

  19. Can we afford it in the future? • Recent developments *: • Disk storage is becoming more popular in archiving. • Physicallimits of hard disk drive seemsreached. • Kryder’slawseemstofail, as disk storage densityseemsnotto keep up the pace of a yearly 30-40% increase of storage density. • Monopoly of hard disk producers Seagateand Western Digital is risky as pricesmight go up, especially in case of shortage. Risk: storage costscanbecome a bottleneck for long-term preservation. * David Rosenthal blog post, available at: http://blog.dshr.org/2012/12/talk-at-fall-2012-cni.html

  20. Cloud storage: hot… or not? Storage in the cloud

  21. Benefits of cloud storage • Scalable • Availability • Pay per TB per month • No need for own ICT infrastructure • Less maintenance

  22. However… in preservation terms: • Is it sustainable? • Who is responsible for the data? • Which jurisdiction is applied? • What if I want to migrate to another cloud? • Continuity: no money? No data! • Advise: be cautious to use the cloud for long-term storage. Read on: http://www.ncdd.nl/blog/?p=2347

  23. Thank you! Questions? Jeffrey DOT vanderhoeven AT kb DOT nl

More Related