1 / 14

Tom Creighton creightonnt@familysearch

Considerations Regarding Archival Cloud Storage. Tom Creighton creightonnt@familysearch.org. So You Think You Want To Do It In The Cloud?. Service Availability Data Durability & Integrity Service and Data Security Access Methods COST. Service Availability. Amazon aws

galen
Download Presentation

Tom Creighton creightonnt@familysearch

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Considerations Regarding Archival Cloud Storage Tom Creighton creightonnt@familysearch.org

  2. So You Think You Want To Do It In The Cloud? • Service Availability • Data Durability & Integrity • Service and Data Security • Access Methods • COST

  3. Service Availability • Amazon aws • 99.9% availability guarantee (S3) • Regional • Inter-regional availability probably higher • Few inter-regional services • Rackspace • Cloud Files 100% • Credits based on sliding scale of errors returned • Gartner did not like HP or aws SLAs but really was about compute

  4. Data Durability • AWS S3 & Glacier offer 99.999999999% durability • Store 10,000 objects you’ll lose one every 10,000,000 years. • (104 objects * 10-11 err/obj/yr = 10-7 err/yr) • Store 1 billion objects, you’ll lose one object every 100 years • Synchronous read after write • Eventual consistency of copy • Periodic fixity checks

  5. Data DurabilityAT&T (EMC Atmos) • Erasure coding + replica • 9/12 Configuration • 9 data fragments • 3 coding fragments • 99.999% durability • Tolerate 3 drive loss • 10/16 • Tolerate 6 drive failures • 99.99999% durability ?? • 60% overhead • Lower write performance

  6. Service & Data Security • SSL Privacy on access • SHA-256 tree hash on archive upload (integrity) • Encryption transparent • Amazon manages encryption key (AES-256) • Key per object • Future might allow customer managed key

  7. AWS Security Certifications • See aws.amazon.com/security • Compliance • SOC 1 (SAS70), SOC 2, SOC 3 • PCI DSS Level 1 • ISO 27001 • FedRAMP • ITAR • FIPS 140-2 • Others

  8. Access Methods • REST • Via S3 • Object names then apply • Upload is via S3 • Lifecycle policies for migration • Retrieval via Glacier request, then S3 • Direct • User does not provide name – Glacier provides ID • Synchronous write; asynchronous request • SHA-256 has of all 1MB blocks of an archive on upload

  9. Simple Comparison • 3 Scenarios over 10 years • Start: 0 TB, Increase 1 TB/Month • Start: 10 TB, Increase 5 TB/Month • Start: 100 TB, Increase 10 TB/Month • Compare Online Cloud Storage • AWS: S3 & Glacier • AT&T: Dynamic & Negotiated • Rackspace (Open Cloud) • No access charges included • No price reductions reflected

  10. Start 0 TB, Increase 1TB/Month

  11. Start 10TB, Increase 5TB/Month

  12. Start 100TB, Increase 10TB/Month

  13. Questions?

More Related