1 / 19

DPM - efficient storage in diverse environments

DPM - efficient storage in diverse environments. M. Hellmich, O. Keeble , F. Furano , A. Manzi , A. Ayllon , I. Calvet , R. Rocha, M. Regala CHEP 2013 Amsterdam. Experiment with storage. Scaling the Architecture Raspberry Pi Virtual Machines Bare Metal Servers Evaluate the outcomes

jeneva
Download Presentation

DPM - efficient storage in diverse environments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DPM - efficient storage in diverse environments M. Hellmich, O. Keeble, F. Furano, A. Manzi, A. Ayllon, I. Calvet, R. Rocha, M. Regala CHEP 2013 Amsterdam

  2. Experiment with storage • Scaling the Architecture • Raspberry Pi • Virtual Machines • Bare Metal Servers • Evaluate the outcomes • Performance per Watt, Dollar CHEP2013

  3. Storage CHEP2013

  4. This talk is a reaction Change in our software Change in computing services Change in available hardware CHEP2013

  5. Our software: DPM MD DB Disk Disk Disk Disk CHEP2013

  6. Our software: New DPM anyDB MD GW GW Disk HDFS S3 Ceph CHEP2013

  7. Computing Services anyDB MD Gateway Reqs • >> network • CPU • memory GW GW S3 Disk HDFS ceph Metadata Reqs network CPU memory CHEP2013

  8. Available Hardware: ARM • Low power consumption • also when idle CHEP2013

  9. So we focused on Metadata anyDB MD MD MD GW S3 ceph CHEP2013

  10. The Metric • requests/sec served, pure and simple • Stat requests for files • Deputy for other calls • Tested over http CHEP2013

  11. The Setup t client t MD t t client DB MD client client MD client Disk CHEP2013

  12. Performance Comparison CHEP2013

  13. Scaling up with Memcached *VMs are OpenStack VMs CHEP2013

  14. Total Cost of Ownership TCO = Cost Machines + Cost Running Machines Cost Running Machines = Cost Maintenance + Cost Replacement + Cost Power + Cost External Components Cost SoftwareDev CHEP2013

  15. Performance per Watt CHEP2013 1price per kWh = 0.07 Euros 2idle 30 percent of the time

  16. Performance per Euro Note Rasp external equipment is much more expensive! 1 Buying Price + Power Costs or EC2 Pricing (reserved instance) 2 12VMs running on DELL PowerEdge M620 24 core 16GB RAM, price from website CHEP2013

  17. Working with a Raspberry • Smallest possible machine • But very good for testing • Runs EPEL software* CHEP2013

  18. Summary • Evaluation of a DPM metadata server • VMs do well • Especially with memcached • Raspberries work • Are even competitive • ARM may become very useful for storage CHEP2013

  19. Questions? Thank you very much CHEP2013

More Related