1 / 22

Two-Level Load Balancing for Global Data Access in Onedata

Explore the implementation of effective two-level load balancing for global data access utilizing DNS integration and request dispatcher in Onedata system. This solution ensures scalability, high availability, efficiency, and avoids bottlenecks, providing efficient access to distributed user data.

karinb
Download Presentation

Two-Level Load Balancing for Global Data Access in Onedata

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Two-Level Load Balancing for Ł. Opioła, M. Wrzeszcz, Ł. Dutka, R. Słota, J. Kitowski ACC Cyfronet AGH Department of Computer Science,AGH - UST KUKDM 2015 Zakopane, Poland, March 11-13, 2015

  2. Agenda • Global data access • onedata – a tool for global data access • Addressed problem • Two-level load balancing • Test results • Conclusions

  3. Global data access • Global, scientific data access in e-science • User requirements • Onedata – a powerful data globalization system

  4. onedata Distributed user data Globally unified data space in onedata

  5. Globally unified data space in onedata REST API Efficient Access to data without need to migrate it before procesing

  6. Straightforward support for groups and file sharing

  7. onedata architecture

  8. Addressed problem Effective load balancing in a cluster of nodes: • Scalability • High availability • Efficient resource utilization • Bottlenecks avoidance

  9. Two-level load balancing • Level I: DNS • Integrated with the cluster • Distributes load among interfaces • Elasticity in case of node overload or failure • Level II: Request dispatcher • Reacts to load fluctuations • Handles edge cases • Is either one not enough?

  10. Two-level LB at work

  11. Two-level LB at work

  12. Two-level LB at work

  13. Two-level LB at work

  14. Two-level LB at work

  15. Two-level LB at work

  16. Two-level LB at work

  17. Two-level LB at work

  18. Test setup • Requests: • short (quick responses) • long (computationally requiring) • heavy (lots of network I/O) • Test cases: • Round-Robin (for reference) • DNS • Dispatcher • DNS + Dispatcher

  19. Test results – long & short & heavy

  20. Test results - dispatcher

  21. Conclusions • Two-level load balancing was successfully implemented and has proven a viable solution • It combines best features of the two approaches • The overheads do not have impact on the performance of the whole system • onedata can rise up to challenges of global data access

  22. Thank you onedata homepage: http://www.onedata.org VeilFS in PLGrid: http://www.veilfs.plgrid.pl

More Related