1 / 18

Semantic Based Storage QoS Management Methodology

Semantic Based Storage QoS Management Methodology. Renata Słota, Darin Nikolow, Jacek Kitowski. Institute of Computer Science AGH-UST, Krakow, Poland ACK Cyfronet AGH, Krakow, Poland. Research supported by MNiSW grant nr N N516 405535. Outline. Introduction

avent
Download Presentation

Semantic Based Storage QoS Management Methodology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. KUKDM’2011, Zakopane Semantic Based Storage QoS Management Methodology Renata Słota, Darin Nikolow, Jacek Kitowski Institute of Computer Science AGH-UST, Krakow, Poland ACK Cyfronet AGH, Krakow, Poland Research supported by MNiSW grant nr N N516 405535

  2. KUKDM’2011, Zakopane Outline • Introduction • Common mass storage system model and ontology • Storage system performance monitoring and estimation • Use cases • Summary

  3. KUKDM’2011, Zakopane Introduction • There are applications (called data intensive applications), which: • use data storage systems intensively, • Have constantly growing demands concerning capacity and storage system efficiency. • Example of applications dealing with huge amounts of data • Scientific applications (simulations, out-of-core computations, HEP experiments), • Backup & restore, archiving, disaster recovery. • The performance of the data intensive applications depend on the performance of the underlying storage system • Applications running in distributed environments need some QoS (Quality of Service) concerning data access

  4. KUKDM’2011, Zakopane Introduction(cont.) • Storage QoS metrics include • data access latency • data access bandwidth • storage space • data availability • Diversity of storage systems • Hierarchical Storage Management (HSM) systems • Disk arrays • Tiered storage • Problem of efficient storage performance utilization while respecting the storage QoS constrains • Storage performance monitoring and data transfer scheduling are necessary

  5. KUKDM’2011, Zakopane Goal of research The subject of this research is the development of semantic-based storage management methodology allowing to achieve QoS for storage performance metrics. As part of this research the following has been done: • Development of common mass storage system model (C2SM) and relevant ontology • Development of storage performance monitoring sensors and estimators • Implementation of two use cases demonstarting our approach The research has been done within the OntoStor project http://www.icsr.agh.edu.pl/ontostor

  6. KUKDM’2011, Zakopane C2SM • Is a common mass storage system model, which can be used to describe the state of a storage system • Consists of • set of well defined storage performance related parameters • Algorithms specifying the storage system behaviour • Is based on the Common Information Model – CIM • Is used in our methodology for unifying the description of storage performance parameters of heterogenious storage systems

  7. KUKDM’2011, Zakopane C2SM class diagram

  8. KUKDM’2011, Zakopane OntoStor ontology • Developed based on C2SM • Has been created semi-automatically using the ‘cim2owl’ tool, which has been developed at the DCS AGH • The ontology is used in our methodology to find storage resources using semantic queries

  9. KUKDM’2011, Zakopane OntoStor ontology diagram

  10. Storage System Performance Monitoring • Two components have been defined in our methodology: • sensors – obtaining storage performance parameters, • Storage system dependent • estimators – estimating the future storage performance based on the data from the sensors. • Simulational, rule-based, statistical • Three types of systems supported – local disk, disk array, HSM system KUKDM’2011, Zakopane

  11. Testbed

  12. KUKDM’2011, Zakopane OntoStor Portal

  13. KUKDM’2011, Zakopane Disk array monitoring

  14. KUKDM’2011, Zakopane Monitoring of loaded disk array

  15. KUKDM’2011, Zakopane Estimation test

  16. KUKDM’2011, Zakopane Use cases • Two use cases have been implemented using the proposed methodology • Data access optimization with replication • Finding the best location for newly created replica for write access • Selecting the best existing replica for read access • Components used: • Sensors, estimators, monitoring system, C2SM • SLA (Service Level Agreement) monitoring • Components used: • Sensors, monitoring system, OntoStor ontlogy, QoS metrics ontology, C2SM

  17. KUKDM’2011, Zakopane SLA monitoring QoS metric limits How to obtain values of QoS metrics How to monitor performance parameters Description of parameters

  18. KUKDM’2011, Zakopane Summary • Semantic-based storage management methodology based on monitoring and estimation of storage performance has been presented • Common mass storage model and its relevant ontology has been proposed • A set of sensors and estimators has been implemented • Two use cases have been implemented using the proposed methodology • The proposed methodology has been used in FiVO/QStorMan.

More Related