1 / 20

Srikumar Venugopal 1 , Rajkumar Buyya 1 , Susumu Date 2

GRIDS. Gridbus Resource Broker for Application Service Costs-based Scheduling on Global Grids: A Case Study in Brain Activity Analysis. 1.Gri d computing & D istributed S ystems (GRIDS) Lab. The University of Melbourne Melbourne, Australia www.gridbus.org/

briar-richardson
Download Presentation

Srikumar Venugopal 1 , Rajkumar Buyya 1 , Susumu Date 2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GRIDS Gridbus Resource Broker for Application Service Costs-based Scheduling on Global Grids: A Case Study in Brain Activity Analysis 1.Grid computing & Distributed Systems (GRIDS) Lab. The University of Melbourne Melbourne, Australiawww.gridbus.org/ 2. Cybermedia Center, Osaka University Srikumar Venugopal1, Rajkumar Buyya1, Susumu Date2

  2. What does a Resource Broker do? • Gets user/application requirements • Discovers resources like computational nodes, data sources, etc. • Establishes costs, user credit, etc. • Makes decisions about the optimal schedule for jobs • Dispatches jobs Application Accounting Services Information Services Resource Broker Cataloguing Services … Grid Nodes

  3. Architecture of Gridbus Scheduler Application Visual Parametric Tool Results Gridbus Scheduler Agent+Job Results Access Service (Globus) Bill Globus Globus CPU orPE PE GTS GridBank Cluster Scheduler PE Grid Node Grid Node (e.g., ANL) Grid Node (e.g., VPAC) Grid Node (e.g., UofM) Data Catalogue Grid Info Service ASP Catalogue Grid Market Directory

  4. Gridbus Scheduler • Interfaces with: • Application Development - Visual Parametric Tool • Information Services - Grid Market Directory (Cost), GRIS ,etc. • Accounting Services - Grid Trading Service, GridBank • Cataloguing Services - Application Catalog, Replica Catalog • Job Dispatcher • Nimrod-G (for parametric jobs) • Gridbus Dispatcher (for data intensive, reservation, P-GRADE support, etc.) – work in progress • Supports: • User-specified QoS parameters such as Deadline, budget, etc. • Application Cost or Hardware Cost (CPU, etc) • Cost from Grid Market Directory or Flat File • Cost, Time or Cost-Time Optimization.

  5. Application Service Costs? • Present Approach to Processing Cost - • Timeshare or CPU cycles used • Users – more interested in the cost of getting job done than amount of processing power consumed • New Approach to Cost - • Application Service Costs – charge for using the application once. • Different costs for different applications – depends on provider • Broker finds Cost through Grid Market Directory.

  6. Scheduling Algorithms Gridbus Scheduler implements • Cost Optimization • Minimize computational cost (within deadline) • Time Optimization • Minimize execution time (within budget) • Cost-time Optimization • Similar to cost-optimization • Implemented for first time.

  7. Scheduling (contd..) • Uses past performance to forecast each machine’s capacity • The rate of completion is averaged to compensate for any spikes or troughs • Cost Optimization • Gives maximum jobs to the cheapest machine • Time Optimization • Gives jobs to machines based on consumption rate but limited by budget per job • Cost-Time Optimization • Distributes jobs among the machines of consumption sorted by their consumption rate

  8. Cost Optimization: No. of Jobs Done vs time

  9. Cost-Time Optimization: No. of Jobs Done vs Time

  10. Time Optimization: No. of Jobs Done vs Time

  11. Comparison of Scheduling Algorithms • All experiments were started with • No of Jobs = 200 • Deadline = 2hrs • Budget = 600 Grid $

  12. Case Study: Brain Activity Analysis • In Collaboration with Osaka University, Japan • Computationally and data intensive

  13. B A MEG Data/Brain Activity Analysis • MEG (Magnetoencephalography) • Achieve both non-invasiveness and high degree of measurement accuracy cf. EEG (Electroencephalography), ECoG (Electrocorticography) • Measure functional data on multiple points around the head • Promising among medical doctors and brain scientists. A: B: http://www.ctf.com

  14. Analysis Results Analysis Results MEG data analysis DV transfer Osaka Univ. Data Generation Osaka Univ. Hospital Data Analysis Life-electronics laboratory, AIST Cybermedia Center • Provision of MEG • Provision of expertise in the analysis of brain function

  15. DV transfer Osaka Univ. Virtual Laboratory for medicine and brain science • Knowledge sharing • MEG sharing? • Data Sharing Data Generation Osaka Univ. Hospital Data Analysis Analysis Results Life-electronics laboratory, AIST Cybermedia Center • Provision of MEG • Provision of expertise in the analysis of brain function Analysis Results MEG data analysis

  16. Requirements • Computational and data intensive problem • The number of MEG instruments available is small. • Knowledge of scientists is distributed. • No database? • Different group uses different analysis methods for different data.. • Many medical institutions and hospitals have no computers and that can satisfy doctors’ analysis demand.

  17. f At 1st Phase, wavelet transform t f At 2nd Phase, Wavelet cross-correlation f’ t f t Wavelet cross-correlation analysis Sensor B Sensor A • This analysis procedure needs to be performed for each pair of MEG sensors.　E.g. 64ch -> 2016 Raw MEG Data This image indicates that a brain signal with frequency f’ was detected earlier in Sensor B than in Sensor A.

  18. 64 sensors MEG 2 Data Generation 3 1 Data Analysis 5 Grid Resource Broker (Nimrod-G+Gridbus) Results 4 Life-electronics laboratory, AIST World-Wide Grid [deadline, budget, optimization preference] New Approach: Users QoS Requirements driven MEG Data Analysis on the Grid Analysis All pairs (64x64) of MEG data by shifting the temporal region of MEG data over time: 0 to 29750: 64x64x29750 jobs • Provision of MEG analysis

  19. Grid Enabling MEG data analysis • Nature • fine-grained jobs • small data sets • Data Sets on Source Node • High Latency for small jobs • Lower Efficiency • Hence, data sets were replicated on each node • Application changed to access local datasets • ./metameg-datapath time_offset time_offset_step meg_sensors_count Meg_data_path • Output is collated at the source node and then visualized • Grid Enabled in very short time ~ 1 week

  20. Conclusion • Introduced Gridbus Resource Broker using Application Service Cost • Described the Scheduling Algorithms followed • Presented Case Study of Brain Activity Analysis using our Resource Broker • Future Work: • Integration with Accounting Mechanisms such as GridBank • Support for Group Scheduling and Economic-based Advance Reservation of Resources

More Related