1 / 17

Architecture for Big Data Analytics Leveraging Edge/Fog Paradigms

This paper presents a proposal for a reference architecture that combines cloud, fog, and edge computing paradigms to enable efficient and scalable big data analytics. The architecture includes an Analytical Orchestrator and Life Cycle Management for self-rescheduling and self-monitoring. The proposed architecture aims to address the challenges and shortcomings of existing technologies and provide a comprehensive solution for big data analytics in Industry 4.0.

bodiford
Download Presentation

Architecture for Big Data Analytics Leveraging Edge/Fog Paradigms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards an Architecture for Big Data Analytics Leveraging Edge/Fog Paradigms ECSA, September 9–13, 2019, Paris, France Josu Díaz-de-Arcaya Raúl Miñón Ana I. Torre-Bastida

  2. Outline • Introduction • Background • Motivation • Challenges • Our Proposal • Conclusion & Future Work

  3. Industry 4.0 Introduction

  4. Industry 4.0: Enablers Introduction Big Data IoT Cloud Computing a style of computing in which scalable and elastic IT-enabled capabilities are delivered as a service using Internet technologies. Gartner is high-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing that enable enhanced insight, decision making, and process automation. Gartner is the network of physical objects that contain embedded technology to communicate and sense or interact with their internal states or the external environment. Gartner

  5. Big Data Architectures Background In order to deal with Big Data stringent requirements two leading architectures arose Lambda Kappa https://www.oreilly.com/ideas/applying-the-kappa-architecture-in-the-telco-industry

  6. Computing models Challenges An IT infrastructure which need to widen through cloud fog and edge computing paradigms

  7. Resource Negotiators Challenges 2015 2006 2010 • Shortcomings • Existing technologies are cluster based. • They are not specific for Analytical Workloads, nor do they understand its phases.

  8. Horizontal Orchestrators & MLAAS Challenges Hadoop YARN / Mesos / Kubernetes Orchestrator node_1 node_0 node_2 node_N Machine Learning into production MLaaS

  9. Motivation Motivation We have identified the need for a new reference architecture that includes all the resources in the different layers and should be capable of self-rescheduling and self-monitoring

  10. General Reference Architecture Definition OurProposal

  11. Implementation Technologies Our proposal

  12. Vertical Orchestrator Out proposal Cloud This is where we are going to focus our research and development efforts Fog AnalyticOrchestrator and Life Cycle Management Edge

  13. Our contribution Analytical Orchestrator Module Catalogue Analytical Pipeline Cost Estimation Analytical Pipelines Validation Agent 1 Kernel Agent 2 SCM Agent 3 Analytical Pipeline Orchestration Analytical Pipeline Monitoring Agent N

  14. Our contribution Analytical Orchestrator Module

  15. Conclusion & Future Work Our Contributions Cloud • Reference Architecture that encompasses Cloud & Fog & Edge • We will focus on the Analytic Orchestrator and Life Cycle Management Fog AO&LFM Edge

  16. Future Work Conclusion & Future Work Cloud • In order to implement the Orchestrator: • Define the available resources and the pipelines in detail (PMML, PFA, …). • Optimisation of the pipeline deployment plan will be implemented using an algorithm based on JMETAL framework. Fog AO&LFM Edge

  17. Thanks!

More Related