1 / 10

Apache Bahir

This presentation gives an overview of the Apache Bahir project. It explains the Bahir project in terms of it's Spark and Flink extensions and why it is useful and important. <br> <br>Links for further information and connecting<br><br>http://www.amazon.com/Michael-Frampton/e/B00NIQDOOM/<br><br>https://nz.linkedin.com/pub/mike-frampton/20/630/385<br><br>https://open-source-systems.blogspot.com/

semtechs
Download Presentation

Apache Bahir

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What Is Apache Bahir ? ● Provides extensions for Apache Spark and Apache Flink ● Open source / Apache 2.0 license ● Streaming connectors and SQL data sources ● One grouped location for extensions ● Initiated in 2016 from Spark project ● A source for current and future extensions

  2. Apache Bahir Flink Extensions ● Streaming Connectors – ActiveMQ connector – Akka connector – Flume connector – InfluxDB connector – Kudu connector – Netty connector – Redis connector

  3. Apache Bahir Spark Extensions ● SQL Data Sources – Apache CouchDB/Cloudant data source ● Structured Streaming Data Sources – Akka data source – MQTT data source (new Sink)

  4. Apache Bahir Spark Extensions ● Discretized Streams (DStreams) Connectors – Apache CouchDB/Cloudant connector – Akka connector – Google Cloud Pub/Sub connector – Cloud PubNub connector – MQTT connector – Twitter connector – ZeroMQ connector (Enhanced Implementation)

  5. Apache Bahir Importance ● Seems like a small project ? But it covers – Multiple Spark extensions – Multiple Flink extensions – Possible future extensions ● Why is it important ? – Knowledge of this project … – Aids reuse, avoids the need to recreate connectors – Saves money and time !

  6. Apache Bahir Status ● OK great project but is it current ? ● Started in 2016 but is it still going ? ● Check Github ● https://github.com/apache/bahir-flink – Last update 27/05/2020 => current ● https://github.com/apache/bahir – Last update 20/01/2020 => current

  7. Apache Bahir Documentation ● Flink connector documentation describes – Dependencies – Version compatibility – Source and sink classes – Linking for cluster execution

  8. Apache Bahir Documentation ● Spark connector documentation describes – Linking – Configuration – Examples ●Scala ●Java ●Python ● Taking MQTT as an example ● Documentation is comprehensive

  9. Available Books ● See “Big Data Made Easy” Apress Jan 2015 – See “Mastering Apache Spark” ● Packt Oct 2015 – See “Complete Guide to Open Source Big Data Stack ● “Apress Jan 2018” – ● Find the author on Amazon www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ – Connect on LinkedIn ● www.linkedin.com/in/mike-frampton-38563020 –

  10. Connect ● Feel free to connect on LinkedIn –www.linkedin.com/in/mike-frampton-38563020 ● See my open source blog at open-source-systems.blogspot.com/ – ● I am always interested in – New technology – Opportunities – Technology based issues – Big data integration

More Related