1 / 7

An introduction to Apache Chukwa

A introduction to Apache Chukwa, what is it and how does it work ? Why is it important to monitor Hadoop DFS and how can it help us ?

semtechs
Download Presentation

An introduction to Apache Chukwa

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Apache Chukwa • What is it ? • How does it work ? • What can we collect ? • Architecture www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  2. Chukwa – What is it ? • For log collection and analysis • Designed for big data • Designed for Hadoop • Uses HDFS and MapReduce • Scaleable • Robust • Provides a tool kit to analyse logs www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  3. Chukwa – How does it work ? • Chukwa agents on source nodes • Transfer data to collectors which save data to HDFS • Data sinks contain raw unsorted data • Data sinks clean data • Demux adds structure to create Chukwa records • Chukwa records go to database • Are ready to be analysed www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  4. Chukwa – What can we collect ? • Metrics • System logs • Defined format • Undefined format • Low latency • Access to log data www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  5. Chukwa – Architecture ? www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  6. Chukwa – Architecture ? • Chukwa agents • Reside on the Hadoop machines • Collect raw data • Use adaptors for data sources • Use http to transmit data • Operate on data chunks • Can fail over between collectors www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  7. Contact Us • Feel free to contact us at • www.semtech-solutions.co.nz • info@semtech-solutions.co.nz • We offer IT project consultancy • We are happy to hear about your problems • You can just pay for those hours that you need • To solve your problems

More Related