1 / 9

An Introduction to Apache Hama

An Introduction to Apache Hama, what is it ? Why is it useful for scientific computations on big data ?

semtechs
Download Presentation

An Introduction to Apache Hama

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Apache Hama • What is it ? • What is it used for ? • How does it work ? • Architecture www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  2. Hama – What is it ? • A Distributed computing framework • For massive scientific calculations • For distributed computing • Uses Bulk Synchronous Parallel Computing ( BSP ) • Written in Java • Licensed via Apache • Available for Hadoop greater than 1.0.x • Requires Java 1.6.x or higher www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  3. Hama – What is it used for ? • large scale computations that • exceed the abilities of Map reduce • Machine learning • Matrix • Graph • Network Algorithms www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  4. Hama – How does it work ? • A series of super steps • Each of which is • Concurrent computation on local data • Point to point communication • Synchronise all processors • Uses Hadoop RPC for communication www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  5. Hama – Architecture Hama consists of BSP Master, GroomServers, ZooKeeper www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  6. Hama – Architecture • Role of BSPMaster • Maintain groom server statuses • Controlling cluster super steps • Maintaining job progress information • Scheduling Jobs • Assign tasks to groom servers • Fault control • Provide cluster control to users www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  7. Hama – Architecture • Role of GroomServer • Performs BSP tasks • Reports status to master • Runs on a single HDFS data node • Starts with a BSP Peer instance www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  8. Hama – Architecture • Role of ZooKeeper • Manages barrier synchronisation of BSP Peers www.semtech-solutions.co.nz info@semtech-solutions.co.nz

  9. Contact Us • Feel free to contact us at • www.semtech-solutions.co.nz • info@semtech-solutions.co.nz • We offer IT project consultancy • We are happy to hear about your problems • You can just pay for those hours that you need • To solve your problems

More Related