apache chukwa l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
An introduction to Apache Chukwa PowerPoint Presentation
Download Presentation
An introduction to Apache Chukwa

Loading in 2 Seconds...

play fullscreen
1 / 7

An introduction to Apache Chukwa - PowerPoint PPT Presentation


  • 191 Views
  • Uploaded on

A introduction to Apache Chukwa, what is it and how does it work ? Why is it important to monitor Hadoop DFS and how can it help us ?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'An introduction to Apache Chukwa' - semtechs


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
apache chukwa
Apache Chukwa
  • What is it ?
  • How does it work ?
  • What can we collect ?
  • Architecture

www.semtech-solutions.co.nz info@semtech-solutions.co.nz

chukwa what is it
Chukwa – What is it ?
  • For log collection and analysis
  • Designed for big data
  • Designed for Hadoop
  • Uses HDFS and MapReduce
  • Scaleable
  • Robust
  • Provides a tool kit to analyse logs

www.semtech-solutions.co.nz info@semtech-solutions.co.nz

chukwa how does it work
Chukwa – How does it work ?
  • Chukwa agents on source nodes
  • Transfer data to collectors which save data to HDFS
  • Data sinks contain raw unsorted data
  • Data sinks clean data
  • Demux adds structure to create Chukwa records
  • Chukwa records go to database
  • Are ready to be analysed

www.semtech-solutions.co.nz info@semtech-solutions.co.nz

chukwa what can we collect
Chukwa – What can we collect ?
  • Metrics
  • System logs
    • Defined format
    • Undefined format
  • Low latency
    • Access to log data

www.semtech-solutions.co.nz info@semtech-solutions.co.nz

chukwa architecture
Chukwa – Architecture ?

www.semtech-solutions.co.nz info@semtech-solutions.co.nz

chukwa architecture6
Chukwa – Architecture ?
  • Chukwa agents
    • Reside on the Hadoop machines
    • Collect raw data
    • Use adaptors for data sources
    • Use http to transmit data
    • Operate on data chunks
    • Can fail over between collectors

www.semtech-solutions.co.nz info@semtech-solutions.co.nz

contact us
Contact Us
  • Feel free to contact us at
    • www.semtech-solutions.co.nz
    • info@semtech-solutions.co.nz
  • We offer IT project consultancy
  • We are happy to hear about your problems
  • You can just pay for those hours that you need
  • To solve your problems