HBase Tracing
This presentation is the property of its rightful owner.
Sponsored Links
1 / 15

HBase Tracing PowerPoint PPT Presentation


  • 203 Views
  • Uploaded on
  • Presentation posted in: General

HBase Tracing. Elliott Clark. Who Am I. Elliott Clark [email protected] HBase PMC HBase Committer HTrace Mantainer Cloudera Engineer CDH Distribution. What’s Going on During Failovers. Motivation. Want to improve MTTR First need to know what’s going on Google’s Dapper. Options.

Download Presentation

HBase Tracing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Hbase tracing

HBase Tracing

Elliott Clark


Who am i

Who Am I

  • Elliott Clark

    • [email protected]

  • HBase PMC

  • HBase Committer

  • HTrace Mantainer

  • Cloudera Engineer

    • CDH Distribution


Hbase tracing

  • What’s Going on During Failovers


Motivation

Motivation

  • Want to improve MTTR

  • First need to know what’s going on

  • Google’s Dapper


Options

Options

  • Log all the things

    • We already do that.

    • Our logs are way too verbose as it is.

    • Hard to combine multiple logs

      • HRegionServer

      • HMaster

      • Client

  • Guess

  • Zipkin

  • Not Invented Here


Hbase tracing

  • Spoiler:

  • Guessing Didn’t Win


About zipkin

About Zipkin

  • Created by Twitter

    • Thanks

  • Modeled after Dapper

  • Scala

  • Finagle

    • Functional

    • Netty

    • Non-blocking

  • Thrift

  • Cool UI

  • Pluggable Storage Backend


Dapper s terminology

Dapper’s Terminology

  • Span

    • Segment of a remote call

    • Contains Annotations

    • Has a parent trace

    • Can have multiple children spans

    • Can have multiple annotations

  • Annotations

  • Trace

    • Grouping of spans


Hbase tracing

  • Can’t use Cassandra and look at myself in the mirror


Hbase tracing

  • Implementation

Hooking up Zipkin, HTrace, and HBase


Zipkin hbase storage

Zipkin HBase Storage

  • Replaces need for Cassandra

  • Functional

  • Created a non-blocking client shim

  • Fixed Key Length

    • ID’s are long

    • Mapping table

      • ID -> Name

      • Name -> ID

      • Inspiration from OpenTSDB

  • Open source on Github


Htrace

HTrace

  • Way to instrument Java Code

  • Used In HBase trunk

  • Coming to an install near you in 0.96.0

  • Pluggable where it emits Spans

    • To a file

    • To sys.out


Htrace and zipkin

HTrace and Zipkin

  • Modularize HTrace

  • Create htrace-zipkin

  • Create ZipkinSpanReceiver

    • Receives spans from HTrace instrumented code

    • Converts to Thrift objects

    • Sends to Zipkin

      • Background Flush

      • Batching


Hbase tracing

  • Demo

Let’s hope this works.


  • Login