140 likes | 245 Views
HBase and Metrics2. Headline Goes Here. DO NOT USE PUBLICLY PRIOR TO 10/23/12. Elliott Clark | Engineer January 2013. Speaker Name or Subhead Goes Here. HBase Metrics Circa 0.90 to 0.94. Spread around lots of places Using old deprecated Metrics Using Lots of ConcurrentHashMaps
E N D
HBase and Metrics2 Headline Goes Here DO NOT USE PUBLICLY PRIOR TO 10/23/12 • Elliott Clark |Engineer • January 2013 Speaker Name or Subhead Goes Here
HBase Metrics Circa 0.90 to 0.94 • Spread around lots of places • Using old deprecated Metrics • Using Lots of ConcurrentHashMaps • Some metrics stored two places • Inconsistent Naming
Lets Fix it • Replace everything with metrics2 • Rename everything • Make things faster • Remove un-needed metrics
Problems • Reflection would be hard and slow • Have to support both Hadoop 1 and Hadoop 2 • Need to add functionality to MetricsRegistry
Solution • Two implementations • One for Hadoop 1 -- hbase-hadoop1-compat • One for Hadoop 2 – hbase-hadoop2-compat • Layer connecting hbase-server to correct compat • Use ServiceLoader so there’s no compile dependency
Jar Dependencies hbase-server hbase-hadoop-compat hbase-hadoop2-compat hbase-hadoop1-compat
Performance • Performance improvements • 9% on scans. • ~5% on gets • < 1% on all calls.
Naming • Metrics have changed names • All Camel Cased • Count used as suffix to denote a counter • Metrics have descriptions
Better Sink Support • Can filter metrics at the source • Some sources can take per region metrics • Others only need small subset
What else Does the HBase user see ? • More Histograms meaning more data • Metrics should be MUCH easier to find in JMX • Ability to Filter what metrics are reported where • OpenTSDB will need a patch to work
Wrap Up • Trunk (soon to be 0.96) now on Metrics2 • Still needs more documentation