Prometheus

1. What Is Apache Prometheus ? ● A system for event monitoring and alerting ● Open source / Apache 2.0 license ● Records real-time metrics in a time series database ● Released by Cloud Native Computing Foundation ● Storage in memory / on disk / various integrations ● Has multiple visualisation options ● Written in Go

2. What Is Apache Prometheus ? ● Provides PromQL query language ● Has a highly dimensional data model ● Has many third party integration options ● Many client libraries available ● Alerting via PromQL and alertManager ● Has wide community support

3. Prometheus Client Libraries ● Official client libraries include – Go, Java, Scala, Python, Ruby ● Unofficial client libraries include – Bash, C, C++, Common Lisp, Dart – Elixir, Erlang, Haskell, Lua for Nginx – Lua for Tarantool, .NET / C#, Node.js – Perl, PHP, R, Rust ● Also use text base exposition formats to – Expose metrics to Prometheus

4. Prometheus Storage ● By default Prometheus stores to local file system ● But this storage is not clustered or replicated ● Does not scale and is not durable, alternatives include Google Cloud Spanner: (R/W) QuasarDB: (R/W) AppOptics: (W) ● ● ● Graphite: (W) SignalFx: (W) Azure Data Explorer: (R/W) ● ● ● InfluxDB: (R/W) Splunk: (R/W) ● ● Azure Event Hubs: (W) ● IRONdb: (R/W) TiKV: (R/W) ● ● Chronix: (W) ● Kafka: (W) Thanos: (R/W) ● ● Cortex: (R/W) ● M3DB: (R/W) VictoriaMetrics: (W) ● ● CrateDB: (R/W) ● OpenTSDB: (W) Wavefront: (W) ● ● Elasticsearch: (W) ● PostgreSQL/TimescaleDB: (R/W) ● Gnocchi: (W) ●

5. Architecture ● Prometheus server scrapes and stores time series data ● Client libraries for instrumenting application code ● Push gateway for supporting short-lived jobs ● Special-purpose exporters for services like – HAProxy, StatsD, Graphite, etc. ● Alertmanager to handle alerts ● Various support tools

6. Architecture

7. Prometheus Usage ● Works well for – Recording any purely numeric time series data – Machine-centric monitoring – Monitoring dynamic service-oriented architectures ● Does not works for – Situations where you need 100% accuracy – Such as for per-request billing

8. Prometheus Users

9. Visualisation ● Uses Grafana data source for visualisation ● Can create Grafana graphs and dashboards ● Can import pre-built dashboards from Grafana.com ● Can have dashboards update in real time ● See next slide for an example

10. Visualisation

11. Prometheus PromQL ● Select and aggregate time series data in real time ● Display data as graph, table or send to HTTP API ● PromQL expressions evaluate to – Instant vector ●A set of time series containing a single sample for each time series, all sharing the same timestamp – Range vector ●A set of time series containing a range of data points over time for each time series – Scalar, String ● Instant vector is the only type that can be directly graphed

12. Prometheus PromQL Examples Return all time series with the metric http_requests_total ● – http_requests_total Return all time series with the metric ● – http_requests_total and – the given job and handler labels: – http_requests_total{job="apiserver", handler="/api/comments"} Return a whole range of time ● – (in this case 5 minutes) for the same vector – making it a range vector: – http_requests_total{job="apiserver", handler="/api/comments"}[5m]

13. Available Books ● See “Big Data Made Easy” Apress Jan 2015 – See “Mastering Apache Spark” ● Packt Oct 2015 – See “Complete Guide to Open Source Big Data Stack ● “Apress Jan 2018” – ● Find the author on Amazon www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ – Connect on LinkedIn ● www.linkedin.com/in/mike-frampton-38563020 –

14. Connect ● Feel free to connect on LinkedIn –www.linkedin.com/in/mike-frampton-38563020 ● See my open source blog at open-source-systems.blogspot.com/ – ● I am always interested in – New technology – Opportunities – Technology based issues – Big data integration