Production Visualization and Metrics Tools

Being able to generate data about your app is crucial, but you also need to be able to understand it. That’s where production visualization and metrics tools come in. These tools take (or are given) your data and provide you with key metrics and/or digestible visualizations of what’s happening.

The first decision (after deciding to use one of these tools at all), is between open source (we cover Graphite and ELK stack) and SaaS (we cover Keen IO, Librato, and DataDog) options. The downside to the SaaS ones is that you have to weave them into your code, which is essentially a big investment. Weaving them into your code creates a huge dependency, so if you get to high scale, you’re in trouble. The downside to open source tools is they require you to install and set them up yourself. This requires a lot of machines if you get to high scale that now require monitoring, which you don’t have with SaaS. Either way, the cost of changing tools in the visualization/metrics or log analyzer space is huge.

There can be some uncertainty around using these tools instead of, or in addition to, log management tools. There is plenty of overlap between these two areas (the ELK stack covers both for example), but there are differences too. Your decision will come down to your budget and environment, but one thing to be aware of is the implicit cost of log analyzers. Any data center you’re hosted on will charge you for the bandwidth you use, and log analyzers tend to use much more bandwidth than metrics tools. One way to decide if you need log analyzers or visualization/metrics tools is journaling. If you need journaling, which is info on specific events or users, you have to use a log analyzer. If you don’t need journaling, you can use one of the metrics tools.

ELK stack

Graphite (and Grafana)

Keen IO

Librato

DataDog

 


ELK Stack

 

elk-stack

 

The ELK stack is a combination of three open source products from Elastic: Elasticsearch’s search and analytics capabilities, Logstash as the logs aggregator, and Kibana for the fancy dashboard visualization. We’ve been using it at Takipi for a while, feeding it from Java through our logs and Redis, and it’s in use both by developers and for BI. Today, Elasticsearch is pretty much built-in with Logstash, and Kibana is an Elastic product as well, making integration and setup very simple.

When a new deployment rolls out, the dashboards follow custom indicators that you can set up about your app’s health. These indicators update in real time, allowing close monitoring when freshly delivered code takes its first steps after being uploaded to production.

kibana

When to use it: If you want to deploy open source tools. The ELK stack crosses several tool areas (log management, search and analytics, and visualizations), so if you’re interested in using tools in all those areas and want them to be tightly integrated. You can implement them separately as well, but much of the benefit is in the combined group.

Price: Free, but there are Elastic paid plans that include different levels of professional support for the tools.

Pros:

  • Strong communities
  • Kibana has a wide range of uses, including business analytics, that other visualization and metrics tools do not
  • The combination of Elasticsearch for metrics and Kibana for visualizations is very smooth and tightly integrated
  • No price barrier

Cons:

  • As they are open source tools, you need to install and set them up yourself
  • Requires a lot of machines that need monitoring if you use at higher scale

Grafana

 

graphite

Graphite is an open source tool written in Python for storing data and providing graphs and visualizations of it on demand. It does not collect any data itself, but instead works with other tools for that purpose. Grafana is an open source tool that provides a metrics dashboard and graph editor for Graphite. It essentially enhances what you can get from Graphite and provides more features. Graphite and Grafana are designed to be language agnostic. Also worth noting is Hosted Graphite, which essentially offers Graphite and Grafana as a paid SaaS model.

grafana

When to use it: If you want an open source tool that is focused only on visualizations and metrics without diving into more complicated analytics or data collection. If you want a strong DevOps focused tool.

Price: Free

Pros:

  • Reduces reliance on log shipping. Looking at Graphite’s graphs can diagnose some issues without diving into logs
  • Excellent at creating real-time graphs of numeric and time-differentiated data
  • No price barrier
  • Grafana provides many additional features and spiffy looking visuals to Graphite

Cons:

  • Requires a separate effort (and coding) to send data to it, as it collects nothing on its own
  • Some lulls with development at times, but it seems to be active these days
  • Open source tool, so the installation, setup, and scaling concerns fall on your shoulders
  • Graphite on its own is fairly unappealing from a GUI standpoint

 


Keen IO

 

keenio-logo

 

Keen IO is a SaaS analytics infrastructure platform. It collects the data you specify from a wide range of possibilities, and its APIs perform the analytics and visualization tasks that you request. It’s designed around providing flexibility and choice to users for whom other visualization and metrics tools aren’t satisfying.

keenio

When to use it: If you can’t get the exact data you want from other tools but can’t or don’t want to build your own custom analytics infrastructure. If analytics is a primary focus or concern for you, and you want more than an out-of-the-box solution provides.

Price: Sliding scale from free to $2000/month, based on the number of events per month you need.

Pros:

  • Requires relatively low overhead compared to similar tools
  • Strong visualizations for real-time or archived data
  • Gives you the ability to find out nearly anything about your data
  • Easily shareable and extractable data

Cons:

  • Requires you to weave it into your code, which creates dependencies and scaling challenges
  • Using a customizable tool like this means that you’ll have to customize it, which makes installation and setup a greater lift

 


Librato

 

librato-logo

 

A hosted service for monitoring and managing cloud applications, Librato can create custom dashboards in seconds without a need to set up or deploy any software. It focuses on graphics and visuals to create a very smooth look and feel compared to other dashboards. It recently became part of SolarWinds.

librato

When to use it: If you want to be able to work with highly customized metrics and reports. If you want beautiful visualizations and good alerting for the data that you send it.

Price:
Librato is priced per metric you want on a metered time basis. The pricing is complicated to figure out, but is tailored to what you actually use more so than other pricing models in this space.

Pros:

  • It would be hard to find anything that Librato doesn’t know how to talk with and help make sense of its data.
  • Good at collating and visualizing data. Dashboards are fluid and look great at any size.
  • Provides custom reports on metrics & alerts through email, HipChat, Campfire, and just HTTP POST requests to integrate with anything you can think of.
  • Delivers alerts automatically when metrics cross certain thresholds.
  • Strong for analytics on the data you send to it.

Cons:

  • Pricing is complicated and obscure
  • Requires you to weave it into your code, which creates dependencies and scaling challenges
  • Doesn’t collect data on its own. It shows you only what you send to it.

 


Datadog

 

datadog-logo

 

DataDog is SaaS monitoring tool targeted for DevOps teams that takes data from your app and a wide range of other tools and provides metrics and visualizations. It unifies data produced from your infrastructure and software into one location. They are currently built around aggregating and presenting data rather than performing analytics.

data-dog

When to use it: If you want a DevOps focused tool that works well with an extensive ecosystem of other tools. If you want a tool that presents you a unified view of data generated across your hardware and software.

Price: Free version, and a Standard version with a sliding scale of $15/host/month. Scale is based on number of hosts.

ELK stack vs Graphite vs Keen IO vs Librato vs DataDog:The first step to choosing which of these tools to use is if you want to go open source (Graphite/ELK stack) or SaaS (Keen/Librato/DataDog). The second step is to determine the importance of analytics to you vs more visualization-only tools. If analytics is a big driver, Keen IO or ELK stack is where you should start your search. If you’re more DevOps focused, Graphite and DataDog are targeted for this audience. Librato is a bit broader and has its fingers in both camps. If visualizing what’s going on in your app and environment is your driver, Librato, DataDog, Graphite, and ELK stack are worth exploring.

Resources:

7 New Tools Java Developers Should Know

15 Tools Java Developers Should Use After a Major Release

 

Takipi – Takipi shows you when and why your code breaks in production. It detects caught and uncaught exceptions, HTTP and log errors, and gives you the code and variable state when they happened. Visualization and metrics tools help you understand and see (and sometimes analyze) the data your app generates. Unlike these tools. Takipi generates data itself, telling you about all your errors in production and gives you the actionable information you need to fix them. No APIs or dependencies are needed for Takipi. Start a free trial