Building a lightweight, flexible D3.js dashboard (Part 1 of 3)

Part 2
Part 3

(You can find the complete sourcecode for the below example here and an interactive version here) I've written before that I think Tableau is the best data exploration / visualization tool on the market. I still think that, but Tableau does suffer a few drawbacks:

  • Cost. The last time I purchased a Tableau license (which was a while ago), the price was $1000 per seat. That's expensive -- prohibitively so for most start-ups.
  • Scalability. One Tableau license does not a BI platform make. Tableau worksheets are great for presentations to large groups, but if PMs / Product Owners / business stiffs need to check data every day, looking over an analyst's shoulder at a Tableau worksheet becomes unrealistic. Enter Tableau Server, which allows for Tableau worksheets to be served and interacted with via the web -- at a considerable cost. More Tableau licenses are needed (as many as active connections must be maintained) on top of a server license.
  • Ease of use. Creating simple visualizations in Tableau can be done easily by someone not familiar with the nuances of the software, but achieving the more sophisticated solutions on which Tableau's reputation was built requires expertise that can only be acquired through ascending Tableau's steep learning curve. Tableau's customer support is almost non-existent -- it takes the form of a user forum, in which a handful of experts help an endless throng of newbies style their charts and connect to non-standard databases. Tableau isn't intuitive, and developing deep proficiency in the product is -- again -- expensive: Tableau offers courses all around the world which purport to teach sophisticated visualization techniques for an exorbitant fee. If you work for Microsoft, this is no problem; just don't forget your gold card at home. If you work for a start-up, learning Tableau this way is not a viable option.
  • Flexibility. Tableau is good at visualizing data in a database, but it's not good at transforming that data. Its Calculated Field functionality works great for simple manipulations (say, converting a number into a percentage), but it's incredibly cumbersome to use for more advanced calculations. This means that any statistical analysis has to be done at the data layer -- which presents de-aggregation problems for some calculations (like LCV).

For these reasons, most start-ups won't have the luxury of using Tableau. In fact, most start-ups won't have the luxury of an analytics group or dedicated analyst. What most start-ups have is a sizable dataset and a need to understand the core business metrics that can help them iterate on their product. This set of posts will provide an introduction to D3.js, a javascript visualizations library, and a tutorial on how to implement a lightweight, flexible dashboard using it. D3.js provides a framework for creating visualizations out of structured data -- it allows for the effortless creation of graph and chart objects that can be binded to the DOM and transformed. By creating a D3.js front-end on top of a middle transformations layer, the flexibility problem that Tableau suffers from is solved: data transformations can be done programmatically and then served to the front-end. The first step in creating a dashboard is to include the relevant libraries; I'll include D3.js and the jQuery and jQuery UI libraries. I'll also add some style elements for later use. Your blank HTML document should look like this:

A D3.js Dashboard
 
 

The metrics div will hold our visualizations object. Most of the style elements should be self-explanatory. The next step in creating the dashboard is to import some data. Your functional dashboard will probably retrieve data from your database via an AJAX call to a helper script (which will query the data and format it in JSON). For the purposes of this tutorial, I'll generate objects and populate them with random data. Each object will have two elements: Date and DAU (daily active users). Add this bit of code to the document.ready() function:

var data = [];
// this is our data array
var startingDate = new Date(2012, 8, 18);
// this is a date object
for (var i = 0; i < 10; i++) {
// loop 10 times to create 10 data objects
var tmpObj = {};
// this is a temporary data object
tmpObj.date = new Date(
startingDate.getFullYear(),
startingDate.getMonth(),
startingDate.getDate()+i
);
// the data for this data object. Increment it from the starting date.
tmpObj.DAU = Math.round(Math.random() * 300);
// random value. Round it to a whole number.
data.push(tmpObj);
// push the object into our data array
}

We have enough of a framework to start visualizing the data we generated. But first we need to create some helper functions: since we are storing our data in objects, we'll need functions to get the minimum and maximum values of the elements in the objects to create the axes of the graph. D3.js actually provides methods for doing this called d3.min() and d3.max(), but these only work on arrays, not on objects. The two helper functions are defined below:

function getMaxObjectValue(this_array, element) {
var values = [];
for (var i = 0; i < this_array.length; i++) {
values.push(Math.ceil(parseFloat(this_array[i][""+element])));
}
values.sort(function(a,b){return a-b});
return values[values.length-1];
}
function getMinObjectValue(this_array, element) {
var values = [];
for (var i = 0; i < this_array.length; i++) {
values.push(Math.floor(parseFloat(this_array[i][""+element])));
}
values.sort(function(a,b){return a-b});
return values[0];
}

Now we have everything we need to build our chart object:

//size the visible area of the graph using
//the margin values
var width = 500, height = 500;
var margin = {top: 30, right: 10, bottom: 40, left: 60},
width = width - margin.left - margin.right,
height = height - margin.top - margin.bottom;
var minDate = (data[0].date),
maxDate = data[data.length-1].date;
minObjectValue = getMinObjectValue(data, 'DAU');
maxObjectValue = getMaxObjectValue(data, 'DAU');
//create the graph object
var vis= d3.select("#metrics").append("svg")
.data(data)
.attr("class", "metrics-container")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")");
var y = d3.scale.linear()
.domain([ minObjectValue - (.1 * minObjectValue) , maxObjectValue + (.1 * maxObjectValue)])
.range([height, 0]),
x = d3.time.scale()
.domain([minDate, maxDate])
.range([0, width]);

A couple of things to note:

  • When we create the visualization, we specify the data to use in the data() method. This expects an array.
  • Since our objects were created in order of ascending date, we can pull the first and last elements of the array and use the date data fields as the minimum and maximum values for the date axis.
  • We create a visualization object (SVG), style it with the classes we defined earlier, then append a grouping element to group the visualizations we'll create later
  • We create two axis scales using the d3.scale() and d3.time.scale() methods. d3.time.scale() allows us to scale the x axis by date. Domain is the range of the values we'll use for the scales. The Range() method helps scale the data to fit into the physical size of the graph.
  • Our visualization object is called vis.

We have now created a visualization object and attached it to the DOM. If you were to load this HTML file in a browser, you'd see this:

d3-dashboard-screenshot

Now we'll add the axes and axes labels with this code:

var yAxis = d3.svg.axis()
.scale(y)
.orient("left")
.ticks(5);
var xAxis = d3.svg.axis()
.scale(x)
.orient("bottom")
.ticks(5);
vis.append("g")
.attr("class", "axis")
.call(yAxis);
vis.append("g")
.attr("class", "axis")
.attr("transform", "translate(0," + height + ")")
.call(xAxis);
//add the axes labels
vis.append("text")
.attr("class", "axis-label")
.attr("text-anchor", "end")
.attr("x", 20)
.attr("y", height + 34)
.text('Date');
vis.append("text")
.attr("class", "axis-label")
.attr("text-anchor", "end")
.attr("y", 6)
.attr("dy", "-3.4em")
.attr("transform", "rotate(-90)")
.text('Daily Active Users');

If you reload your HTML file, you'll see something like this (remember that the DAU values are randomly generated, so the Y axis values will be different each time the javascript is executed): Now we can add the line chart with this code:

var line = d3.svg.line()
.x(function(d) { return x(d["date"]); })
.y(function(d) { return y(d["DAU"]); })
vis.append("svg:path")
.attr("d", line(data))
.style("stroke", function() {
return "#000000";
})
.style("fill", "none")
.style("stroke-width", "2.5");

This creates a line object, gives it values (x values from the date element and y values from the DAU element of the data objects), appends a path object to the visualization object (vis) based on the line's data, and gives that path object a "stroke" (color) of black. You should now see a line graph on your chart: We can now add some data points to the path by creating circle objects and appending those to the visualization object (vis). We will also include some mouseover effects for browsing through the data points:

var dataCirclesGroup = vis.append('svg:g');
var circles = dataCirclesGroup
.selectAll('.data-point')
.data(data);
circles
.enter()
.append('svg:circle')
.attr('class', 'dot')
.attr('fill', function() { return "red"; })
.attr('cx', function(d) { return x(d["date"]); })
.attr('cy', function(d) { return y(d["DAU"]); })
.attr('r', function() { return 3; })
.on("mouseover", function(d) {
d3.select(this)
.attr("r", 8)
.attr("class", "dot-selected")
.transition()
.duration(750);
})
.on("mouseout", function(d) {
d3.select(this)
.attr("r", 3)
.attr("class", "dot")
.transition()
.duration(750);
});

A few notes:

  • We first create the dataCirclesGroup group element to group all circles together. This will be more important in the next blog post, in which some graphs will contain multiple line charts.
  • We append circle objects, style them with the "dot" style defined earlier, give them a "fill" (color) of red and a radius (r) of 3. Note that all size parameters in d3 are automatically interpreted in pixels.
  • We then create some "mouseover" and "mouseout" functions to increase and decrease the size of the circles.

At this point, we have a fully functional line chart with circles superimposed on each point that animate on mouseover. The next post in this series will extend this basic functionality into a framework that allows for dynamic chart creation and multiple graphs on each chart.

© 2012-2013 Mobile Dev Memo