D3.js Data Visualizations

Screen Shot 2011-11-28 at 8.38.22 PM

I’ve been playing with an incredible visualization library over the past few weeks called D3.js. I’m really amazed at how much work Mike Bostock and others have put into this library (they really know their stuff!) and I’ve been very impressed with the direction D3 has taken. I’ve used my fair share of charting libraries in the past and while they are good at getting you 80% of the use cases you need, they fall down in the 20% of the edge cases PM or UX inevitably throws at you. Whenever I use a charting framework, the 20% always takes A TON of time. Many times the 20% is unplanned and sprung on the developer later in the development cycle which leads to unnecessarily long timelines, or short-sighted hacks that you pay for later.

I’ve spent hours digging into charting frameworks looking for classes to extend or behaviors to inject in order to cram some functionality into it that were never intended by the framework creators. Several of my posts in the past have actually demonstrated some of the tricky solutions I’ve had to use to twist a framework’s arm to give me what I need. D3 however is very different.

D3 is built to interact with the DOM and gets you as close to the visual representation as you possibly can without hiding it behind an API you have to learn. If you know how to manipulate SVG and the DOM then you have a boost on a charting framework which requires you to learn their particular abstraction. This allows you to leverage that knowledge you may already have, or provide you the opportunity to learn techniques which will outlive any charting framework down the road.

D3 is also extremely good at providing transition support. You can easily inject transitions whenever data is added, removed, or updated with some simple method chaining. The example I created shows exactly how easy this is to pull off. I’d love to see any charting framework allow for custom transitions like the ones in my example without leaving the developer writhing on the ground in agony.

Here is my example. Click the redraw button to randomly change the data and see the animations I made! View source to see the javascript I used to create it. Here are a couple of my favorite things about this example (and D3 by extension):

  1. Notice how easy it is to create transitions and even chain them with a delay by simply adding enter() and exit() handlers. Updates are handled by the selectors. Lines enter from the bottom of the y axis and drift into their positions then exit in reverse when they are destroyed.
  2. Each axis has beautiful transitions for adding/removing/updating grid lines and labels. The axis component comes in the D3 library and is pretty easy to configure.
  3. As data points increase to the point where showing circle data point renderers doesn’t make sense, I can animate them to radius = 0 and hide them from the user in a subtle way.
  4. D3 is really good at interpolation. You can click that “redraw” button til you’re blue in the face and D3 handles all the transitions without breaking or getting stuck in a bad state!
  5. D3’s scaling system is a beautiful thing. I can bind dates to the x axis with ease and request an x position for any date and receive it. It makes positioning extremely simple.

If I peaked your interest in D3, feel free to check out D3’s home on github which includes some great tutorials. I also like Mike’s basic examples of how D3 works here and his talk at W3Conf. I also found the tutorials here and here very useful.

New Job at Adobe

This post has been a long time coming, but a few weeks ago I decided to take a job with Adobe working in the Omniture Business Unit.  It was a pretty difficult decision (I really love Rain!), but in the end I felt that joining Adobe was the best move for me and my family.  My job here will be to help bring Flex and AIR technologies to SiteCatalyst and other products within the Omniture Suite.  I’ll also be spending a little more time in PHP and traditional front end web technologies (HTML, CSS, JS).

My biggest fear in leaving Rain was that I would miss out on some of the great aspects of agency-life:  lots of new and varying projects.  I had many opportunities to lead high-profile projects doing cutting edge work to help our clients stay ahead of the curve.  Besides Flex and AIR, I also got to build mobile applications on iOS and Android.   My two favorite projects were Playground Sessions (soon to be in a closed Alpha release) and PianoMarvel (both AIR desktop applications that teach people how to play piano).  Rain is currently positioned as a developer-heavy digital agency and prides themselves on being able to do all the crazy awesome technically complex projects that other more traditional agencies don’t have the chops for.  Needless to say I learned a lot working there.

One of the things that gets me excited about working for Adobe is that I’m going to have similar experiences here.  There are a lot of great ideas that have been brewing for a long time and I’ll get to be the catalyst to see them through with my unique experience.  I say unique because at this time Aaron Hardy (great buddy at Rain who joined Adobe at around the same time I did) and myself are really the only experienced Flex/AIR developers in the entire Omniture business unit of Adobe.  Plus, ever since I began learning Flex, I have begun to develop an affinity for Adobe and their products.  It has always been one of my goals to work for Adobe for many years now because I love the company and the community.  Now I’m excited to play a much more central role in it!

PieChart with variable inner radius

At Rain I have the privilege of working with some of the most talented designers in the industry. They often dream up new ways to visualize data. While it’s exciting to see what they come up with, it’s also kind of daunting from time-to-time because I have to figure out a way to build these views/components!

Recently I had the challenge of trying to implement a pie chart which changed its innerRadius whenever a wedge in the chart was rolled over. The designer wanted it to grow and animate to that new radius.  Essentially they wanted something kinda like this:

Continue reading PieChart with variable inner radius

Transmission AS3 – Jar directories

Just added a small little feature to Transmission today. If you’re not familiar with Transmission, it’s an AIR 2 to Java Native Process communication layer.

Instead of having to specify each jar one at a time, you can specify a directory that contains all of your Java jars. The TransmissionDemo project contained in the repo has been updated to show this.

Example from TransmissionDemo
Example from TransmissionDemo

Transmission AS3 – Remote debugging AIR and Java

Today I added a new feature to Transmission which allows for debugging the java side of your application. The step-by-step details for making it work are as follows (I will be using the TransmissionDemo project in the repo to illustrate this).

1. An optional parameter has been added to the init method of the Flex side of Transmission which by default sets remote debugging to true. Make sure that in production this parameter is set to false.
Transmission init method changes

2. Open up Flash Builder, and import the TransmissionDemo and Transmission projects. Then run the TransmissionDemo project. Now it’s time to set up the Eclipse debugging environment. Minimize the AIR application for now…

3. Open up Eclipse and import the TransmissionDemo and Transmission projects.

4. Now we will setup the remote configuration in Eclipse by going to the Debug Configurations screen. Right click on the TransmissionDemo project -> Debug As -> Debug Configurations

Open the Debug Configurations menu

5. Now let’s set up the connection. First right click on Remote Java Application and choose New. Next make sure the TransmissionDemo project is chosen. Finally choose Standard (Socket attach), host: localhost, and port: 8000 and then click the Debug button.


5. Set a breakpoint on the first line of the DemoMessageHandler.java file in the TransmissionDemo project within Eclipse.

6. Return to the TransmissionDemo AIR window and click on the “Send Transmission Message” button.

Open TransmissionDemo and click on the button

7. You should hit your breakpoint.


Transmission AS3 gets a logo and forum support!

Just finished making a logo for Transmission.  Let me take you through my revision process.

I first started with kind of a retro-ish look.


Then I kind of toyed with the radio tower idea in the Transmission name for a little bit:


It was OK, but I still wasn’t loving it.  So I went in a different direction:


Then finally I settled on a logo I liked.  Kind of coming full circle but with a more modern look.


I’m definitely not a designer and it’s pretty simple, but I kinda like it.  Let me know what you think!

We also have a forum set up for questions.

As always, the code is hosted here.

Transmission AIR 2 to Java — Hide Java from Mac Dock

Often when developing an application for AIR 2.0 we realize that for whatever reason we simply can’t accomplish the task at hand without involving another language such as Java. There are several reasons why this might be the case, the fact that AIR 2.0 doesn’t work with non-storage USB devices such as a MIDI piano, another might be the use of Java multi-threaded capabilities for time sensitive operations that won’t bog down your UI. For developers looking to easily integrate the 2 languages, Transmission holds the key for quick and effective communication between AIR and Java using AMF.

One pitfall I came across a few months ago dealt with the fact that Java applications always show up on the Mac dock, even while using Transmission. This is problematic because starting up an AIR application which starts up Java shows 2 applications running on the Mac dock. This might be confusing for users, plus it gives them the opportunity to kill the Java application which inhibits your AIR application from taking advantage of Java. To resolve this issue I have provided a fix. On the Windows side, you don’t need to worry. Java will only show itself in the windows toolbar if UI code is present. Otherwise it will be hidden by default.

The process to do this on a Mac is to wrap your application’s jars in a Mac OS X application. To do this, we need to use a Utility called Jar Bundler. This should be found here: /Developer/Applications/Utilities or you can pull it up in Spotlight. When you open the application, you should see a screen like this:

Jar Bundler allows jars to be wrapped as an OSX application
Jar Bundler allows jars to be wrapped as an OSX application

Here are the steps you’ll need to take in order to get an application:

1. Click on Choose next to the Main Class option. Next, navigate to Transmission.jar that came with the transmission project and select it. It should populate the Main Class field with transmission.Transmission automatically.

2. Under arguments type: -transmissionControllerPackage transmission.demo.TransmissionDemoController. For your own project replace the TransmissionDemoController package with your own TransmissionController’s package and class name (make sure your own TransmissionController implements ITransmissionController).

3. Select the Classpath and Files tab and choose all other jars that belong to your project besides the Transmission jar you added in step 1.

4. Choose some properties in the Properties tab.

5. Click Create Application and choose a destination for the new application.

Bam! Your application is completed. Now we need to make sure it hides itself when it is run. Right click on your application and choose “Show package contents”. Navigate to Contents and open the Info.plist file. This file contains the preferences you set in the Jar Bundler. To make the application hide we need to add a couple more. Before the Java item, add the following rows

1. LSUIElement (in the value column enter “1”)

2. NSUIElement (in the value column enter “1”)

Your plist should now look something like this: plist

The last thing we need to do is configure the application inside of your Flex Transmission configuration. In your Flex project, rather than doing Transmission.init() using the Java executable, the TransmissionController package, and the array of classpaths, simply do the following:


Now everything should work!


Transmission – AIR 2 to Java Communication Layer

A few months ago I wrote a few posts dealing with how to create a Merapi AIR to Java production environment. If any of you have read those posts or tried doing it yourself you’d realize what a tricky and cumbersome process it is. Now with AIR 2.0, we have native process available which allows us to reduce this workload significantly. To really take advantage of this new functionality, I built an AIR 2.0 to Java communication layer called Transmission. No more custom installers, requiring Java to start AIR, and keep-alive processes to ensure Java is still running.

While architecting this, I decided I wanted to create an environment that would be familiar for people coming from Merapi. There is still a concept of Message objects and MessageHandlers which respond whenever a Message object with its registered type is passed across Transmission. The library is being hosted on Google Code and is licensed under the Apache 2.0 license which gives you the freedom to use it in commercial applications. I included a demo in the repo that should help you get started.

New AIR/Java app Piano Marvel hits the AIR Marketplace!

Over the past year at Rain Interactive, I have spent all my time and energy as project manager and lead developer on the most revolutionary piano teaching software to date called Piano Marvel (not that I’m biased or anything :).  This unveiling also reveals the inspiration behind most of my posts over the last year.  Our team’s goal was to create an application that could display interactive sheet music, record keystrokes on a MIDI keyboard, allow toggling of metronome and accompaniment playback, and give an evaluation based on the notes that were pressed.  The application has been in beta for a few months now and we have been amazed with the feedback we have received from students and teachers across the nation. Continue reading New AIR/Java app Piano Marvel hits the AIR Marketplace!

Add alpha transparency to your DataGrid using StyleableDataGrid

Recently, I ran into another issue with DataGrids.  I received a design comp from one of our designers who wanted alpha transparency in a DataGrid for alternating rows, the header, and highlight/selection.  Basically, he wanted something like this (I added a red gradient and vertical stripes on the background to make the alpha more apparent):


After a quick look through the styles and properties of Flex 3’s DataGrid, I discovered that I wouldn’t be able to do these three requests unless I (once again) subclassed the DataGrid to add these behaviors.

In this component, I added the following styles:

alternatingItemAlphas (complements alternatingItemColors)
headerSeparator (boolean -- draws the column separator on the header)

Note that the alternatingItemAlphas will not appear to work unless DataGrid’s backgroundAlpha style is set to zero.

To see an example with View Source enabled, click here.