2011-07-13 The Tale of Two Visualizations


Today F-Secure's Mikko Hyppönen stepped on the stage of TEDGlobal 2011 conference to give talk (see the video) about the next computer virus assisted end of the world. Judging by the Twitter response the talk was well received. No surprises there - Mikko has received the "The Best Educator in the Industry" award, given every ten years.

Jani has since a long time had this silly dream of being a part of the TED phenomenon, so he contacted Mikko and offered my lifeblood our expertise to create a visualization to provide some whiz-bang for the talk. Mikko was intrigued. At the time none of us had any idea what the visualization should be ABOUT, but what the hey. Last friday we finally started actually creating something visible. As luck would have it, Juhani Eronen from CERT-FI had some interesting anonymized data produced with AbuseHelper, yearning for visual flair: March-June 2011 data about network abuse, such as phishing, spamming, malware activity and so on, directed to and from Finland. This data was collected from various sources, and from it we could infer approximately when the incidents were finally solved.

A Python Walked into a Bar... Chart

Our first idea was to create a map visualization. As always. The animation was supposed to show how incidents were popping around the world and how fast they were taken care of, somehow. On the other hand at least Jani and I have been talking a lot how map visualizations are, like, so 2010. In this case especially we felt that throwing the stuff on the map wouldn't provide added value, as the geographical incident distribution was pretty limited. We wanted something fresh and new! Like bar charts!

Indeed, the resulting visualization, created with Python and ?PyQt4, is a pretty basic bar chart where the horizontal axis represents time (the beginning of March in the left, the end of June in the right) and the vertical axis represents the count of new incidents that appeared to do something nasty at that point of time. One bar is about two days, one line through the the vertical axis is about 1000 incidents. Now, when the animation starts going, you can see how unhandled incidents (red color) are detected and then turning into handled ones (grey). In the end we also show the cumulative amount of work still left at each point of time. Sort of "incident debt", if you will.

The original, higher definition video can be downloaded here.

The process of creating the video actually went pretty smoothly. It's probably the first time ever when a visualization turned out exactly like I first imagined in my head. But, alas, the above video isn't quite finished, missing things like labels and legends. That's because...

2011 Is the New 2009

On monday we presented the bar chart video to Mikko. Turns out that there were two factors that made the animation unsuitable: It would probably take some time to explain what's actually happening, what with the the weird double time dimensions and the sudden shift to cumulative charting. The other, arguably worse, problem was that it had nothing to do with Mikko's talk. Bugger.

So we sent Mikko a link to this old visualization that had been hanging around unpublished since 2009. This "ping pong view", created from data provided by Hillar Aarelaid from CERT-EE, depicts internet criminals moving their servers around the world and from jurisdiction to jurisdiction when they are about to get caught. Simple, fun, effective, and immediately illuminating even if you don't know the nitty gritty technical details. Mikko liked it, Jani digged up the original data, and I produced a remastered 720p version. BEHOLD:

Check out the original HD video here.

Apparently Mikko showed this during his talk. I hope. Otherwise this blog entry is just sad.

Now What?

We actually liked the bar chart, and have gotten some great feedback about it. It certainly needs more work, but its double time aspect shows the changing situation in a nice way. You can divine some interesting stuff from it (left as an exercise for the reader or Jani), especially if you consider that Finland is supposedly "mostly harmless" in the global ranking of network doomness.

This was also a great reminder for us how to keep it simple. In this case a map was a perfectly fine option, it tells a story and reinforces the point Mikko was making. That's all we could ask for, really.

-- jviide 2011-07-13 21:13:56

Media Tracking

return to the blog ...