2009-01-01 18-15 Flashy Botnet is Flashy

Update (2009-05-06): We have released the tool, Logster, we used for creating this visualization! Check it out at http://www.clarifiednetworks.com/logster

Update (2009-01-09): Updated the downloadable higher resolution (640x370) video, it should now work better with several video players. Get the new version here. Also added an even higher resolution (1280x720) version, available here.


Some time ago fellows from F-Secure collected a bunch of neat log data on botnet IRC channel joins. They then asked us to visualize the joins on a world map, much akin to what we did with the Kaminsky DNS patching logs. We gleefully agreed.

So, this is the result:

(or download the higher resolution (640x370) version here.)

People have asked us how those animations are done. As usual, we used our beloved Python and PyQt for log parsing, drawing and generally putting the whole thing together. We also happened to have a Maxmind's GeoIP package with Python bindings around, so geolocating most logged IP addresses was a breeze. NASA's Visible Earth had nice world maps free for use, so we didn't have to launch one of Clarified's own satellites into the orbit for snapping pictures. Thanks, NASA.

Then we just draw some colors on the map and flash them around.

Of course, the devil is in the details. What we really draw on the maps are these kinds of smooth blobs: blob.png. The first attempt was to just figure out coordinates for each activated IP address and draw a blob with a suitable color and opacity describing the IP's status. Here's an early test of what it looked like:

map1.png

Ok, but not too pretty. The map underneath gets fudged and some most active areas get a bit too crude edges, as many blobs get drawn over each other. A quick fix was to sort of "quantize" the pixel coordinates of each IP, say to the closest even integer. Then we collect all addresses with the same quantized coordinates into just one drawn blob. The hardest part was to figure out a combined color and opacity for each blob from the related addresses, but some hand tuned weighted average magic seemed to be fine. The result was better control of the saturation and smoother blobbage even in crowded areas:

map2.png

It might look less dramatic and flashy than the previous image, but this method yields much more pleasant animations. The animation is done by decaying each IP's respective color and opacity over time, and taking this into account when quantizing the blobs. When an IP is seen active again, its color and opacity is renewed back to max. Now just add a small flash for each IP activation, and there you have it. Of course there'll still be lots of fiddling with different parameters to make the whole thing nice.

Enter the shameless plug: Now you don't have to do all the fiddling yourself, as we have productized the code into a package called Logster. Find out more from our Logster page.

-- ?jvi 2009-01-01 19:16:16


return to the blog ...