The Crimea Conflict and Airline Tracks

How does military conflict affect commercial airline flight paths? Below are the daily flight heat maps around Ukraine from late March to early April. Each frame is an aggregate of all flights for a particular day. The darker the color, the more flights went there that day. On 3/31, you see a lot of flights circled a city in Crimea (maybe Simferopol?). The next day the airlines started steering their flights around Crimea (except some flights coming from Russia).

#+CAPTION: Flight patterns over Crimea changed after the conflict broke out

#+NAME: fig:crimeaconflict

I know others have plotted this before, but it's been fun writing something to walk through the data on my own. Python/Matplotlib was taking forever, so I wrote something in Go that allowed me to clean up the original data (eg, split dateline-crossing flights) and filter it down to regions of interest. I still plot with Matplotlib, but now that the data is simplified it takes a lot less time to do.

Parsing Flight Data

This week at home I've been parsing through some flight data I acquired, looking for a good way to generate heat maps for where planes fly. I thought of a few elaborate ways to do this, but in the end, I just ran a chunk of 10k flights through a simple xy plotter in matplotlib using a low alpha (transparency) value. It's interesting to see the flight corridors emerge as you throw more data at it. The below is about half of the data for a single day.

Code

I've now posted the plotting code in github, under my airline-plotters project: github:airline-plotters

Finding Offensive Terms in Ebooks

he downside of reading expired-copyright ebooks is that many of them use offensive, racist terms. I first noticed this a few weeks ago when I downloaded Tom Swift and His Motor Cycle to read to my son. I'd heard of the Swift books and the premise sounded fun (young inventor has adventures and fixes anything with mechanical problems). Unfortunately, a short way into it Tom meets a bumbling man named Eradicate, who equates technology with magic and speaks in broken English. It's pretty awful stuff and has a lot of racist terms in it (I wound up doing a lot of filtering and correction while reading it to my son). In the edit logs of Wikipedia's Tom Swift page, I was also surprised to see that a lot of the "this is offensive" comments were struck down by a moderator's "show me proof" comments.

Anyways, I was wanted to see how other books in Project Gutenberg have offensive terms. I downloaded the 2010 iso with 29k books and then went about writing a go program to count offensive terms and score each book (1 point for each "could be" term, 100 points for each "definitely" term). The code flagged some 5900 books, though many of those innocently use terms that are commandeered by racists. Below is a plot, where each square is a book in the collection (grey is zero, dark is low score, light is high score).

While there are plenty of books that use the terms for a good purpose (eg, Huck Finn), I suspect many of them just use the terms because that's just how society was a hundred years ago, unfortunately. I think I'll be doing some more greps before I pick another bed time story.

Code

People asked me for the code for this, so I posted it on Github. You can find it (and documentation) in my gut-buster project: github:gut-buster

CT Scan

After goofing off with Slicer and a CT scan I had last year, I've come to the conclusion that my head would make an excellent bread bowl.

Adobe's Leaked Passwords

One can spend an endless amount of time slicing the adobe data up. Here's a list of the most common gov addresses in it. Team NASA for the win. It'd be interesting to see how corporate policies affect how often employees wind up registering with a site like adobe.