Manhattan Works and Sleeps
Inspired by John Nelson’s breathing earth and Conveyal’s aggregate-disser post, I wondered if I could make a breathing city. Manhattan looks somewhat lung-like, so it seemed natural. Should be a fun, quick project. How naive I was.
Search and Recover
Conveyal had already gathered the data I would need to do a dot density plot, so it should be easy to find it using their post as a starting point. But wait they didn’t share links to the source files and they didn’t respond to my email. Google should solve that… hours of surfing later, I find what I'm looking for in four different places: population, employment, land use, building footprints.
Java
Excellent, now just run it all through Conveyal’s conveniently open source tool. Except its written in Java, so lets install the Java SDK. Oh and it has several library dependencies. Finding… installing… finding… (several hours later)... installing... not working. Clearly I know far too little about java to get this going.
Python Wrestling
We use python at Darkhorse, and learning some geographic libraries could be useful. Let’s use the code for the racial dot map project as a starting point for creating a python version of Conveyal’s disser tool. I just need several new libraries which translates to more hours finding, installing, re-installing, searching, uninstalling, removing, copying, installing again because computers are petty and vindictive. Then finally shapely and osgeo are working… yay!
Baby Steps
Now we just take several hours to learn how not to use these libraries and eventually we stumble across one or two things that work, then a couple more, and then we crawl toward a messy program that does what someone else has already done, but at least I speak this one’s language.
Combine CSVs with Shapes
What? Excel stopped allowing you to edit and save DBF files, when did that happen? This tool is a bit buggy, but it brings that feature back.
QGIS forces antialiasing
You can’t turn it off. If you want to create single pixel markers, it just won’t let you color them properly, I tried for far too long (if you know how, please let me know). Good thing Excel is a poor man’s GIS (Yep every one of the frames was made in Excel)
Find More Data
So now I can make a plot that doesn’t breathe. But I want to show change in the typical workday. I’m gonna need more data for that. Several searches and false starts later we find work related activity percentages by time of day. Manhattan probably has a different profile than the US average, but close enough.
Find even more data
But each dot is a person, so I can’t just have them flicking on and off randomly to match the time of day percentages. They need to go to work for a while then come home for a while, so I need to give each dot a schedule. Maybe this is overkill, but I can’t stop now. More searching finds us a rough hours of work distribution.
Solver time
Now I’m forced to assign schedules to the ~1.5 million people living in Manhattan and the ~2 million people working in Manhattan. But the sum of those schedules needs to resemble to hours of work distribution and the percentage at work for each hour of the day. Time to break out Excel’s solver engine. With it we can create ~200 schedules with probabilities to match those profiles. Then we can distribute them to each of of our people.
Data is done
Finally we have data for each of 24 hours for both home and work. We’re making some huge simplifying assumptions (e.g. Manhattan’s work profile is the same as the rest of the US, when people aren’t at work they are at home, there are only 200 possible ways to spend your day, when we build this someone will want to see it) but we have a reasonable data set.
Design
Now we just make some maps and push pixels around on the screen until they look good, then painstakingly create 24 versions to string together for the animation. Eric Fisher has some great tips for making and coloring dot density plots in this post. Then we’ll add a bar chart, no an area chart, no a line chart, wait… I’ve got it, a mesmerizing heart rate monitor looking thingy to go with our breathing theme. Nice.
Relief
See, it is super easy and takes almost no time at all to create something like this, as long as your definitions of “super easy” and “no time” are flexible enough to include difficult and time-consuming.
Note: A previous version of this GIF had the orange work line in the 'heart rate' chart incorrectly shifted one hour. This has since been corrected.