Urban footprints: some building outline data sources

This is an informational post about where to find building outline data, which I've used a lot in previous GIS projects. It might also be of interest to architects, engineers and anyone interested in urban studies and planning more generally. I like using this kind of data to explore cities as it gives us a good idea of the layout of the urban fabric, as in the example below (New Orleans). The links mainly refer to data from the US, Canada and Great Britain but other parts of the world are covered to various extents by OpenStreetMap.

New Orleans

Let's start big, with OSM... Steve Bernard has produced an excellent video which explains how you can get OpenStreetMap data directly into QGIS very simply - he uses Madrid in the example. The accuracy and coverage varies a great deal across the world, so you need to bear this in mind when downloading and using it - but on the whole it is a fantastic resource. The example below shows Mogadishu, where the coverage is incomplete for buildings but pretty good for the road network. 

© OpenStreetMap contributors

Another useful OSM-related resource with decent global city coverage is CAD Mapper, where you can download areas up to 1km square for free. However, I'm focusing on open data today so will not go into detail on this. The best OSM download source is I think GEOFABRIK (German for 'geo factory), a German GIS consultancy who extract and process OSM data and then make it available for free online. It's really nicely structured and easy to find what you're looking for. Here's the download page for New Zealand, for example - followed by the contents, where you can see the building data on top of a current OSM base map. At time of writing, the zipped shp folder for the whole of New Zealand was 146MB.

The New Zealand GEOFABRIK download page (20 July 2015)

Auckland, NZ - very good building coverage here

The OSM sources are great, since the licence is very generous and you can use the data for just about anything, so long as it's properly cited. However, many towns, cities and counties across the world also provide building footprint or outline data (the terminology varies from place to place) so I've put together a list below of ones I know about. Some of them (e.g. Detroit, NYC) cover land parcels or tax lots so are slightly different but in the main it's just building outlines. I've included visuals for some of the datasets, so you can get an idea of what they look like.

New York City - from the BYTES of the BIG APPLE website you can download the MapPLUTO dataset, for all 5 Boroughs in New York City. Tax lot level rather than building outlines, but it's an extremely rich dataset with loads of useful land use planning variables in it, including 'year built' and number of floors. A little sample of the data are shown below, for the area around Central Park.

A little sample of the data (using Qgis2threejs)

Chicago - the building footprints layer is avaiable in two versions online, one of which says it is deprecated but I've heard from the Chicago GIS team that this isn't the case. It's just that due to limited staff the dataset is only edited when necessary. Also contains a 'year built' and height variable.

San Francisco - another really good city buildings dataset, from SF OpenData. Also lots of useful variables in this dataset, including height. I really like this one.

Dallas - you'll probably get a disclaimer box in a pop-up when you go to download this. I've linked to the general GIS page and the file you want is called Structures (Building Footprints) in the Planimetric Data section - it's about 81MB to download and the unzipped file is well over 100MB.

Atlanta - again, I've linked to the GIS page, this time from the City of Atlanta and you need to download the 'Impervious Buildings' layer. If you're looking to map the sprawl of Atlanta, this won't work as it covers the City area only. Still, a very useful dataset.

Denver - excellent open data from Denver. This dataset covers all permanent structures and buildings for a 152 square mile area of the City and County of Denver. Available in a number of different formats.

Seattle - this dataset was created in 2009 by Pictometry International Corp but is now in the public domain. It is available via the City of Seattle's data website.

Los Angeles - this is a fantastic dataset for the County (not just the City) of Los Angeles, which is the most populous county in the United States (just over 10 million). Made available via the LA County GIS Data Portal. It is a little hefty (581MB) so be careful! In the example below I show all the buildings in LA County but the City of Los Angeles in dark shading, just to emphasise its crazy shape.

Boston - this was created in 2012 and is available via the City of Boston. Contains a number of different fields, including base elevation of the structures, the elevation of the highest point above sea level and fields on building type.

Detroit - like New York, not strictly a buildings outline file but instead a property lot level dataset. Very impressive dataset produced by Data Driven Detroit's Motor City Mapping project. I've used this data a lot in talks and teaching as it's a really good example of its type.

Now some links to further datasets which I know of but haven't used that much...

Washington DC - link is to the download page, but direct link to zip is here (559MB unzipped)

Baltimore - the top link on this page

Philadelphia - via OpenDataPhilly

Massachusetts - buildings for a wide range of towns and cities in the state

Boulder - this is from Boulder County, Colorado. Available in a number of different file formats.

Bloomington, Indiana - one of many smaller cities with excellent geodata

New Orleans - an excellent dataset, not just because of the unusual shape of the city!

Toronto - don't be confused by the '3D Massing' terminology here. Scroll down to the 'Data download' section

Vancouver - doesn't cover the whole city and they were digitised in 1999 but still a useful dataset.

Waterloo - this is from the Region of Waterloo and was up to date as of January 2014.

Hobart, Tasmania - an nice example of building data from Hobart in Australia. Contains a 'year constructed' variable.

Wellington, NZ - can't overlook New Zealand! I think you need to register to download this but it's Creative Commons 3 so still open. 

The list wouldn't be complete without mentioning OS OpenData for Great Britain, provided by Ordnance Survey. A new dataset with detailed buildings became available in March (the OS Open Map - Local) dataset. The building data is a very small part of this collection but one I find very interesting. I've patched together a few cities here to get the ball rolling but you can download your own. There's also a 'tile finder' to help you identify which OS tile you need to cover your area of interest. 

This could save you some time 

I think this just about covers it. Get in touch if you have any other great data sources for building outlines.

WIMD 2014 Shapefiles and Maps

The Welsh Index of Multiple Deprivation 2014 was published on 26 November, using more up to date and improved indicators. The Welsh Government have provided some nice interactive mapping, but James Trimble's version is I think even better. I've done basic interactive versions in the past, but today I just want to share the raw GIS data and a few maps, for anyone who is interested in either looking more closely at their area or doing a bit of spatial analysis themselves. First of all, here are a few basic WIMD maps, clipped to building outlines (click images to enlarge).

I made these in QGIS, using an automated atlas production technique I've described on the blog before. If you are looking to produce some WIMD maps, you might want to try this method. I've also produced some WIMD 2014 maps in standard choropleth format, as you can see below. These are okay in areas which are densely populated but I find them more misleading for larger rural areas where there are not many people.

I know a lot of people across the public sector in particular will be looking closely at the data, so I thought it would be helpful to make the underlying shapefiles publicly available since the data are open. If you click on the link below you'll be taken to a download page via Google Drive. Any questions, then feel free to get in touch.

Download files: WIMD 2014 Shapefiles - for LSOAs and LSOAs clipped to building outlines.

Some tips for charts in Fusion Tables info windows

I've recently been working with some mortgage lending data from banks in Great Britain to produce a new mortgage lending maps site. Once again, I've used Google Fusion Tables to map the data because it's relatively quick and easy - further information can be found here. What's not so easy is getting the info windows to do exactly what you want, particularly when you want to include charts of the type shown below. In this post, I explain a little more about how you can get the info window to display such a chart, what can go wrong when trying to do so and what the code underlying code looks like.

The chart shown above is what you'll see when you click on any polygon in my mortgage lending map website. It takes the data from the underlying Fusion Table and provides a unique info window and chart for each postcode sector - in the case of the above it is for the Cardiff postcode sector of CF24 4, which as you can see had more than £117 million of outstanding mortgage debt owed by 839 households at the end of June 2013. The default info windows created in Fusion Table maps simply contain a number of default data records from columns from the underlying table. With this data, I was keen to show a visual comparison of lending mix in each area, and I wanted to do this using a horizontal bar chart. I'd done info window charts before (e.g. in my Deprivation in Scotland website) but these were line graphs showing change over time.

There is some general help on putting charts in info windows from Google, and this is a very good place to start, and you can also find a lot of explanation for what the different bits of chart code mean in Google's Charts Gallery, or in the Chart Feature List, but to help anyone who might be trying something similar to what I've produced, I thought I would provide an annotated code example. I also want to provide some general troubleshooting advice. Here's what the code looks like inside the Fusion Tables interface:

And here is a Word document with comments added explaining what each bit of code does. You'll notice that it's a bit messy but it produces a very nice looking chart. One thing that I noticed about all this is that if you want the numerical data to appear in the info window with comma separators - as above, for the £117 million figure - then it appears to disappear from the chart. This is what happened to me anyway. My advice would be to keep the number format for your data as 'none' in the Fusion Table field options. My other advice would be to remember to save a good version of your code but most of all to experiment with different things and then share the results.

I hope someone finds this useful! I know I would have before I got into this. 

The Age of Buildings in the City of Chicago

Following my last post, on the geography of New York City, I've been exploring other building-level datasets to see what they can offer us in relation to telling us more about the fabric of the cities we live in. This time, I've focused on Chicago's 'Building Footprints' dataset. It's not nearly as detailed as New York's PLUTO data but it does include variables on (e.g.) number of floors and year built. As with the NYC data, it is not perfect but we can still make good use of it to understand the development of the city and its structure. I've mapped the city using number of floors as a proxy for height and shaded it by building age to produce the following overview (blues = older buildings, reds = newer).

Even bigger version

Besides looking relatively interesting, the above graphic also reveals something about the phased construction of the City of Chicago and - possibly - something more about the data itself. As with the New York City PLUTO data, I produced a chart of the 'year built' column just to give me some idea of its distribution. It looks better than the New York City chart but I'm still not convinced it is 100% accurate (the year built data run from 1852 to 2010). 

Were there really nearly 15,000 buildings constructed in 2006 and only 78 in 2000? Possibly, but it would be good to know more about the accuracy of the data. In total there are 820,154 building footprints in the dataset and there are a range of different columns - which you can read more about in the metadata file. Once again, it's pretty cumbersome to work with in a normal desktop GIS setting but my machine can just about handle it. 

HS2 geodata - for download

Yesterday I wrote a short post on the Guardian's Datablog about my difficulties getting hold of the route data for the proposed routes for the new high speed rail lines in England. Coincidentally (or maybe not) HS2 responded to my request at almost exactly the same time as the piece appeared online. Anyway, sometimes it does take time for public bodies to respond to requests so my real question was why the shapefiles were not available for download, given that they are available under the Open Government Licence. I have a few ideas about why this must be but it would be good to have some information from this on HS2, though maybe they're too busy with other things! Clarity on this issue might, however, reduce the likelihood of data conspiracy theories and enhance transparency.

Anyway, enough about that. So that other people don't have the same wait as me to get hold of the GIS data I've made them available here via the link below. A few important points to bear in mind...

1. The Phase 1 (London to West Midlands) route is the 'post-consultation' route from January 2012.

2. The Phase 2 (Leeds and Manchester) routes are the 'initial preferred routes' from January 2013.

3. There is an interactive map of the Phase 1 route on the HS2 web pages, which is quite useful.

4. Users of the data need to remember to acknowledge the source.

5. It's not my data - I'm just making it available.

6. You can also get these files from Barry Cornelius, but not - as yet - from data.gov.uk

7. The route data available for download here doesn't necessarily reflect the precise location of where the train lines will be built - particularly for Phase 2. 

HS2 shapefiles, as of 2 May 2013

European Geodata - NUTS, etc.

Need to find digital boundaries for the European Union? Tried googling but no success? They are of course available via UKBorders in the UK but if you can't get them this way then you can find them via search but it's not that easy.

The first thing you need to do is go to the Eurostat Geodata pages. Then from the menu on the left you can select 'Reference' or 'Archive' to get to the download section. Once you're there you just need to click on the kind of data you want (e.g. 'Administrative units/statistical units') and you are taken to the download page where there are lots of links...

And that's it.

Some interesting datasets, including:

• Corine Land Cover
• Urban Morphological Zones
• Degree of urbanisation
• NUTS boundaries (down to NUTS3)
• Elevation
• Settlement names

British National Grid and Google Maps

This post follows on from a recent one about the Scottish Index of Multiple Deprivation 2009 and in particular the interactive mapping function they provide.

The most deprived area in Scotland, according to the new Index, just happens to be centred on Celtic Park, Glasgow. This is, in technical terms, Data Zone S01003279. But, there's a slight problem - the Data Zones are somewhat displaced. It's not too big an issue in relation to the SIMD data because Data Zones match Ordnance Survey data and the Scottish Government have all this behind the scenes so they do of course know on a street-by-street basis where things are.

However, for the online map interface, it does mean that the Data Zone is not in the right place. It also means that people using the interactive mapping facility could end up thinking they live in one area, when they actually live in another since the level of displacement often moves Data Zones into the wrong street.

So, it got me thinking. What would you have to do to get the Data Zones in the right place on Google Maps (this also applies to Google Earth)? It's quite simple really (if you're a nerd and you have the right software)... N.B. the SIMD data is not based on Google Maps, but rather the Microsoft Virtual Earth mapping facility.

1. Convert the Data Zone Shapefile from British National Grid to WGS84 in ArcGIS. Best done via ArcToolbox/Data Management Tools/Projections and Transformations/Feature/Project and then selecting the right files for transformation to the correct projection. More useful information can be found here.
   
2. Use Kevin Martin's excellent Export to KML tool to convert the Shapefile to KML. You can set transparency here and it will export as a transparent KML layer, as in the maps below.
   
3. If you've got a Google account you can then upload the KML file directly into the My Maps facility and the Data Zone appears there. From there you can add a description and edit in other ways.
   

Here's what it looks like if you don't get the projection right the first time (click the link below the map to see the full thing) - note how the edges of the Data Zone don't match the streets.


View Data Zone S01003279 - Displaced in a larger map

And here's how it looks if you convert to the correct projection before you export to KML (again, click the link below the map to see it full size):

View S01003279 in a larger map

So, a simple bit of GIS work before putting all this online solves the problem.