Urban footprints: some building outline data sources

This is an informational post about where to find building outline data, which I've used a lot in previous GIS projects. It might also be of interest to architects, engineers and anyone interested in urban studies and planning more generally. I like using this kind of data to explore cities as it gives us a good idea of the layout of the urban fabric, as in the example below (New Orleans). The links mainly refer to data from the US, Canada and Great Britain but other parts of the world are covered to various extents by OpenStreetMap.

New Orleans

Let's start big, with OSM... Steve Bernard has produced an excellent video which explains how you can get OpenStreetMap data directly into QGIS very simply - he uses Madrid in the example. The accuracy and coverage varies a great deal across the world, so you need to bear this in mind when downloading and using it - but on the whole it is a fantastic resource. The example below shows Mogadishu, where the coverage is incomplete for buildings but pretty good for the road network. 

© OpenStreetMap contributors

Another useful OSM-related resource with decent global city coverage is CAD Mapper, where you can download areas up to 1km square for free. However, I'm focusing on open data today so will not go into detail on this. The best OSM download source is I think GEOFABRIK (German for 'geo factory), a German GIS consultancy who extract and process OSM data and then make it available for free online. It's really nicely structured and easy to find what you're looking for. Here's the download page for New Zealand, for example - followed by the contents, where you can see the building data on top of a current OSM base map. At time of writing, the zipped shp folder for the whole of New Zealand was 146MB.

The New Zealand GEOFABRIK download page (20 July 2015)

Auckland, NZ - very good building coverage here

The OSM sources are great, since the licence is very generous and you can use the data for just about anything, so long as it's properly cited. However, many towns, cities and counties across the world also provide building footprint or outline data (the terminology varies from place to place) so I've put together a list below of ones I know about. Some of them (e.g. Detroit, NYC) cover land parcels or tax lots so are slightly different but in the main it's just building outlines. I've included visuals for some of the datasets, so you can get an idea of what they look like.

New York City - from the BYTES of the BIG APPLE website you can download the MapPLUTO dataset, for all 5 Boroughs in New York City. Tax lot level rather than building outlines, but it's an extremely rich dataset with loads of useful land use planning variables in it, including 'year built' and number of floors. A little sample of the data are shown below, for the area around Central Park.

A little sample of the data (using Qgis2threejs)

Chicago - the building footprints layer is avaiable in two versions online, one of which says it is deprecated but I've heard from the Chicago GIS team that this isn't the case. It's just that due to limited staff the dataset is only edited when necessary. Also contains a 'year built' and height variable.

San Francisco - another really good city buildings dataset, from SF OpenData. Also lots of useful variables in this dataset, including height. I really like this one.

Dallas - you'll probably get a disclaimer box in a pop-up when you go to download this. I've linked to the general GIS page and the file you want is called Structures (Building Footprints) in the Planimetric Data section - it's about 81MB to download and the unzipped file is well over 100MB.

Atlanta - again, I've linked to the GIS page, this time from the City of Atlanta and you need to download the 'Impervious Buildings' layer. If you're looking to map the sprawl of Atlanta, this won't work as it covers the City area only. Still, a very useful dataset.

Denver - excellent open data from Denver. This dataset covers all permanent structures and buildings for a 152 square mile area of the City and County of Denver. Available in a number of different formats.

Seattle - this dataset was created in 2009 by Pictometry International Corp but is now in the public domain. It is available via the City of Seattle's data website.

Los Angeles - this is a fantastic dataset for the County (not just the City) of Los Angeles, which is the most populous county in the United States (just over 10 million). Made available via the LA County GIS Data Portal. It is a little hefty (581MB) so be careful! In the example below I show all the buildings in LA County but the City of Los Angeles in dark shading, just to emphasise its crazy shape.

Boston - this was created in 2012 and is available via the City of Boston. Contains a number of different fields, including base elevation of the structures, the elevation of the highest point above sea level and fields on building type.

Detroit - like New York, not strictly a buildings outline file but instead a property lot level dataset. Very impressive dataset produced by Data Driven Detroit's Motor City Mapping project. I've used this data a lot in talks and teaching as it's a really good example of its type.

Now some links to further datasets which I know of but haven't used that much...

Washington DC - link is to the download page, but direct link to zip is here (559MB unzipped)

Baltimore - the top link on this page

Philadelphia - via OpenDataPhilly

Massachusetts - buildings for a wide range of towns and cities in the state

Boulder - this is from Boulder County, Colorado. Available in a number of different file formats.

Bloomington, Indiana - one of many smaller cities with excellent geodata

New Orleans - an excellent dataset, not just because of the unusual shape of the city!

Toronto - don't be confused by the '3D Massing' terminology here. Scroll down to the 'Data download' section

Vancouver - doesn't cover the whole city and they were digitised in 1999 but still a useful dataset.

Waterloo - this is from the Region of Waterloo and was up to date as of January 2014.

Hobart, Tasmania - an nice example of building data from Hobart in Australia. Contains a 'year constructed' variable.

Wellington, NZ - can't overlook New Zealand! I think you need to register to download this but it's Creative Commons 3 so still open. 

The list wouldn't be complete without mentioning OS OpenData for Great Britain, provided by Ordnance Survey. A new dataset with detailed buildings became available in March (the OS Open Map - Local) dataset. The building data is a very small part of this collection but one I find very interesting. I've patched together a few cities here to get the ball rolling but you can download your own. There's also a 'tile finder' to help you identify which OS tile you need to cover your area of interest. 

This could save you some time 

I think this just about covers it. Get in touch if you have any other great data sources for building outlines.

The Era of Radical Concrete meets Twitter

Back in 2008, when I moved into my office at the University of Sheffield, there were three mashed up boxes sitting under the desk. My curiosity got the better of me and I spent the next few hours looking through thousands of slides in hanging folders. This turned out to be the image archive of Jimmy James, the first Professor of Planning in our Department and former Chief Planner in England.

In the summer of 2013, after securing funding from the University of Sheffield Alumni Fund, I was able to hire Joe Carr and Philip Brown (recent graduates) to digitise the entire collection and upload it to our dedicated JR James Archive Flickr site. After more than 2 million views and a year online, the BBC magazine did a piece on the archive, called 'The era of radical concrete' which created huge interest and led to a follow up from residents of the places featured in the images. But one image in particular puzzled us... It was simply labelled 'Housing Scotland' (see below for the original slide and the online scanned image)  and we just couldn't fine an exact location for it, but we thought it was in the New Town of Cumbernauld, 15 miles or so from Glasgow.

Note the 'MAY 67' date stamp

Are you the boy? Let us know!

These images were used in teaching planners and urbanists for many years but some of them were not in the right place when we found the slides. We did well to track down the location of most images since between me, Philip and Joe we have a pretty good knowledge of most of the UK and we had thought the one above was of Cumbernauld, because that's where it appeared in the archive.

The New Towns section of the original archive

However, our interest was piqued when a commenter on Flickr said it definitely wasn't Cumbernauld. We tried a few times on Twitter, but to no avail. Even @MunicipalDreams couldn't help us. So, I tried one last time yesterday on Twitter and after 17 retweets we had suggestions that it was definitley NOT South Africa (!), but it might be 'somewhere in Cumbernauld', Falkirk, Dyce, Cwmbran, Killingworth, Peterlee or Saltcoats - among other places. I spent half the night looking round Craigavon, Crawley and Harlow, just to be sure. Nothing.

But this is where Twitter and the power of the crowd came to the fore. Actually, I should say this is when the determination of Michael Coates came to the fore, because ... wait for it ... he found where the photo was taken. It is in fact Abbots View in Haddington, East Lothian - as you can see from the embedded Street View image below. I've shown the 2008 Street View here as it is a better match for location and light but this is definitely it. The perspective of the Google cameras is a little different from Jimmy's original lens but it's a clear match so I declare Michael Coates winner of the Internet today and forever, though the Visual Resources Centre at Manchester Metropolitan University also deserve credit for joining the hunt.

So, that's one mystery solved. But what about the other mystery? Who is the boy on the bike?

By my estimation he'd be about 50 now and if anyone tracks him down we'll really be impressed. In the meantime, if you haven't already looked at the JR James Acrhive, what are you waiting for?

Some thoughts on mapping spatial patterns of deprivation

In my research into the geography of deprivation across the UK, I frequently use maps to illustrate the spatial patterns associated with the areas identified as 'least' or 'most' deprived according to official indices such as the Scottish Index of Multiple Deprivation or the English Indices of Deprivation. Lots of other people do similar kinds of things, including mapping gurus such as Oliver O'Brien from UCL (his work is much nicer). A recent example is shown below, which I also tweeted this week. It's difficult to know exactly how people will interpret such maps, particularly when they are only seeing them on twitter without much in the way of context being provided, so this short blog fills some of the gaps and discusses some wider issues.

Full size version

In previous academic papers (e.g. Urban Studies, 2009; Regional Studies, 2012; Local Economy, 2012) I've written about deprivation quite a bit, and on the need for the debate to centre not just on 'deprived' areas but more widely upon the wider dynamic of socio-spatial inequality. It's a shame that the focus is still very much on 'poor' or 'deprived' areas so in an attempt to draw attention to the urban inequalities which exist across the UK I attempt to illustrate the socio-spatial disparities within different cities. I also did this in a report on Sheffield from 2011 where I tried to draw attention to the socio-spatial inequalities within English cities, as shown below. It's not that concentrated deprivation isn't a problem (far from it) but rather that it's part of something much bigger.

Full size version

These kinds of maps do draw attention to the general issue but of course they can lead to all sorts of other conclusions and claims because as we know, maps are an abstraction from reality and they do not represent an absolute 'truth'. These maps simply colour small areas within cities according to how they are classified by a government metric which attempts to say how 'deprived' places are. This may be a dubious practice in many respects, but it is woven into the fabric of how places are understood in a policy context and how problems are defined. It's important that we understand what this kind of mapping allows us to say and what it does not. Some of this is covered on the 'What does it all mean' tab of my Scottish Index of Multiple Deprivation 2012 map site, but I want to make a few more points here...

1. Colours. They are not intended to match up to any political party but some people inevitably make such inferences. The maps say nothing about the causes of the patterns, or who is responsible for them. But it doesn't stop people from talking about it and that's no bad thing. There is a lot more that could be said about colour choice but I'm going to leave it there for now.

2. The trouble with choropleth maps. Maps shaded according to some value (such as deprivation rank) present a misleading picture in a number of ways but two important ones stand out here: a) not all people in the area are 'deprived' or 'non-deprived'. This is the classic 'ecological fallacy' issue at work - the third paragraph here says more about that; and b) the shaded zoned themselves cover much wider areas than people actually live in so a big blue or red area gives the impression of a lot of something, when in fact the population of larger spatial units is similar to the smaller ones (as it often is with LSOAs or Data Zones in the examples above). 

3. The sometimes arbitrary nature of local authority boundaries. Places like Leeds are often said to be 'overbounded', whereas Manchester is 'underbounded'. This means that the local authority boundary either extends beyond the core urban area or it doesn't include much of it at all. So, in the cases of Manchester and Liverpool above if you were to extend the boundary of the map you would see more areas that are not so deprived. However, the point here is that local authorities have to deal with the financial, social and spatial implications of these patterns. What happens beyond the boundary is not part of their remit - even if it does impact upon them. The boundaries may be arbitrary in some respects but they have very real implications.

4. Why not take a different approach? A good idea, and one that Oliver O'Brien in particular has been very successful with. If we only look at where people actually live then we get a more realistic (but still not 100% accurate) picture of the spatial patterns associated with deprivation. This can be done using dasymetric mapping, where we assign the attributes of areas to individual features. This isn't a perfect definition, and the technique itself can lead people to assume a higher level of accuracy than can actually be obtained from the underlying data but it has advantages over standard choropleth maps in relation to depicting the places where people actually live or work. See also Neal Hudson's London tenure map in this style. The new OS Open Data VectorMap District buildings layer for Great Britain allows us to do this, so I've produced an example map for Glasgow based on the one in the first image above. This time you can only see the areas where there are buildings (though many are not residential properties).

Full size map

Is the map above more 'truthful' than the normal choropleth? Probably not. However, this is all irrelevant if we aren't concentrating on the underlying patterns we're trying to draw attention to in the first place. The point is that in cities like Liverpool, Manchester and Glasgow the high levels of deprivation/poverty/disadvantage sit in stark contrast to areas at the other end of the scale. Also, places that we think of as 'deprived' are often far from it - as Peter Matthews might also argue. It's this kind of inequality which I'm attempting to highlight with my mapping - though I do of course like a nice looking map (I've also produced more than a few stinkers in my time). The point of all this? I hope that these maps can start a conversation about the underlying issue. I'll end with an extract from my 2012 Regional Studies paper on the issue...

The Highlands and London: lots of space vs. lots of people

I've been doing some work recently looking at local authority data from across the UK. One thing that always fascinates me about this is the differences between areas at a very basic level, such as the different sizes and populations of local authority and other sub-national areas. The two examples which always stand out in my mind are the Highland council area (because that's where I'm from) and Greater London (because it's so big and dynamic). The other reason the Highland council area stands out is because it is by far the biggest local authority in the UK. It's bigger than both Wales and Northern Ireland, yet it only has 232,000 people. London, on the other hand, would fit into the Highland council area about 20 times over yet it has 8.2 million people - as you can see from the map (click to enlarge - and see full screen here).

The map above shows Greater London and the Highland Council area mapped to the same scale, with London moved to the middle of the Highlands (not expecting this to become official Scottish Government policy any time soon, but you never know). Greater London is about 40 miles across but it is totally dwarfed by the Highlands. What is the relevance of all this? Well, it is interesting to think about this in the context of local government and the challenges they currently face.

Within Greater London you have 32 Boroughs plus the City of London all performing a variety of functions and in the whole of Scotland there are 32 Council Areas (which esteemed blogger Peter Matthews can list alphabetically off the top of his head). In the context of the ongoing fiscal crisis, there has been some talk of merging all local authorities in Scotland. I doubt it is going to happen but you never know. A Scotland on Sunday article on it from late 2012 seems to suggest there is some truth to the idea.

SIMD2012 - An Interactive Website

With the release of the latest version of the Scottish Index of Multiple Deprivation I thought I would take the time to put together an interactive mapping website so that people who are interested in exploring spatial patterns of deprivation could easily interact with the data. The official Scottish Government interactive mapping site has some nice features but I find it a bit cumbersome and the map interface is too small for my liking so that's why I've produced my own version, based on Google Fusion Tables.

Putting this together has prompted me to develop some additional mapping tools using Fusion Tables code and these can be accessed via the 'Tools' tab on the new website. The 'search and zoom' allows you to enter a place you want to look for and when you hit 'Search' the map immediately pans and zooms to that location.   The other tool I've created simply lets you turn the SIMD map layer on and off, which is quite a useful feature.

I've just looked at the relative ranks of places within Scotland in this site. For details of absolute change you can see the employment and income domain data available from the new SIMD 2012 website.

Employment Deprivation in England and Scotland

I'm doing a talk today at the Employment Research Institute in Edinburgh, so I thought I'd put my presentation here and also post a few nuggets of information. Employment deprivation is the subject of the talk and I've been exploring spatial patterns in relation to this. What is 'employment deprivation'? It's involuntary exclusion from the labour force.

A few bits of information... Out of the different indices of deprivation for England and Scotland between 2004 and 2010, the highest figure for employment deprivation was in a LSOA in Rochdale, with 75% of people out of work. The highest figure in Scotland was a Data Zone in Glasgow in 2004 and one in Edinburgh in 2006 with 65% of working age people not in work. There are no major surprises in the general spatial pattens but there are interesting findings in the spatial analysis in the presentation (well, that's what I think).

Further Fusion Experiments

A short post today to report on some further experimentation with google maps, fusion tables and Scottish deprivation data. I've been exploring patterns of deprivation for a talk I'm giving at the Employment Research Institute in Edinburgh this coming week. Some results follow but in the examples below, the red areas are most deprived, yellow next most and blue less deprived. First of all we have an extract from Wick...

Then one from Inverness...

And finally one from Campbeltown...

The point here is not really to map deprivation data, though of course that's what I've been doing. Instead, it is to test the capabilities of fusion tables. More specifically, I've tried to make the data from the Scottish Index of Multiple Deprivation from 2004, 2006 and 2009 appear in a chart when you click on an area - and I have been able to get it to work. When you click on any area in any of the maps above (best to use a full screen version), you'll see a chart in the info window that pops up, like this:

Information on how to do this can be found here. It's not that difficult but it does take a while to figure out. My view is that fusion tables are a very powerful way of visualising data and making it easily accessible online. Still in its infancy really but worth a look.

Deprivation and Map Distortion (SIMD 2009)

I'm doing a seminar at the Employment Research Institute in Edinburgh later this month on the topic of employment deprivation in England and Scotland. Recent posts have mostly been about England and patterns of deprivation there and I've been neglecting my native land. So, I thought I would use some of the data to take a look at patterns of deprivation in Scotland using a population-weighted cartogram. The result is below (click to view full size).

Now I need to get on with some more in-depth analysis of employment data...

Scottish Index of Multiple Deprivation 2009 - Google Map

Recently I've been experimenting with google fusion tables and the mapping and database opportunities they present. One dataset I used for this was the new English Indices of Deprivation for 2010 - something the Guardian picked up. This was only really a quick experiment (literally done over an hour or so) but I was impressed with the capabilities.

Since, I'm from Scotland I also did a Scottish version - click the image below to go to the test site. Again, this is only experimental but I find it useful for exploring the data in an easy to use manner.

Technical note: this also led me to discover the amazing shpescape aka Shape to Fusion for converting shapefiles to fusion format - code which was written by Josh Livni at google. Enough nerdspeak for now.

Scottish Index of Multiple Deprivation 2009

Being of Scottish origin and interested as I am in understanding and measuring deprivation, the recently published Scottish Index of Multiple Deprivation 2009 is today's topic. It updates previous indices (SIMD2004, SIMD2006) and it measures small area deprivation in a way similar to other deprivation indices used across the UK. More information on the indicators used can be found here, and useful background and methodology information is also available. There's now even an interactive mapping function, based on Microsoft Silverlight and Virtual Earth mapping technology.

There has been quite a bit of press coverage of this, for example - The Times, The Herald, the BBC, and of course the Sun. There was also a press release from the SNP. Here's a screenshot showing the location of the most deprived area in Scotland according to the SIMD2009 (click the image to see it full size).



By the way, the link between deprivation and football stadia is not unique to Glasgow or Scotland. North Liverpool and Anfield on the English IMD2004 and IMD2007 is another good example, as is the Millennium Stadium and the Welsh IMD2005. I will have to do another post on this topic...