Showing posts with label open source. Show all posts
Showing posts with label open source. Show all posts

Tuesday, 23 December 2014

Simple Animations with QGIS - A Long Tutorial

This rather long post explains how you can make images and animations like the ones below using only open source software (mostly QGIS) and open data. I've just used flights from Hartsfield-Jackson International Airport in Atlanta because it's the busiest airport in the world and serves a wide variety of destinations. The methods are relatively straightforward but it does take time to get your head around if you're new to the software and the data. If you're already a competent QGIS user it should be pretty easy. Once you've got the method nailed down you can apply it to all kinds of different scenarios and datasets. First of all, though, here's a static image of outbound flights from ATL projected onto a globe.

Flight destinations from Atlanta (full size)

The first thing you'll need to do is get some data. In this example I've taken some general country boundaries from Natural Earth and for the flights data I've used the OpenFlights dataset. I then created an azimuthal orthographic projection centred on Hartsfield-Jackson International Airport in Atlanta. Hamish Campbell already wrote an excellent tutorial on how to do this in QGIS, so just follow that if you want to use a projection that makes your country layer appear as if it were on a globe. The only extra tips you'll perhaps need to replicate Hamish's method are as follows. To get the lat/long of a place you want to centre your projection on, just search for it on Google Maps and then look in the address bar for the coordinates. The python script on Hamish's page just needs to be copied and pasted into a text document, saved with a .py extension and then placed in the correct folder on your computer (on a PC with QGIS 2.4 it would be something like this: C:\Program Files\QGIS Chugiak\bin). Also, when you clip the layer to a global projection, QGIS will create a clip circle and a new clipped layer. These may not appear at all or if they do they'll maybe be very blocky. If so, go to the properties for the layer in QGIS and on the Rendering tab just untick 'Simplify Geometries'. I normally save new copies of these layers using Save As... from each layer.

A faster animated version of ATL outbound flights (big)

That should be your global projection sorted. You can then apply it via the Project menu in QGIS and then Project Properties... CRS and then 'Project on the fly' as in the tutorial.  To achieve some of the visual effects above, I just duplicated the clipped circle layer (you'll have to Save As... from the temp clipped layer to do this) and applied an inverted polygon style and a shapeburst fill with a blue to black gradient (Nyall Dawson did a great blog post on this, which you might find useful). I also did a similar thing with the land layer, just to make some of the smaller islands stand out. You'll also need to make the outline colour the same as the fill colour in the circle to avoid a line appearing through your earth.

A very slow version, with a pause at the end (big)

So far, so good. But what about the flight paths and animated dots? Well, to create the lines you can follow my blog post on flow mapping in QGIS and use the sample dataset I posted there. You'll need to calculate two new columns for this shapefile (see below) and use the MMQGIS plugin for QGIS (installed, as ever, via the Plugins menu). If you just add this file to your global azimuthal orthographic projection there will be so many lines and it may take a long time to display so there are a few intermediate steps I'd recommend... 

1. Open the new global flights dataset in a blank QGIS project using the default projection and then remove duplicate lines using Modify, Delete Duplicate Geometries in the MMQGIS plugin. Many routes (e.g. JFK-LHR) are served by multiple airlines and I wanted to only show origins and destinations. This also makes the file much smaller. 2. Although the flight connections would appear as straight lines on our global projection, I like them to look a little curved; partly for effect and partly to bring out the curvature of the earth but also because flight paths are not straight lines in reality. So, once I've removed duplicates I then 'densify' the lines in QGIS by adding in 50 intermediate vertices - done via Vector, Geometry Tools, Densify polygons in QGIS. 3. I then added this new flights layer to my ATL-based global projection and I clipped the layer using the Hamish Campbell method, and saving the resulting layer as a new shapefile. You should now have a globe centred on the location of your choice, plus some nice curved airline flight paths.

Same as above, but with labels and pause at end (big)

For the next stage, the way I did it was to open the dbf part of the new flight paths shapefile in Excel and then calculate an 'offset' lat and long column which I could use to animate the dots. You just need to read the Animate Columns part of Michael Minn's MMQGIS page to understand this. Once you've calculated the new lat/long offset columns you can save the csv. Once you've done this, import the csv into QGIS using the Add Delimited Text Layer (comma icon), using the airport origin lat/long as the x,y coordinates in the import dialogue. Filter the new layer so it only shows ATL origins and you'll just see one dot for ATL but actually there are many dots in the same location as they all have the same origin lat/long. Save the filtered layer as a new shapefile and then run the Animate Columns tool in MMQGIS using the appropriate fields and the number of animation frames you want (50 works well with this example). An important point here is that you need to make sure your QGIS window is quite small as the extracted image frames will be the same size as your QGIS map frame and if it's too big it will make a massive GIF.

You now have all you need to create an animation. There are many ways to create an animated GIF, but using GIMP is very simple. You can download this free, open source image manipulation programme in a few minutes. You then just need to go to File, Open as Layers and then select all the frames you just created in QGIS and GIMP will add them to the project and they'll appear in the Layers pane. You don't need to reorder them as they are numbered correctly from the MMQGIS export. From here you can go to File, Export and then select the GIF file format and use the animation options here. Try 50 milliseconds between frames as with 50 frames this will create a nice short 2.5 second animation that isn't too slow. You should use Filters, Animation, Optimize for GIF and then export from that window if you want a much smaller file size. I created another one of these visuals, centred on LHR and showing flows from JFK, LHR and PEK.

Same techniques, different data (and also a bit crazy)

That's quite a lot of information and quite a few steps but if you try this and still can't make it work feel free to get in touch via twitter or e-mail. Why would you want to do this? I'll leave that up to anyone who wants to try it but displaying movements of people and goods is relevant across a number of disciplines so hopefully some will find this useful.


Other tips and information: depending upon which location you're choosing, some of your lines or dots might be going the wrong way round the earth but you can fix this with a bit of simple maths in the offset calculation. In GIMP, you can add a different frame duration by adding a number and then ms to the layer info - e.g. 1000ms - so that it creates a pause effect, as in the examples above. I created an ATL point and a destination airports points layer from the imported csv so that I could manually create a couple of extra frames to add in to the end of my animation. One to show destinatinon airport names and the other just to label ATL. For the glow effect in the flow lines in the static image I used the Feature Blend 'addition' option in layer properties in QGIS. 

Acknowledgements: As ever, I've borrowed ideas and techniques from other QGIS users, including Hamish Campbell, Nyall Dawson, Nathan Woodrow. I decided to have a look at this after an e-mail exchange with Waldo Tobler about migration data. Thanks of course to the excellent OpenFlights team who make their data available under an Open Database Licence.

Tuesday, 26 August 2014

Why you should start using QGIS

I've been a user of GIS since the late 1990s and in that time have mostly used ESRI software, such as ArcView 3.2 and ArcGIS versions 8 to 10. The first piece of GIS software I ever used was MapInfo 5 and I continue to use it now and again (in version 9.5 or above) - mostly for manipulating large datasets with hundreds of thousands or millions of records. I still really like both of these for different reasons so this post is definitely not a proprietary-GIS-bashing piece. It's just an encouragement to current GIS users to take a serious look at QGIS if you haven't already. I've been using it on and off for a couple of years and in that time have seen serious improvements. Most recently, I've done a good bit of mapping with it - as in the example below (commuter flows in Scotland, in case you're asking).

A flow map made in QGIS 2.4

There are many reasons to start using QGIS. The most obvious one might be that it is a cross-platform free and open source GIS that can do many things as good as or better than paid-for software. Take a look at the QGIS Flickr map showcase for some more examples. Of course, it is possible to make stunning maps with other open source packages such as R, but there is a really steep learning curve and many people don't have the time or inclination to get into it.

If I was to pick my four favourite features of QGIS, I'd have to go with the following:

1. The high quality map rendering and symbology options available to you - for example, QGIS handles layer and feature transparency in such a way that you can produce really attractive maps. QGIS includes by default so many nice looking, sensible colour schemes that it's much easier to produce quality maps. Anita Graser (QGIS author and guru) highlighted the way QGIS integrated ColorBrewer at version 1.4, for example. With version 2.4, you can also automatically invert colour schemes - which was one of the rare things that frustrated me in previous versions.


Flow map layout created in QGIS 2.4


2. The Processing Toolbox, with which you can access a huge range of spatial analysis and data management tools to perform a massive variety of tasks. See screenshot below for how it looks. If you want to add x,y coordinates to a polygon layer, this can be done really simply here, in addition to so many other geocomputation tasks (e.g. calculating area, line lengths and so on). Beyond the basics there are also so many other more complex tasks you can perform here.

The Processing Toolbox in QGIS 2.4

3. QGIS Plugins - which really are fantastic. The one I probably use the most is OpenLayers, which allows you to add a large number of different base layers to your QGIS map - from Google streets and Bing Roads to OpenStreetMap and Stamen Toner layers. As I write, there are currently 214 available plugins listed in QGIS 2.4. Another fantastically useful plugin is Table Manager, which allows you to very quickly change field headers in attribute tables.

The Plugins menu in QGIS 2.4

4. Flow mapping in QGIS. This is something I've done a lot of over the years but recently I've been blown away by the simplicity and elegance of the way QGIS can convert massive CSV files into large flow maps. MapInfo had served me very well in the past - and is still amazing when you use a single line of MapBasic - and recently ArcGIS has improved, but it still has a way to go. All you need to do in QGIS is format a CSV file and have one Geom field with the LINESTRING command and x, y coordinate pairs formatted as in the image below. Once you import this file using the Add Delimited Text Layer tool the job is done. The results - following a bit of styling - can be amazing. What I love even more about this is that I picked up this tip from a StackExchange post by a 73 year old retiree! Isn't the internet amazing?


This csv file, is easily turned into images like the one below

Travel to work flows - car and train

As I've been writing this I've mentally added several more things to the list but I'll stick with the above for my favourite four right now. I do, however, also love the labelling options, the coordinate system selector options, the vector tools and the fact the user community is so helpful. I still find the Print Composer a bit fiddly for creating maps in but this is a relatively minor issue.

I expect I'll always take a portfolio approach to working with GIS software and continue to use QGIS alongside ArcGIS and MapInfo, but I'd be a bit lost without QGIS now. I'm probably quite behind the curve with all this and I should have got more into QGIS a long time ago but it's still relatively early days in the mass take-up of the software, though some UK councils are now big users

So, why should you start using QGIS? Because it's absolutely fantastic, really powerful and pretty straightforward to learn if you already know your way around another GIS. Oh, and it's free - though it costs money to develop so you can always donate here.