sqldf("select * from irisTable")

This simple statement will request all columns (all indicated with *) from a table which you are basing your work on. Before we start we will create the sample data. We will use the Iris dataset and change column names to remove any full stops and spaces.

irisTable <- iris

names(irisTable) <- c("Sepal_length", "Sepal_Width", "Petal_length", "Petal_width", "Species")

However, what if we only want a selection of all columns? Further, we may wish to order our data by a particular column. This can be achieved by appending ‘order by <col_name> desc or asc.

sqldf("select Species, Sepal_length from irisTable")

By specifying the column names as they are displayed in the data.frame we can select these specific columns. At this stage we have the basic functionality available in the standard Pivot Table tool. Moving on to aggregating our data we can use the ‘group by’ function as demonstrated below.

sqldf("select Species, count(*) from irisTable group by Species")

Having added the group by statement we are now bringing the real power of Pivot Tables to R. If we assume that Species is a list of row names we have brought together all of our records into a much smaller number of categories. Questions we may now want to ask are: What is the average Sepal Length of each Species? What is the Min, Max or average petal length? these function may be simply applied by wrapping the chosen variable with the function you wish to perform.

sqldf("select Species, avg(Sepal_length) from irisTable group by Species")

A full list of these functions can be found at W3 SQL. In addition to math operations a number of text based operations are available such as length() which determines the length of a string, UCASE() converts to upper-case and LCASE converts to lower case.

The next aspect of Pivot Tables are filters. Filters allow subsets of the data to be generated for example we want to study a specific district, or are looking for values between two points. You will notice that once we have used the ‘group by’ statement we use ‘having’ rather than ‘where’.

sqldf("select Species, Sepal_length from irisTable group by Species having Sepal_length between 5 and 6")

sqldf("select Species, Sepal_length from irisTable group by Species having Species='virginica'")

The above two example highlight the use of where with two different approached. The first example is looking for values between, and including 0 and 10. The second example is looking for all records relating to Berkshire. It should be noted that where clauses are not case sensitive.

One issue which many users experience in R is the generated field names created during joins, or maths functions. In its basic format, sqldf will produce similar column names however this can be rectified using the ‘as’ statement.

sqldf("select Species, Sepal_length as 'Avg Sepal Length' from irisTable group by Species having Species='virginica'")

Though brief, this tutorial is able to address all and more of the functionality found within Pivot Tables and yet leaves many features of sqldf unused. Additional functionality includes table join, SQLite backend databases and data recording.

In addition to myself are fourteen other equally intrepid Drs-to-be including 2 geographers, 2 physicists and an array of forensic scientists. All in all epitomizing the interdisciplinary nature of modern research. In order to break the ice the department (SECReT) organised our first joint excursion to the Future Security in Bonn, Germany! As you can imagine, the game of ‘I spy a SECReT student’ at the airport was gripping and to give you a taster of the conference setting i’ve included a picture below.

Future Security 2012, Bonn

Though the conference presented a number of challenges (Plenary talks in German) the conference was fairly rewarding and highlighted the cross discipline nature of many aspects of Security Science and allowed everyone to identify shared group in our own knowledge. Further, with the taught component of the year starting the week, everyone is placed on a fairly even playing field moving away from their present background to cover modules including Global Security Challenges and Information Security.

Okay so that will do for now and hopefully the posts will start flowing once again looking at GIS, Tutorials and now Crime and Security Science!

Firstly, we can consider the creation of the cartogram. For simplicity’s sake, the dataset used is the simplified world outline provided by www.gadm.org/ and for our imagery we use one of the NASA Earth Marble Images downloaded in the geotiff format.  Using the Cartogram Geoprocessing tool version 2 available at http://arcscripts.esri.com/details.asp?dbid=15638 we are able to perform the cartogram function on the global boundaries vector layer and in parallel any additional spatial layer. The image below being the population cartogram marble. One issue with the cartogram tool is that the skew of the image morphs the image away from its simple rectangular format so the image must be slightly cropped for the benefit of the 3D conversion.

The 3D version of the globe is created using Blender, a free and open-source 3D modelling package with many comparable features to commercial packages such as 3DsMax. Once the cartogram texture is created, it can be mapped to a sphere mesh and subsequently through the use of keyframes can be animated to spin in a smooth and earth-like manner. Further details will be provided later through a tutorial. Further reality could be added to the map though the addition of night lighting and clouds giving the earth a more realistic appearance.

1. 3D Laser Mapping – Mapping vs Survey Grade by Andrew Fuller
2.  Ten things not to do with maps by Mary Spence
3. Everyone can be a hydrographic surveyor by Tim Thornton

The 3D Laser Mapping talk though not my main focus proved to be very informative and highlighted the development of terrestrial laser scanning. The first point was that survey grade accuracy is typically <3cm whereas mapping grade is <100cm. The talk also reiterated the importance of mission planning when conducting high accuracy surveying. Due to the varying positions of the different positing systems during the day it is important to conduct a survey when the greatest  number of satellites are overhead. This is even more important when working in urban canyons where aerial visibility is severely limited.

Andrew Fuller talked in some detail about the use of Inertia Monitoring Units IMU which work to correct position whilst allowing the scanner to know its position even when satellite coverage is inconsistent. To see the results of the surveys have a look at the video below. It is quite easy to see how this could be transferred into the games industry to create real world places.

The next talk was one with a balance of humour and purpose. Mary Spence gave a rapid and well illustrated sample of common cartographic mistakes and possible solutions. Her talk worked through the most common ailments including colour, labelling map furniture etc highlighting correct vs incorrect use.

As was highlighted, many of the suggestions made where common sense however, commonly forgotten. I will give a few examples of Mary’s recommendations below.

1. Map Content

The final point of Mary’s talk was on the subject of map content. All too often a cartographer seeks to embed far more information onto a map than is required and will typically lead to congestion and confusion. Mary highlights the importance of planning and the consideration of output type and size, purpose and user. She used the example of one customer who used the phrase “It seems a shame not to include it” having digitised more data than required.

2. Marginalia

Again a common mistake with map makers is the use of text such as “Map of….”, “Legend showing..” and “This map contains…”. In the majority of cases these statements are redundant and remove the professional impression of the map. If it requires the designer to tell the reader it is a map surely it is not a very good map. Additionally when a designer uses arty borders, north arrows (a topic of their own) and insets then the question must be asked is the designer trying to hide something about the actual maps quality.

3: Do not lie

This line was shown alongside the Kingston university 3D campus map which the university paid a large amount of money to complete. The project included the use of helicopter surveys yet the final product is over generalised and as such is a very poor abstraction of reality. This was one of the motivations of my past lecturer Dr Ken Field to get students to create a more appropriate map.

The final talk Everyone can be a hydrographic surveyor by Tim Thornton was possibly one of the most significant. Tim Thornton presented the work of TeamSurv, an open-source bathymetric mapping project. The project aims to collect large amounts of depth data from yachts and cruisers and to collate and map the data. The project appears to be well engineered and takes into account many of the questions regarding data quality that may be asked. Firstly, all data submitted is collated into a database and a certain density of recordings is required before a depth is placed on the map. Through the use of regression analysis all depths may be plotted and those which are beyond a sensible range are excluded. User who regularly submit  poor data are blacklisted or asked to recalibrate equipment. The next portion of the project is the correction due to tidal heights and sea temperature. The project takes into account sea temperature on the speed of sound/light in water and uses almanac data verified alongside live tide sensors to make all results consistent.

The majority of data collection to date has taken place within the Solent, however as the project grows they are now collecting data on a global scale. In addition the project has had some of their results in the Solent verified through wide beam sonar bathymetric scans. These professional surveys suggest the TeamSurv data is correct to +- 10cm. Coverage within the Solent is now good however, there are a number of blank spots at shallow regions.

I think this project is very interesting and is certainly one to follow. As the project expands and more users contribute it will be interesting to see strategies designed to increase coverage and infill certain parts of the dataset.

Smith spent the majority of his life in the pursuit of the map and is known to have travelled extensively through his life. His greatest hindrance was a lack of finance and as such  had to fit his work around his job as a land drainer and engineer.

The book tells of the sampling that smith conducted through his life and how he accessed funding through numerous publications and favours. The book also tells of the darker side of the scientific community of the time who seemed intent on plagiarizing  both his work and ideas. This disrespect for his work led to many of Smiths setbacks during life and later to a short stint in debtors prison. Smith did however eventually receive the credit he was due and receive the first Wollaston Medal for his lifetime achievements.

Smiths final map of the geology of the England as shown below is testament to the skills he has gained during his life in the measurement and depiction of the geology of the country. Though a reasonable number of the final maps where produced, few are now left in regular circulation though one may be seen in the offices of the royal geological society.

The moral story of the map is as powerful as the creation of the map. Holding a lifetime passion for a subject and being able to step back up to the mark when you and your work are mocked will allow you to create and achieve great things. During his life William smith had to fight for any recognition with the aristocrats of the time yet he is now considered one of the greatest men in the history of geology.

…

A particular feature of the book is the regular assessment of some of the most commonly used.. and misused fonts. Of particular interest being the fonts Gill Sans (p.48) and Comic Sans. Though the book does not dissect fonts in the same was as some of the more traditional books, it does lead the reader to contemplate their font of choice and maybe even to appreciate and understand why or what different fonts mean what they do.

Just for those who really want to see a bit more on the anatomy of font you could start with the graphic below.

From my perspective I feel that typography plays a significant role in the world of cartography and data visualisation for a number of reasons. Firstly as a cartographer you are working with a range of different font sizes, purposes and colours. Through a knowledge of type one is able to engineer these fonts to work in harmony and to make the information as clear and useful as possible. In addition through the use of appropriate font style the cartographer can influence the perceived purpose of the map. An elegant gothic font may promote an historical or artistic map whereas a clean sans serif may suggest a purposeful information map.

This first visualisation allows the user to reference the location of individual bikes however, an error within the processing code resulted in all bikes going to incorrect destinations. Having solved this problem, the next sketch removes the basemap to provide a less congested visualisation. The white points indicate bike journeys and the red flashes indicate the start of a journey.

This visualisation is an improvement on the first but the mix of colours does not work as well as it could. This is addressed in the final visualisation below where the  trip beginnings flash white. Though simple the effect through the video is to see the concentration of starts moving from the outside zones into the centre and about 5pm the reverse with many trips initiating in the city of London region.

The final product works well and indicates and shows the movement of bikes through the day. Problems with the visualisation however are the lack of a spatial reference to orientate the user and also an indicator of the total number of bikes in use at any time.

The book further describes the challenges faced by the ordnance survey in the sense of the Napoleonic wars and the continues distractions which they faced: In particular completing a full map of Ireland before completing the original First series, Measuring the distance between the Greenwich and Paris Observatory and mapping Jerusalem.

The first image is a view of the office as it is being created in Sketchup and it is evident that the overall image quality is good but nowhere near realistic. Though this may be the case, the modelling process is so efficient that this was created in under and hour.

The next phase is the export from SketchUp and import into Lumion. This process is particularly simple and relies on the use of the COLLADA file format. Using the menu File > Export > 3D Model. In the save menu it is then important to set save options to include 2 sided faces, export edges and triangulate all faces.

Once in Lumion the import process is fairly simple. In the objects menu select import file and then select the COLLADA file. Important consideration at this point is whether to create new textures with Lumion or use the default Google SketchUp.

The video above has been created using the Lumion video engine and exported at 720 dpi.   The video maker allows a simple screen-grab based approach to creating a path and then intelligently renders a path between them. The video editor has significant functionality and allows people, vehicle and object animations to be included.

A few faults evident in the video was the flickering on the Mac and tv displays which was a result of multiple textures on the same plane. This may be alleviated by lifting the primary texture.

As is evident in the video the main aspect of this code is the keyboard based interaction of the scene. Panning around the scene is available with the traditional A,W,S,D navigation scene and zooming and tilting available using ‘z’ and ‘l’.

The navigation is created through the use of the processing keyPressed() function. This code is in its own .pde file within the main sketch to simplify the sketch structure. In order to create the pan and zoom feature, the camera function is used.

Camera() function syntax:
camera(eyeX, eyeY, eyeZ, centerX, centerY, centerZ, upX, upY, upZ)

By setting both eyeX and centerX (and the same for Y) to be the same variable the camera will always remain directly above the image. Consequently by adding or subtracting from the cameraX and cameraY variable the scene appears to move.

Processing file available Here