Simultaneous Capturing of Lidar and Imagery - 05/02/2020

How Major Cities May Benefit from a Hybrid Sensor System


People continue to migrate from rural areas to major cities, driving sustained urban growth and increasing the demand for accurate, detailed and up-to-date 3D city models. The creation of such models is still a cumbersome endeavour but new advancements, such as the combination of three sensor types – nadir camera, oblique cameras and a Lidar unit – in one and the same geodata acquisition system, may bring relief. Aerial surveys conducted in major cities in the UK and Ireland demonstrate the potential of this solution.

Cities will continue to grow as long as the world’s population keeps flocking to urban areas. In western countries, migration from rural to urban areas started in the early 1800s, and by the early 1900s 15% of the global population was living in cities. This will have increased to 60% by 2030, thanks in part to the large-scale shift towards urban living in Africa and Asia which began in the mid-1990s. This population concentration has created dozens of urban agglomerations with over 10 million inhabitants, called megacities. According to a study of the world’s demography by Euromonitor International, in the decade between 2020 and 2030 six new megacities will emerge: Chicago, Bogota, Luanda, Chennai, Baghdad and Dar es Salaam. By 2030 there will be 39 megacities, which between them will be home to nearly 10% of the population and produce 15% of the world’s gross domestic product (GDP). The liveability and sustainability of megacities rely on well-functioning roads, subways, railways, bridges, schools, hospitals and other public services.

3D City Models

Most modern cities and megacities have become complex, multifaceted 3D landscapes. Those responsible for the management, security and development of these major cities require detailed 3D models of buildings and infrastructure to support them in their tasks. 3D models are usually produced manually from stereo photogrammetry. Accurate and detailed 3D city models are in high demand, but expensive to produce. Realistic views require rendering with images, which is a labour-intensive activity. Often 3D city models consist of rendered polygon meshes that are common in computer graphics, gaming and animation. For 3D city modelling, mesh models are used for visualization, line-of-sight analysis, risk assessment, noise modelling, flood modelling, master planning and much more. They are also indispensable for creating smart cities.

Figure 1: View of the CityMapper, which combines – in a single pod – a nadir-looking multispectral camera (RGB + NIR), four oblique RGB cameras and a Lidar unit. (Courtesy: Leica Geosystems)
Figure 1: View of the CityMapper, which combines – in a single pod – a nadir-looking multispectral camera (RGB + NIR), four oblique RGB cameras and a Lidar unit. (Courtesy: Leica Geosystems)

Mesh Models

A mesh is a group of edges, lines and faces that define the surface shape of a 3D object. The faces often consist of a network of triangles usually referred to as a triangular irregular network (TIN). The higher the point density is, the denser the mesh and the more detailed the representation of the 3D object will be. If the points constituting the triangles are represented in the same reference system as images, photorealistic 3D objects can be combined to create an entire city highly automatically. Airborne Lidar and vertical and oblique aerial images are well suited to the creation of 3D city models. Mesh models are often regarded as purely visual 3D models. However, limiting their use to visualization purposes alone would do an injustice to their full potential. Particularly when georeferenced at decimetre accuracy or even higher, such 3D data can be used for measuring distances, heights, surfaces and volumes. In addition it allows line of sight and other types of analysis, shading and flood modelling. Once it is known that a collection of adjacent meshes form a building or other coherent object, it is possible to assign an address, market value, BIM information or other information to the conglomerate of meshes. Attributing semantic information allows queries and intelligent analysis to be conducted.

Bottlenecks

There are several bottlenecks in the creation of 3D mesh models of cities. One of the bottlenecks concerns the acquisition of homogeneous data over the entire survey area. Point clouds generated from images show impediments, even if the images are acquired with high overlaps and using dense image matching techniques, resulting in the extraction of no – or unreliable – points for some areas. Reasons for these shortcomings include:

  • Occlusion: in narrow streets or urban canyons, some objects may block the view of other objects such as building facades. To extract 3D points from images, it is a fundamental requirement that objects are visible in at least two images.
  • Presence of shadows: although dense image matching based on semi-global matching is highly robust against the absence of edges and texture, it may still cause unreliable matches.


Today’s airborne Lidar systems are able to create point clouds with high point density. However, the points have only one spectral value, which is the intensity of the return pulse. Nevertheless, the returns are not affected by the presence of shadows caused by sunlight, which is a clear advantage compared to photogrammetry. Since objects only have to be visible from one viewpoint, airborne Lidar is less affected by occlusion than photogrammetry.

Figure 2: Principle of the Maltese cross concept. (Courtesy: M. Lemmens)
Figure 2: Principle of the Maltese cross concept. (Courtesy: M. Lemmens)

CityMapper

To tackle the above-mentioned bottlenecks of photogrammetry on the one hand and airborne Lidar on the other, Leica Geosystems has combined three sensor types in the world’s first hybrid airborne geodata acquisition system. Called CityMapper, this system combines – in a single pod – one RCD30 CH82 multispectral camera for capturing nadir imagery, four RCD30 CH81m cameras for capturing oblique imagery and one Lidar unit (Figure 1). The nadir and oblique-looking heads are arranged according to the Maltese cross concept (Figure 2). The images of nadir camera captures RGB and the near-infrared (NIR) at 0.78 to 0.88µm. The four oblique cameras capture RGB imagery at 45 degrees forwards, backwards, left and right. The nadir imagery can be captured at a ground sample distance (GSD) of three centimetres with a potential accuracy of 6cm root mean square error (RMSE). At the centre of the image, the GSD of oblique images is approximately 75% of the nadir GSD due to a combination of looking angle and focal length. When the nadir imagery has a GSD of 5cm, the GSD of the oblique imagery at the centre will be 3.7cm. The Lidar unit emits laser pulses with a wavelength of 1,064µm and a pulse repetition frequency of up to 700kHz. The accuracy is 6cm and the point density is 15 points per square metre at a flying height of 750m. None of these three sensor types nor their specifications are new. The workflows of generating outputs from the three data types are well established and operational in many photogrammetric companies. What is new is that incorporation in a single pod and using a single control unit enables the imagery and Lidar point clouds to be acquired simultaneously. This simultaneous data acquisition offers many advantages for the creation of 3D city models. For example, Lidar pulses are able to penetrate street canyons where there may be shadows due to obscured sunlight, and in narrow streets – where occlusion might prevent cameras from obtaining two views – Lidar can double the chance of successful data capture.

Figure 3: Oblique RGB image of the Tower Bridge, London. (Courtesy: Bluesky)
Figure 3: Oblique RGB image of the Tower Bridge, London. (Courtesy: Bluesky)

Use Cases

CityMapper has been extensively deployed in Asia for accurate and detailed 3D mapping of new megacities with huge skyscrapers, which have been constructed at a breathtaking rate since 2000, and interest is now growing in Europe too. In 2018 and 2019, UK-based aerial mapping company Bluesky captured parts of London, Manchester, Birmingham, Cambridge, Oxford and several other UK cities. Figure 3 shows an oblique image of the Tower Bridge and Figure 4 shows a 3D mesh model of the area around St Paul’s Cathedral, both in London, and Dublin, Ireland, was also surveyed (Figure 5). The nadir and oblique images as well as the Lidar point clouds are processed using HxMap, which contains tools for data download and raw quality control. Additional tools for aerial triangulation, radiometric adjustments and point cloud registration and georeferencing are available. Subsequently, the data is further processed to digital surface models, digital terrain models, orthoimages and other products derived from CityMapper imagery and Lidar point clouds which Bluesky calls ‘MetroVista’. As part of a major transport infrastructure project, a photorealistic MetroVista 3D mesh model was used to gain insight into the impact of proposed construction by adding the object to the accurate 3D mesh model at the exact location and orientation. The model of the existing real world combined with the proposed development allowed the nature and scale of the development to be communicated to stakeholders and the general public.

Figure 4: Partially rendered 3D mesh model of St Paul’s Cathedral, London. (Courtesy: Bluesky)
Figure 4: Partially rendered 3D mesh model of St Paul’s Cathedral, London. (Courtesy: Bluesky)

Challenges

Needless to say, the creation of sophisticated, high-accuracy products which benefit city governments, managers and planners alike is not without its challenges. An aerial photogrammetric survey produces large data volumes that require parallel processing and hence significant investment in hardware and software in order to process, disseminate and share them efficiently and reliably. Other challenges include the weather in the UK, which is often not ideal for conducting aerial surveys. The flying height – between 1,000m and 1,800m – is lower than standard aerial surveys, which can be beneficial since it is often below cloud cover. However, the lower flying height can also draw increased attention from air traffic control in the crowded airspace above cities and megacities, leading to delays. Moreover, the flying speed – which at around 220km/h is slightly slower than for traditional survey flights – increases the data capture time, which may cause issues with permissions or weather windows.

Acknowledgements

This feature article is partly based on the results of aerial surveys conducted by Bluesky in the UK, using Leica’s CityMapper. Particular thanks are due to Penny Boviatsou (Hexagon) and Laura Eddy (Bluesky).

Figure 5: Oblique RGB image of the Aviva sports stadium, Dublin, Ireland. (Courtesy: Bluesky)
Figure 5: Oblique RGB image of the Aviva sports stadium, Dublin, Ireland. (Courtesy: Bluesky)

Last updated: 13/02/2020