Point Clouds and Smart Cities - 25/10/2018

The Need for 3D Geodata and Geomatics Specialists


The ‘smart city’ concept entirely relies on a permanent stream of massive amounts of data acquired by a great variety of sensors distributed throughout the city. Smart use of all this data requires integration with 3D city maps for which point clouds, acquired by laser scanning or photogrammetry, are the main sources. The author of this article identifies the abilities of point clouds to support the smart city concept.

The benefits of point clouds for monitoring urban processes were recognised quite early on. Back in the early 1990s, airborne Lidar could accurately provide the height component which is so important for many urban needs. Not only the geometric accuracy appeared to be amazingly high, but also the point density. The needs include design and inspection of utilities such as water mains, sewer systems, tunnels, bridges, roads, railways and power lines, and the creation of 3D city maps in which the shapes of buildings and other objects have been reconstructed with high spatial detail. One prerequisite is the availability of geodata, which represents the environment in its full spatial and time dimensions at a highly detailed level. Before being ready for use, geodata has to be acquired. This seems trivial, but is far from it. Thanks to tools like Google Maps, too many laymen take the existence of geodata for granted. The desire of many local authorities to implement the smart city concept will result in a strong need for increasing the acquisition of geodata. Since geodata acquisition is time-consuming, labour-intensive and associated with a number of technological issues and costs, it may be readily envisaged that the creation of smart cities will be associated with a bothersome burden.

Smart city concept

People continue to move to cities. Population growth within the limited boundaries of a megapolis increases traffic, housing, utility density and the risk of calamities, and puts a strain on scarce resources. As a result, the economic, social and environmental impacts are tremendous. Added to this, nearly one billion people live in cities which may be hit by floods, droughts, cyclones, earthquakes or other natural disasters. Humans, power, water, gas, wastewater, bits and bytes – they all are transported through a labyrinth of roads, tunnels, pipes and cables. Flood, fire or other shocks can disrupt these arteries and veins of the city. To cope with these threats and optimise the use of scarce resources, many city authorities want to exploit the opportunities offered by today’s technology; they want their city to become ‘smart’.

Figure 1: Bird’s-eye view of a part of Delft, The Netherlands, drawn in 1580.
Figure 1: Bird’s-eye view of a part of Delft, The Netherlands, drawn in 1580.

In the smart city concept, information and communication technology (ICT) is fully exploited to increase the efficiency of mobility, retail and delivery services, to minimise energy consumption and air pollution, and to reduce the response time to casualties and hazard sites. The goals are often formulated in abstract terms such as liveability, economic progress, quality of life, sustainable development or attractiveness. In the smart city concept, decisions are not made ad hoc or based on political beliefs or preoccupations. Instead, decision-making is informed, based on the plethora of data obtained from sensors and analysed using smart algorithms – possibly based on machine-learning principles – running on high-performance computers. A city cannot be smart without the permanent availability of data collected by sensors mounted on static or moving objects. To illustrate this, the importance of point clouds for water management is discussed below.

Water management

Flood threats are increasing due to rising sea levels and rapid urbanisation. Flooding is caused by an accumulation of water which may result from heavy rainfall, swollen rivers or dike breaches. Flooding causes degradation of assets, economic losses, injuries and health risks. Renovation of dikes, creation of water overflow areas and protection of vulnerable dunes are essential for urbanised lowlands such as the Mississippi River area and the Rhine/Meuse Delta. For these purposes, detailed and accurate knowledge is required on the variations in elevation of the river bed and the surrounding area, height of dikes, flood waves, along-track and across-track slope of the river and the water resistance associated with land use. The use of a digital elevation model (DEM) or, even better, a 3D map derived from point clouds as input for flood models provides insight into areas at a high risk of inundation. The development of a new urbanised area will affect the flow of water. Is the sewerage system still able to drain off the water after heavy rainfall? Does the rainwater still flow in the direction of the river or reservoirs? Should the drainage layout be improved? These are all questions a smart city should be able to answer, and the solutions lie in the expert analysis of 3D geodata by geomatics professionals.

Figure 2: 3D map of the commercial centre of Frankfurt, Germany, created from free and open-source Lidar data and satellite images; buildings, ground and streets consist of one textured mesh (source: Turbosquid.com).
Figure 2: 3D map of the commercial centre of Frankfurt, Germany, created from free and open-source Lidar data and satellite images; buildings, ground and streets consist of one textured mesh (source: Turbosquid.com).

Geodata

Essential in the smart city concept is the availability and continuous production of digital data, which may be collected dynamically or statically, continuously or intermittently. Smartphones and social media channels such as Twitter deliver real-time data on human movements and how people use the city’s space, either by public transport or their own methods, and other facilities. Static sensors distributed over the city produce data on particulate matter, noise pollution, temperature, groundwater level and so on. Cameras can detect traffic accidents and enable rescue workers to assess injuries while still rushing to the scene of a disaster. The permanent stream of real-time data captured by a dense sensor network needs to be processed, analysed, used in decision-making and disseminated to stakeholders. For this, the location of each sensor, whether static or dynamic (such as a smartphone in a cyclist’s pocket), must be known in a uniform coordinate system. To guarantee the smart use of all sensor data, the skeleton of a smart city consists of cadastral maps, master plans, utility maps (containing the location of underground pipelines, cables, sewerage systems, manholes and much more), as-built plans, maps outlining buildings and roads, and 3D maps at the necessary level of detail.

3D city maps

3D maps include the height dimension which is essential for many smart city applications such as flood risk monitoring, emergency response, viewpoint analysis and calculation of the solar energy potential of parcels and roofs. Making a city really smart starts with the creation of an accurate 3D map covering the entire city. Of course, such a 3D city map should be precise, detailed and up to date. Positional or thematic inaccuracies and outdatedness will impair proper decision-making. The very nature of geoinformation means that damage originating from substandard products may not be immediately recognisable. Mistakes made today may – a decade or two later – cause society significant losses, not only moneywise, but also in terms of injuries, accidents, demolished houses, and human suffering. Accurate 3D mapping is therefore an important, even essential, asset underpinning the smart city concept. Capturing and representing the 3D space of a city is challenging because of the level of detail required. 3D mapping requires the right data, the right software tools and geomatics professionals with the right knowledge.

Figure 3: A RIEGL VMX-250 mobile scanning system scanning the historic market square of Krakow, Poland (source: RIEGL).
Figure 3: A RIEGL VMX-250 mobile scanning system scanning the historic market square of Krakow, Poland (source: RIEGL).

Importance of updating

The advantages of 3D city maps were first recognised many centuries ago. Armed with only pencil and paper, surveyors and cartographers in the 16th and 17th centuries exploited their craftsmanship to produce bird’s-eye views which combined a 2D map with perspective views of buildings (Figure 1). Hundreds of years later, aerial images processed with digital photogrammetric workstations enabled extraction of 3D coordinates of corners of buildings and other key points, albeit in a labour intensive process. Apart from various types of imagery (satellite, aerial and terrestrial), today other data sources are available for creating 3D city maps, including airborne, mobile and terrestrial Lidar and topographic and cadastral 2D maps. Using a mix of these data sources allows a 3D city model to be created highly automatically (Figure 2). As airborne Lidar – and, later on, mobile laser scanning (MLS) systems (see Figure 3) – became more mainstream, this boosted the research into automated 3D mapping. For example, airborne Lidar enabled Rotterdam to generate a 3D map of buildings which was made openly available in 2011. The basic data source was a detailed map with footprints of buildings. The airborne Lidar point cloud was acquired by Fugro’s Fli-Map system in 2010. The accuracy and point density of 30 points/m2 enabled the automatic extrusion of the footprints resulting in a Level of Detail 1 (LoD1) representation of Rotterdam in CityGML (Figure 4). Manual editing was required to eliminate anomalies, some caused by time shifts in the acquisition of the map and point cloud. Since this is now outdated, Rotterdam is currently at an advanced stage of releasing a much more detailed 3D map in close consultation with a broad spectrum of prospective users. Regular updating is a necessity. Indeed, public authorities often become enthusiastic about the smart city concept initiated by one or more city department and release financial resources, but underestimate the necessary costs of the annual or biannual updating to preserve the value of the 3D maps. Despite all the photogrammetric methods and laser scanning research, automated 3D mapping has not yet materialised; today’s detailed 3D city maps are created manually. For example, AccuCities, based in London, UK, commercially offers detailed 3D maps based on the manual processing of high-resolution aerial images (Figure 5).

Figure 4: 3D map at LoD1 of a part of Rotterdam automatically created by combining building footprints with airborne Lidar point clouds (source: Municipality of Rotterdam).
Figure 4: 3D map at LoD1 of a part of Rotterdam automatically created by combining building footprints with airborne Lidar point clouds (source: Municipality of Rotterdam).

DIM and SLAM

Since 2010, the acquisition capabilities of both photogrammetric and laser scanning sensors have been tremendously improved, but so too have the geodata processing methods thanks to innovative advancements such as dense image matching (DIM) and simultaneous location and mapping (SLAM). DIM enables the creation of dense point clouds from overlapping images. Meanwhile, SLAM has been one of the most prominent successes in robotics and enables a robot to move autonomously through an unknown space. No map is needed, and no GNSS; the solution is based on ‘guessing’ the position by exploiting all sensor data. The guesses are iteratively refined using data collected while the robot is moving and the iterative closest point (ICP) algorithm aimed at minimising the difference between successive scans. Landmarks – features which are distinct from the background – are central in the SLAM approach. SLAM solutions are also beneficial for 3D mapping of indoor and subsurface spaces using trolleys, hand-held sticks (Figures 6 and 7) or backpacks (Figure 8). On trolleys, the main positioning and orientation sensors are odometers, inertial navigation sensors and lasers. Odometers count wheel spins enabling computation of the speed and distance covered. When mounted on two wheels, odometers indicate directional changes. SLAM does not consist of a particular algorithm, but is rather an approach in which a diversity of solutions are employed depending on on-board sensors, the nature of the environment and possibilities to connect the trajectories with ground control points.

Today’s photogrammetric software enables high automation of the chain, from flight planning, self-calibration of consumer-grade cameras and aerotriangulation to the creation of DEMs and orthomosaics as well as their confluence: 3D landscape and city models. However, the creation of detailed 3D maps of cities still requires considerable human intervention. DIM is one of the technological innovations contributing to the automation of the process. DIM is based on semi-global matching, the basics and potentials of which have been developed and demonstrated by Hirschmüller (2008). DIM enables the computation of a height value for each pixel, thus producing high-resolution digital surface models (DSMs) and – by filtering out points reflected on buildings and vegetation – DEMs in (semi-)automatic workflows. To demonstrate the capabilities of DIM implemented in commercial software, the section below presents a project comparing feature-based matching software, developed by students, with commercial software.

Figure 5: 3D map of a part of London manually created from high-resolution aerial images, covering 6km2 (source: AccuCities, London).
Figure 5: 3D map of a part of London manually created from high-resolution aerial images, covering 6km2 (source: AccuCities, London).

Photogrammetric point clouds

During a project carried out in the last quarter of their first year, students at the Delft University of Technology (TU Delft) working towards a master’s in geomatics developed the ‘NARUX3D’ software to automatically extract textured mesh models from images captured by unmanned aerial systems (UASs). The software features three main modules which run in succession: image matching, sparse and dense point-cloud generation, and mesh generation. The scale-invariant feature transform (SIFT) was used to detect tie points in overlaps for co-registration of images, resulting in the camera orientations and an initial sparse point cloud using structure from motion (SfM). The sparse point cloud was then densified using a multi-view stereo (MVS) algorithm. To reconstruct a 3D mesh from the dense point cloud, two methods were developed. The first – Poisson surface reconstruction – worked well for smooth surfaces. The second method – PolyFit – was effective for creating compact mesh models from piece-wise planar objects. The performance of NARUX3D was tested at two sites located on the TU Delft campus. During a UAS flight, images of a building (see Figure 9) were taken every two seconds, resulting in 99 images.

Figure 6: FARO handheld scanner.
Figure 6: FARO handheld scanner.

Providing that the overlaps were sufficiently high, the software was able to create point clouds even when the facades consisted of a variety of materials with complex textures. The resulting dense point cloud and mesh shown in Figure 10 (on the left) have been compared with the results from Pix4Dmapper commercial software (Figure 10, on the right). In general, point clouds generated by the latter were much denser, while the meshes were less noisy and had a more realistic appearance. The SIFT matching approach uses feature-based image matching. Distinctive points are detected in the overlaps of images to which descriptors are assigned. Corresponding points in overlaps are then sought based on similarities between descriptors. When little or no texture is present, no reliable matches can be found. This was the case at the second TU Delft site, a white cottage (Figure 11). The roof and facades were coated white, and Pix4Dmapper dealt well with that. However, NARUX3D also extracted points on corners, ribbings and other locations with sufficient contrast. Considering the price of commercial software, the results the students obtained in just two months were quite impressive. Semi-global matching is now a regular part of many commercial photogrammetric mapping software solutions including Pix4Dmapper, SimActive’s Correlator3D, Racurs’ Photomod, Bentley’s ContextCapture, SURE of nFrames, Trimble’s Inpho and others.

Figure 7: GeoSLAM ZEB-REVO mobile 3D scanner.
Figure 7: GeoSLAM ZEB-REVO mobile 3D scanner.

Geomatics professionals

Most local governments do not have enough skilled personnel to implement the smart city concept. The private sector is eager to fill the gap, but companies often lack geomatics expertise too. No geodata or geoICT has much value unless it is in the hands of professionals who have the knowledge and craftsmanship to transform the data into information which supports the local authorities’ goals and needs. And the best-qualified people to do this – experts who can set geodata to work for the benefit of the masses – are the geomatics specialists. Unfortunately, in and around the year 2000, many institutes of higher education experienced a low influx of new geomatics students and some universities abandoned their geomatics programmes altogether. Today, society is confronted with a shortage of skilled, experienced and knowledgeable geomatics professionals. Universities and local governments should join forces to fuel young people’s enthusiasm for a career in geomatics. They are badly needed.

Further Reading

Hirschmüller, H. (2008) Stereo Processing by Semiglobal Matching and Mutual Information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2), pp. 328-341.

McKinsey & Company (2018) Smart Cities: Digital solutions for a more livable future.

Figure 8: Leica Pegasus:Backpack consisting of five cameras and two laser scanners mounted on a carbon-fibre chassis.
Figure 8: Leica Pegasus:Backpack consisting of five cameras and two laser scanners mounted on a carbon-fibre chassis.

Figure 9: Green village sustainable building on Delft University of Technology campus, imaged from a UAS.
Figure 9: Green village sustainable building on Delft University of Technology campus, imaged from a UAS.

Figure 10: Point clouds and meshes generated by SP software (left) and Pix4Dmapper.
Figure 10: Point clouds and meshes generated by SP software (left) and Pix4Dmapper.

Figure 11: Building with little texture (left), point cloud generated by SP software (middle), and point cloud generated by Pix4Dmapper.
Figure 11: Building with little texture (left), point cloud generated by SP software (middle), and point cloud generated by Pix4Dmapper.

Last updated: 20/11/2018