Coloured Point Clouds from In-built Cameras - 21/09/2017

Automated Colouring of Walls, Ceilings and Floors

Ulrich Krispel, Martin Tamke, Torsten Ullrich

Terrestrial laser scanners equipped with in-built cameras are well suited for identifying surfaces in point clouds of building interiors. The colours from the images can be automatically projected on the planar structures, resulting in a textured 3D model of rooms, halls, corridors, stairwells and other indoor spaces. Besides this, using computer vision methods, sockets, switches and other objects attached to walls can be automatically identified in the 3D model and their x,y,z positions determined. This article presents a case study.

Many terrestrial laser scanners (TLSs) not only acquire range measurements from which 3D coordinates of points can be calculated, but are also equipped with an in-built RGB camera which produces spherical panoramic RGB images.

Scanner

One of the TLS devices equipped with an in-built camera is the Faro Focus 3D scanner, which has been used in the present case study (Figure 1). The device weighs 4.2kg and can measure close to one million points per second. A built-in 8MP camera captures RGB imagery which makes it possible to automatically assign colour to each and every point of the point cloud. High dynamic range (HDR) is a technique for capturing images with multiple exposure rates and merging them into a single image. HDR cameras enable image capture even in poor lighting conditions, making surfaces visible in dark or bright areas. The resulting images provide spherical panoramic RGB views.

Assumptions

Using a laser scanner and a spherical camera in tandem makes it possible to assign fully automatically natural colour information to each and every point in the point cloud. When constructing 3D models from point clouds, however, not all points are equally useful. For example, when one wants to model walls, ceilings and floors, returns from furniture, plant pots and coffee cups are superfluous. An automated procedure should take account of the presence of irrelevant points, and such an approach requires some assumptions. One of these is that walls, floors and ceilings can be geometrically modelled as planar structures which are visible in at least one scan with a sufficiently high point density. Furthermore, all point clouds and images have to be registered in one and the same coordinate system and have uniform scale. A further assumption is that the vertical direction is known and, when the interior consists of multiple spaces, so too are the thicknesses of walls.

Figure 2, Point cloud of seven registered scans with front wall removed for a better view of the interior.
Figure 2, Point cloud of seven registered scans with front wall removed for a better view of the interior.

Data Acquisition and 3D Modelling

Seven interior spaces have been scanned from seven scan positions resulting in a point cloud of 77 million points (Figure 2). After registration of the point clouds aimed at bringing all points together in the same coordinate system, first a coarse 3D model of the interior has been manually created from a low-resolution mesh. This has resulted in a geometric outlining in the form of a wireframe of the room surfaces without windows and doors (Figure 3). To be useful for assigning colour to the surfaces, the coarse wireframe has to be refined. For this the authors developed an automated procedure to detect rectangular planes by grouping similarly oriented triangles that lie in the same plane. From each plane of similarly oriented triangles, the smallest rectangular patch is determined. Next, each patch is subdivided into pixels with predefined resolution, e.g. 1mm, to which colour is assigned. This is obtained from the nearest scan which captures the patch by intersecting the line connecting the pixel and the scanner with the panoramic image. Of course, the pixel may be occluded by furniture and other objects present in the line connecting pixel and scanner. This will cause the wrong colour to be assigned to the pixel. This can be avoided by removing points that do not lie on the room surfaces, but this will inevitably lead to colourless gaps. For appealing visualisations, inpainting techniques may help to fill these gaps. Removal of points not lying on the surfaces will be the subject of future research.

Figure 5, Two views of the coloured wireframe with ceilings removed to view the interior.
Figure 5, Two views of the coloured wireframe with ceilings removed to view the interior.

Results

The colour quality of the surfaces is good in areas where the colour information corresponds to the real surface (Figures 4 and 5). The coloured surfaces can be input for object detection. Figure 6 shows an example of the automatic detection of a socket using computer vision classification methods. Wrong colours can be assigned to patches when furniture or objects are present in the line between patch and scanner. The average distance of triangles that will be grouped together needs to be precisely specified; this parameter is mainly determined by the thickness of walls and the resolution of the scanner. In general, the quality is good enough for performing object detection.

Concluding Remarks

This approach may be used for enriching building information modelling (BIM) models with semantic information. Furthermore, the RGB images provide details of wall decorations such as posters, images, signs and other non-geometric information which may be useful for certain applications. Added to this, the colour information can be used to generate smaller 3D models suitable for interactive visualisation while preserving details present in the images. For example, the initial point cloud requires 1.77GB of storage capacity and the uncompressed imagery needs 223MB, while the web export of the coloured wireframe requires only one HTML file of 8.08MB. The source code is available at https://github.com/DURAARK/orthogen.

Acknowledgements

Thanks to Per-Kristian Hansson for conducting the scanning and registration, and to Paul Pierce for granting permission to use the data.

Further Reading

Krispel, U., Evers, H. L., Tamke, M., Viehauser, R., Fellner, D. W. (2015) Automatic Texture and Orthophoto Generation from Registered Panoramic Views. ISPRS – Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-5/W4, 131-137, https://doi.org/10.5194/isprsarchives-XL-5-W4-131-2015, 7p.

Krispel, U., Evers, H. L., Tamke, M., Ullrich, T. (2017) Data completion in building information management: electrical lines from range scans and photographs. In: Visualization in Engineering. 5, 1, https://doi.org/10.1186/s40327-017-0042-5, 11p.

Ochmann, S., Vock, R., Wessel, R., Klein, R. (2015) Automatic reconstruction of parametric building models from indoor point clouds. Computers & Graphics, 54, 94–103, doi:10.1016/j.cag.2015.07.008, special issue on CAD/Graphics, 9p.

Last updated: 19/10/2017