Enhancing building extraction with Lidar and aerial image data

With the expansion of artificial intelligence (AI) applications, automating building extraction from remote sensing data has gained significant traction. By leveraging Lidar and aerial-image digital surface models with deep learning, a Finnish study has explored different data combinations that enhance building detection accuracy in rural and urban areas.

Building extraction from remote sensing data has evolved with advances in deep learning and convolutional neural networks (CNNs). CNNs, and particularly the UNet model, have shown strong performance in extracting structures from complex landscapes. However, accuracy is influenced not only by the model architecture, but also by the type of data used. Digital surface models (DSMs), which provide essential height information, are increasingly popular in remote sensing for their ability to highlight features crucial for building detection. In this context, combining Lidar DSMs and digital elevation models (DEMs) with true orthophotos offers promising improvements in detection, particularly when working with high-resolution data such as 25cm-pixel Lidar-derived models.

Selecting two test areas in Finland

A research project in Finland utilized multiple datasets from the country’s National Land Survey. Two areas were selected: the urban and forested regions of Savonlinna, and the suburban and rural landscapes of Pudasjärvi. High-resolution (25cm) data (either Lidar DSMs or aerial-image DSMs) and DEMs were integrated with true orthophotos to assess their performance. The pixel resolution at the Savonlinna test site was 30cm, and for Pudasjärvi it was 25cm. Each dataset allowed detailed comparisons, measuring the impact of DSM and DEM variations on detection accuracy across diverse terrains. Examples of test datasets can be found in Figures 1 and 2.

Analysis of the results

The UNet model trained on Lidar DSMs consistently demonstrated better accuracy in building shape detection than when aerial DSMs were used. Tests showed that, particularly in forested areas, the model’s performance improved with Lidar, as vegetation and shadows interfered less with the detection process. Aerial DSMs, though effective in urban settings, sometimes blurred building boundaries due to shadows and overlapping features. In contrast, Lidar DSMs provided clearer delineation, capturing nuanced edges of structures. However, in cases where buildings were absent in the Lidar dataset, such as due to roof material reflections or moisture, aerial DSMs were essential to fill the gaps.

Following this comparative analysis of the DSM types, the researchers took a more detailed look at the results from the urban and rural tests, as follows:

Savonlinna (urban and forested areas)

For urban areas, Lidar DSMs reduced false detections around water bodies and produced clearer building boundaries. In forested areas, where shadows from trees can obscure features, the Lidar DSMs consistently yielded more accurate results. When 25cm-resolution data was not available, results showed a slight drop in accuracy, highlighting the benefit of high-resolution Lidar DSMs. An example of the building detection result from the urban area can be seen in Figure 3. The figure also shows false detections when using aerial-image DSMs in the forested area.

Pudasjärvi (suburban and rural areas)

Rural areas presented unique challenges, such as inconsistencies in water height data that led to false positives. To address this, false water heights were removed from the Lidar DSM, significantly reducing detection errors. Models trained with the high-resolution 25cm DEM outperformed those using resampled 2m DEM data, affirming that higher-resolution Lidar data aids in more precise building extraction. Figure 4 shows an example of the building detection result in a rural area. Due to the effect of shadows, one building was missing from the detection result of the aerial-image DSM.

Challenges and data integration strategies

The National Land Survey of Finland acquires aerial imagery in a three-year cycle covering the whole country, whereas Lidar data has a six-year cycle. While Lidar DSMs improve accuracy, issues with missing buildings and false heights in water bodies pose challenges. These inaccuracies, particularly when data is outdated or not synchronized with true orthophotos, emphasize the need for year-matched datasets. Further, although Lidar DSMs are valuable in forest environments, urban areas with many small buildings or complex rooftop materials (e.g. causing reflection or moisture) benefit from a combined approach that includes both Lidar and aerial image data.

Conclusion

Lidar DSMs, paired with high-resolution DEMs, significantly enhance building detection accuracy, especially in forested areas. This study suggests that combining Lidar and aerial image data produces optimal results, catering to the strengths of each data type. Future research should investigate 3D data integration for improved modelling in areas with dense vegetation or complex building structures. The findings support the growing role of Lidar in improving AI-driven extraction processes, especially as applications expand into more diverse landscapes.

Further reading

Hattula, E., Zhu, L., & Raninen, J. (2024). Building extraction in urban and rural areas with aerial and Lidar DSM. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 10, 73-79.

Hattula, E., Zhu, L., & Raninen, J. (2023). Advantages of using transfer learning in remote sensing. Remote Sensing, 15(17).