3DBAG: automatically generated 3D models of 10 million buildings

For a long time, it had been an outstanding research problem, but 3DBAG has turned the large-scale automatic reconstruction of detailed 3D building models into reality. Developed within the 3D Geoinformation research group at TU Delft, 3DBAG is an open and freely accessible data ecosystem – not just a geospatial dataset, but also a web viewer and dissemination platform offering various data formats and web services to maximize accessibility and usability – containing periodically updated models of the buildings in the whole of the Netherlands (approximately 10 million in total) at multiple levels of detail. As a valuable cornerstone for urban planning and engineering, 3DBAG is already widely used by government organizations, businesses and academia for multiple applications. Therefore, it is also being explored how to embed the 3DBAG in the national spatial data infrastructure (NSDI).

The development of the ‘Detailed 3D Building models Automatically Generated for very large areas’ (3DBAG) stemmed from a research question within the 3D Geoinformation research group at the Delft University of Technology (TU Delft): Was it even possible to generate three-dimensional building models at a high level of detail (LoD) – i.e. at LoD2.2, including roof shapes – for the whole of the Netherlands, in a fully automatic way?

This question was closely aligned with the research group’s aim: to stimulate the use of 3D geoinformation in practice. “By offering large-scale, low-cost, highly detailed and up-to-date three-dimensional building models we can significantly reduce the need for manual modelling and interpretation of spatial data. This helps such data to become mainstream so that it can be used in simulations and applications to analyse the environment, maintain urban areas and improve the efficiency of urban processes. Automatically reconstructing LoD2 models for large areas has long been an outstanding research problem, but the availability of high-quality elevation data for the whole country, and the technical know-how in our research group, put us in a unique position to tackle this problem,” states Prof Dr Jantien Stoter, chair of the 3D Geoinformation Research Group, head of the Section Urban Data Science at TU Delft, and also a researcher at Kadaster.

Quality issues with available datasets

Around 2018, the research group started out by building a national dataset of simple block models. “Although simple 3D block models of buildings in the Netherlands were not difficult to generate, and several datasets were already available at that time, the country lacked a good standard source for them. And standardization is important, because one block model is not the other. For example, what does ‘height’ represent – the highest point, or the average height?” says Dr Ravi Peters, who worked alongside Balázs Dukai as one of the lead developers on the research project.

“Moreover, it was not always clear how the available datasets had been generated. This raised quality issues; it’s not ideal to be using data of which the quality or reconstruction decisions are unknown, especially if it’s for an important government project,” continues Peters. “So we started out by focusing on the automatic generation of simple (LoD1) block models with several well-explained representative heights, from which users could choose the best option for their application.”

Increasing level of detail

Since that first version, the dataset itself, the associated website viewer and the underlying software have all continuously been developed into the version that is available today. “In parallel to the first basic version of the 3DBAG, we started working on a project with noise experts from the National Institute for Public Health and the Environment (RIVM). The aim was to explore whether we could automatically generate detailed building models as required for noise propagation modelling in the Netherlands,” recalls Stoter.

Peters adds: “Several ideas were investigated, and eventually we developed a method that could automatically generate LoD1.3 building models. These were still block models, but now each building could be split into several parts, each with their own height level; think of a church with a tower, or a house with a shed attached. We then discovered that, conceptually, our method made it a relatively small step from LoD1.3 to the even more detailed LoD2.2. This work on the 3D Noise dataset eventually led to a completely new version of 3DBAG, with LoD1.3 and LoD2.2 models.”

“This was the first time that such highly detailed LoD2.2 building models were made available on such a large scale. Datasets with detailed building models were available at that time – around 2020 – but on a much smaller scale; a single city was pretty much the extent of it. Not to mention that building those datasets involved a lot of manual work, so it was both time-consuming and cost-intensive,” he continues. According to him, the process of releasing the LoD2.2 version of the 3DBAG was particularly labour-intensive, involving a team of four people working for several months in the run-up to the first release of 3DBAG with LoD2.2 in March 2021.

Data sources

For automated reconstruction, 3DBAG needs high-quality classified Lidar data at a density of at least 8pts/m², along with good data consistency and matching building polygons. The two main sources are the Dutch national register of addresses and buildings (Basisregistratie Adressen en Gebouwen/BAG), which includes building polygons, and the height data from the official Current Dutch Elevation map (Actueel Hoogtebestand Nederland/AHN) that is based on aerial Lidar point-cloud data. “The Lidar data is currently only updated once every three to five years, so we are also experimenting with photogrammetric point clouds generated by dense image matching. The results are not yet as good as Lidar-based models. Therefore, we are further investigating how to improve these reconstruction results, such as by extracting the rooflines from the generated true orthophotos instead of from the generated point clouds, and combining them with the point clouds generated from the images in the 3D reconstruction process,” states Stoter.

“It’s important to recognize that 3DBAG in its current form would not have been possible without the high-quality geoinformation that is collected and maintained for the whole country by the Dutch government and made available as open data. I believe that the Netherlands is unique in providing such easy access to so many geospatial datasets without any restrictions, and it was tremendously useful during the development of our automatic building reconstruction algorithm. The fact that the data sources have an open licence also allows us to offer the 3DBAG as open data, ensuring that our solution is freely available and easy to use,” she continues.

Aimed at practitioners

Right from the start, the developers have aimed to tailor 3DBAG to users’ needs. “Besides providing different levels of detail, we’ve added attributes that appeared to be useful for users, such as building volume, party-wall areas, and roof-, wall and floor areas. We developed these energy-relevant attributes for buildings in a project together with the Netherlands Enterprise Agency (RVO),” comments Stoter. One of the added attributes relates to the data quality. “Even though we work with high-quality source data and strive to generate valid models, around 0.5% of the buildings in 3DBAG still have issues – for example, due to problems with the input data, which is out of our control. Therefore, we run our 3D models through val3dity, a tool also developed in the 3D Geoinformation research group for the validation of 3D primitives according to the international standard ISO19107, so that users don’t need to validate the 3D data themselves. We then include the validation outcome as an attribute. That way, users are able to see at a glance whether they can use the data about that particular building with confidence or not,” Peters adds.

The team also realized it was important to make the data easily accessible, to encourage practitioners to try using it for their applications, according to Peters: “We developed a 3D web viewer largely from scratch. We also make the data available in formats that people are comfortable with, such as GeoPackage, Wavefront OBJ, WMS and WFS. There is now also an online API that serves CityJSON, which is our primary distribution format.” CityJSON is an OGC community standard that was also developed within the 3D Geoinformation research group. “With 3DBAG, we became heavy users of it ourselves, which now helps us to make further improvements to CityJSON,” he explains.

Dealing with exceptions

Needless to say, the development project was not without other challenges. “It definitely wasn’t easy to scale up our pipeline to work for the whole of the Netherlands. When you’re processing 10 million buildings in total, you are guaranteed to find some exceptional cases that break your algorithms,” states Peters.

For example, the team quickly discovered that the large number of commercial greenhouses in the Netherlands created the need for extra attention and modifications to the building reconstruction algorithm, he recalls: “Greenhouses have a very large footprint and a simple shape, but it is difficult to capture the height consistently with Lidar because the laser often penetrates their glass roofs, resulting in large areas without points on the roof in the AHN. This not only led to poor reconstruction results but also turned out to slow the algorithm down significantly. To avoid these issues, we fall back to a simplified 3D model for such greenhouses in order to generate usable models.” In the case of other exceptions, such as underground structures that are included in the building polygons or one building that is ‘floating’ above another, the team ensured that these cases are handled during the reconstruction process (see Figures 3 and 4).

Updates to source data

Another challenge related to the impact of updates to the source data. “3DBAG was initially based on AHN3. When AHN4 was released, we thought it made sense to use that dataset because it would provide the most up-to-date information,” says Peters. “However, we discovered that the new version of AHN had some different properties, and they were not always better for modelling buildings. For a small yet significant number of buildings, AHN4 turned out to have poor data consistency.” This resulted in data gaps which had a negative impact on the reconstruction result (see Figure 6). The team subsequently spent a few months investigating how to get the best out of both AHN versions including in projects with the City of Amsterdam and RIVM. “One idea was to use deep learning to improve the AHN4 point cloud, and another was to use point cloud fusion. Eventually, it was decided to adapt the algorithm to automatically check for data gaps in the new dataset and fall back on the older version if an issue is detected, on the condition that the building had not been changed since the older version. This was considered the most maintainable and efficient solution in terms of user relevance and feasible reconstruction times,” continues Peters.

User relevance has always been a strong factor in the 3DBAG project, and the interplay between the data experts in the research group and practitioners in the field has been key to understanding and aligning with their needs. “For example, when developing the 3D noise data, our focus was initially on LoD2. However, after starting to work with noise professionals, we realized that their simulation software wasn’t actually suitable to handle the highest level of detail; a lower level of detail, namely LoD1.3 containing noise-specific details, was the right solution in this case. That’s the kind of thing that you only learn through intensive collaboration with domain experts,” explains Stoter.

Examples of how 3DBAG data is being used in practice

3DBAG data has been integrated by government organizations, businesses and academia as input and a building block for all kinds of real-life applications. Some of the known applications include:

• Estimating energy demand and consumption in buildings
• Calculating renovation and retrofitting costs, and checking insurance or subsidy claims
• Finding suitable roofs for solar panels
• Simulating the wind flow and pollutant dispersion in urban areas based on computational fluid dynamics (CFD)
• Calculating heat stress and noise pollution (e.g. 3D Noise) in urban areas
• Identifying development potential for rooftops
• Predicting and assessing the impact on buildings in case of subsidence or earthquakes
• Analysing the urban structure and evaluating new housing developments
• Digital twins, e.g. 3dkaartvannederland.nl, netherlands3d.eu and 3dtilesnederland.nl

For more examples, see docs.3dbag.nl/en/overview/media

Spinoff to support implementation

After their contracts as researchers ended, the two lead developers of 3DBAG founded the company 3DGI as a spinoff. This enabled them to fully devote their time to ensuring the accessibility of 3DBAG beyond the scope of academic research projects. This includes offering consultancy and software development for government organizations and commercial companies who need a little extra help with utilizing or adapting the data from 3DBAG.

“We still collaborate closely with the 3D Geoinformation research group on 3DBAG,” says Peters. “Having done most of the development work, we are happy to share our ideas for new functionalities or improvements, and we enjoy giving something back to the community.” Stoter confirms this: “3DBAG is still actively used, investigated, and further developed in research projects within our research group, driven by users’ and organizations’ needs. For example, we have explored adding new features to the reconstruction models, such as textures, doors and windows. Many of our MSc students also investigate innovations for the 3DBAG. If those results are good, they are included in the next version of the 3DBAG, such as the estimation of the number of floors inside each building based on machine learning.”

Open data

Besides taking account of practitioners’ needs, another reason for the wide uptake of the data is the availability of 3DBAG as open data. “For this data to truly make a difference in practice, such as by contributing to the energy transition, the climate adaptation strategy or accelerating the planning, design and construction process of new housing, the availability of 3DBAG as open data is crucial,” she continues. “This encourages and accelerates adoption of the use of 3D data in practice. Open 3D data covering a whole country stimulates further innovations, including in new domains. And that is where the strength lies: in providing reliable, fit-for-purpose, automatically generated data on the whole of the Netherlands, accessible without any restrictions, for anyone who wants it.”

“3DBAG ensures that essential information is available to all, driving real-world impact without the sole reliance on a specific commercial solution. The fact that it is already being used for so many different applications (see box, Ed.) and we’re receiving so many positive reactions shows that there is demand for this. At the same time, we are happy to see companies embedding 3DBAG in their enriched data products, leveraging the use of 3DBAG’s free data even further,” says Stoter.

Platform for future innovation

However, free data does not mean that there are no costs associated with 3DBAG. With this in mind, one of the focus areas is now also on ensuring the necessary funding to safeguard the continuity of 3DBAG. “Initially, it was funded by research projects that required proof of scientific concepts. But 3DBAG has now grown beyond this status into a frequently used data source for practitioners. A more structural funding is required to keep 3DBAG available as a solid, reliable data source and to support its maintenance and further development, independent from individual research projects. As part of this, we are also working to achieve the level of robustness that is required for long-term use and further development and maintenance,” comments Peters.

These software improvements carried out in a collaboration between the 3D Geoinformation research group and 3DGI are financed by Kadaster via the ‘Working on Implementation’ (WaU) government funding programme. “This one-time investment will take the software from a research software to a status that it can be maintained and reused. Besides this, we are in the process of setting up the 3DBAG Innovation Platform to ensure the maintenance and further innovation of 3DBAG. The platform is coordinated by Geonovum, and is an initiative with six other partners: RVO, RIVM, the City of Amsterdam, the City of Utrecht, Kadaster, and Waterschapshuis. This collaboration has provided some seed funding for setting up the platform. Explorations are still ongoing for more structural funding within a national context,” comments Stoter. One ambition is to include the stable versions of 3DBAG in the 3D data products of Kadaster and to give it a clear role in the ‘Zicht op Nederland Data Fundament’ (3D data foundation for the Netherlands). “3DBAG is automatically generated and therefore different from other datasets considered as base data. However, being a solid and reliable dataset, used by many organizations, it is useful to further investigate how such datasets can become part of the NSDI,” she states.

Meanwhile, Peters is continuing to work on the software. “This solution was developed with the Dutch source data in mind, which meant that some of the underlying data properties became assumptions for our software to work with. We are slowly eliminating those assumptions to allow the underlying 3DBAG data pipeline to be used by other developers internationally, including the possibility of using other data sources. Several public and private organizations from abroad have shown interest in the 3DBAG software, and some are testing it with their own data. We hope that our work will encourage and inspire further innovation for many more applications around the world in the future,” he concludes.

More information

www.3DBAG.nl