The Santiago Urban Dataset (SUD) is a new, composite dataset merging handheld mobile laser scanning (HMLS) and mobile laser scanning (MLS) point clouds. With eight distinct classes, it has been developed using data fusion techniques to effectively handle occlusions from parked vehicles and street furniture. Evaluation of SUD shows that it is well-suited for comparative assessments of new deep learning models targeting semantic segmentation tasks, significantly extending its potential applications.

Point cloud data is widely used in urban applications for object detection and classification, with deep learning methods as the state-of-the-art approach. MLS captures large datasets efficiently, but frequently contains occlusions – particularly from parked vehicles, limiting insights into pedestrian urban environments. SUD overcomes this limitation by correcting occlusions by combining point clouds from MLS and HMLS (Figure 1).

Comparison to existing datasets

SUD is a comprehensive real-world dataset covering 2km of streets in Santiago de Compostela, Spain. The dataset undergoes a meticulous manual labelling process, supported by heuristic techniques and deep learning methods.

While some datasets only differentiate between road and sidewalk, SUD goes further by classifying occlusion-free sidewalks, roads and curbs. Additionally, it incorporates other key classes relevant to urban mobility: buildings, vehicles, vegetation, poles and others. This results in a dataset with eight specific classes. Table 1 compares the characteristics of some of the 3D point clouds datasets, sorted chronologically, available in outdoor environments.

Equipment

Point clouds were obtained with a mobile laser scanner and a handheld mobile laser scanner. A RIEGL VUX-1HA MLS scanner was used to acquire the point clouds from a car perspective, while the HMLS ZEB-GO scanner was used to scan the urban environment from a pedestrian perspective. Table 2 shows the technical characteristics of the RIEGL VUX-1HA and ZEB-GO systems.

Scanned area

The survey was conducted in Santiago de Compostela (Spain). The dataset contains 14 segments of six urban streets. Each segment measures approximately 200m. Therefore, the dataset is formed by 1.6km of MLS point clouds and 2km of HMLS point clouds of urban streets (Figure 2).

Labelling workflow

Manual labelling was conducted with CloudCompare, which was combined with automatic processes to reduce the human effort. The following rules were established to obtain uniform and consistent labelling:

• All the points behind the facade line were considered as building points, including flooring of entrances, doors, shop windows, etc.
• Only stationary cars and vans were considered as vehicles.
• Motorbikes, large trucks or any vehicles in motion were classified as others.
• Any element in the ground space that was neither trees nor stationary cars/trucks was classified as others.

First, 200m of MLS point cloud data was manually labelled into five classes (ground, building, vehicles, vegetation, and others) to train PointNet++ for preliminary classification of the remaining data. A k-Nearest Neighbors (KNN) algorithm was then applied to label HMLS data based on proximity to MLS data, followed by manual corrections to address misclassifications. Ground points were further divided into road, sidewalk, and curb using an algorithm that analysed inclination, curvature, and clustering (DBSCAN). Horizontal ground elements were manually separated, and HMLS ground points were classified using KNN with manual corrections. Pole-like elements were extracted from the others class using DBSCAN and height/width filters, with KNN applied for HMLS classification and additional corrections (see the workflow for labelling MLS and HMLS point clouds in Figure 3). Ultimately, both MLS and HMLS data were classified into eight classes: road, sidewalk, curb, building, vehicles, vegetation, pole-like elements, and others (Figure 4).

Baseline

As a baseline approach, a PointNet++ model was tested on the proposed dataset. Given the existence of two classified data sources, three tests (and thus three training sessions) were performed: one with only MLS point clouds, another with only HMLS point clouds and a third using both MLS and HMLS point clouds (Figure 5). To assess the performance of the model in each case, several metrics were employed (see Table 3 data).

The behaviour of PointNet++ in the predictions was as expected. The best identified classes were those corresponding to the largest number of points and distinctive geometries (road, sidewalk, buildings, vehicles, and vegetation). It is also worth noting that the classification with MLS data was better than with HMLS, which may be explained by the lower quality of the HMLS point cloud. On the other hand, the union of MLS and HMLS did not improve the classification but avoided occlusions in sidewalks.

Conclusion

This study introduced a new urban point cloud dataset known as the Santiago Urban Dataset. SUD was labelled with eight distinct classes: road, sidewalk, curb, buildings, vehicles, vegetation, poles, and a category denoted as others. Notably, SUD is a composite dataset, merging HMLS and MLS point clouds. One of the key advancements achieved in this work was the utilization of data fusion techniques to effectively handle occlusions introduced by parked vehicles and street furniture.

The PointNet++ model was selected as a baseline approach for evaluation, and several pertinent metrics were introduced to assess its performance. The outcomes of this evaluation were notably consistent with those observed in other state-of-the-art works. Both the quantity and spatial distribution of errors align with established benchmarks, affirming that the SUD dataset is well-suited for comparative assessments of new deep learning models targeting semantic segmentation tasks. Beyond its applicability in model comparisons, the broad scope of the survey – coupled with the integration of HMLS and MLS data – significantly extends the potential applications of SUD. This dataset can serve as a valuable resource for urban mobility or urban planning studies. In the future, the authors will test other deep learning models on the dataset as well as new architectures. Different frameworks to integrate and fuse HMLS data into MLS data will also be studied.