Producing High-quality 3D Point Clouds from Structure-from-Motion Photogrammetry

Anyone involved with structure-from-motion (SfM) photogrammetry has probably questioned which type of processing software is necessary for them to produce quality 3D point cloud data. Many SfM software packages exist that allow for the generation of 3D point clouds from image sequences acquired via unmanned aerial systems (UASs). Each of these software packages provides different workflows and parameters for implementation of SfM processing and for point cloud densification using multi-view stereo (MVS) algorithms. To examine the influence of terrain on the results, this study compares differences in point clouds generated by three different types of SfM software. 

It is well known that SfM/MVS point cloud fidelity is influenced by several factors, including terrain characteristics (e.g. surface texture). The influence of terrain on SfM software solutions is difficult to characterise and not well examined. As an attempt, this study compares differences in point clouds generated by three different types of SfM software: two well-known commercial packages, Pix4D and Agisoft PhotoScan, and an open-source set of tools called OpenDroneMap.

Terrain types

Five terrain types were imaged utilising a DJI Phantom 3 Professional UAS. These terrain types included a marsh environment, a gently sloped sandy beach and jetties, a forested peninsula, a house, and a flat car parking area. The house survey was the only survey that provided an oblique-perspective 3D point cloud comparison. Each set of imagery was processed with each software package and the point cloud results were directly compared to each other.

Each drone survey was automated so that human error could be minimised during the image collection process. The Pix4D Capture application was chosen for flight automation because it gives the user different options of flight paths (grid mission, double grid mission, circular mission) and offers a larger range of flying heights. Other options that can be controlled with this application are speed, image overlap and camera angle.

Data processing

The first step of the data processing was to verify that all three software packages utilise the same camera calibration model. The camera calibration model is responsible for triangulating the derived 3D points based on the internal camera parameters. The camera model used for this study was a typical five-parameter camera model (R1, R2, R3, T1, T2) where R1, R2 and R3 describe the radial distortion while T1 and T2 describe the tangential distortion. After the calibration model was confirmed, the software settings were analysed and chosen in a manner that allowed for the most similar settings to be set across the three software packages based on recommended or default settings for the version of software at the time of this study (autumn 2016). This was done in an attempt to minimise point cloud differences caused by dissimilar settings.

A total of 15 point clouds were generated: five terrain types each processed by three different software. The three point clouds of a common terrain type were compared against each other to better understand how each SfM software package handled different terrain types and to see how the quality of the point cloud data differed. The following characteristics and statistics were recorded for each point cloud: the density, the average spacing of the points, and the total number of points as well as the minimum, maximum, mean and standard deviation of the X, Y and Z coordinates. Tables 1 to 4 summarise the recorded statistics.

Point cloud density

When looking at these tables it is easy to see that Pix4D and Agisoft PhotoScan stand out as being very close in the comparison between density, spacing and number of points. When considering the number of points, the following percentages were determined. For the marsh experiment, Pix4D produced 7.3% more points than Agisoft PhotoScan and 99.01% more than OpenDroneMap. The beach process resulted in 15.58% more points than Agisoft PhotoScan and 99.2% more than OpenDroneMap. For the peninsula survey, Pix4D created 53.4% more points than PhotoScan and 99.9% more than OpenDroneMap. This is due to false matching issues over water that Pix4D handled better based on the settings utilised. PhotoScan could likely be reprocessed to resolve this concern. Out of all the surveys, the car park survey resulted in the highest-density point clouds among all three software packages. Pix4D generated 15.03% more points in the car park than PhotoScan and 98.8% more than OpenDroneMap. The difference between Pix4D and PhotoScan was due to more densification obtained by Pix4D in short, grassy terrain surrounding the car park (see Figure 1). In contrast, PhotoScan outperformed Pix4D in point density for the oblique-perspective house survey with 26% more points and 96.6% higher than OpenDroneMap.

Standard deviations

The standard deviations of the X and Y coordinates were all relatively close to each other, showing that there is not much horizontal variability between the point clouds. The marsh survey resulted in Agisoft PhotoScan and Pix4D having very similar mean vertical coordinates, with average separation of only 8 centimetres, while OpenDroneMap placed its point cloud over 65 metres above the other two. When relying only on the GPS onboard the UAS, one can expect to see large differences between point clouds along the Z axis. Similarly, the beach and jetties survey reveals a large vertical separation between Pix4D and the other two packages by an average of 15.65 metres. For the car park survey, ground control targets were utilised for absolute accuracy assessment. From this survey, Agisoft PhotoScan had a mean Z value of 1.53 metres, Pix4D had a mean Z value of 1.54 metres and OpenDroneMap had a mean Z value of 1.47 metres. This resulted in a range of 0.07 metres between the three, and only a 1-centimetre difference in mean Z values between PhotoScan and Pix4D (Z here is based on North American Vertical Datum 88).

The statistics uncovered some interesting facts about the house survey, which used higher image overlap (90%) and lower altitude (20m) relative to the other surveys. Interestingly, the mean Z values were 168.03 metres for Agisoft PhotoScan, 167.79 metres for Pix4D and 168.86 metres for OpenDroneMap, giving a range of 1.07 metres between the three. Considering the extreme differences along the Z axis seen in the other surveys without ground control, this is surprisingly low. A higher image overlap and higher image resolution, along with oblique perspective imagery, probably aided each software package in more uniformly resolving points within the X, Y and Z coordinate space; since this would likely result in a higher number of images per matched keypoint during the initial processing, it would help better solve the location of points and camera exterior orientation across all three software packages. Scene texture of the built environment could also play a part.

Open source

Pix4D created the point cloud with the highest density for four out of five surveys. The point clouds produced by Agisoft PhotoScan were found to be less dense and have fewer points than those in Pix4D’s results, except for the oblique-perspective house survey. Nevertheless, the average point spacing of the point clouds was 0.016 metre, with the closest spacing being 0.01 metre and the furthest being 0.02 metre. This demonstrates the understanding that even though PhotoScan produced a less dense point cloud than Pix4D, it maintained a very close point spacing. OpenDroneMap is an open-source project which means two things need to be considered. Firstly, this is an ongoing project where people voluntarily contribute to the improvement of the software. As time goes on, assuming that people continue to aid in the development, OpenDroneMap should become more proficient, more accurate and more user friendly. Secondly, OpenDroneMap is free to use. Having a software package that is capable of producing point cloud data and orthomosaic images at no cost to the user is not something to be taken lightly.

Concluding remarks

Each software package provides a more or less robust set of processing parameters that can be tuned for a specific project. Inherent differences between the processing parameters utilised across the software are responsible to a degree for the differences observed in this study. While this work aimed to minimise the influence that differing parameters had on the results, some differences are unavoidable. Iterating through different sets of parameters for each software package could likely have resulted in different or better results for each of these surveys. This, however, was not the intent of the work. Rather, this study aimed to compare SfM software solutions in order to gain a greater understanding of UAS-SfM point cloud uncertainty over different types of terrain. Furthermore, this work in no way attempts to endorse or recommend one software package over the others. The results are reported as is, with the caveats as above. In fact, the authors are avid users of each software solution examined here. Finally, the important take-home point is that point cloud quality of UAS-SfM software solutions is driven by many factors, of which one is terrain.

Table 1, Point cloud characteristics compared between the point clouds derived by three different SfM processing software packages.

Table 2, A comparison of the minimum, maximum, mean and standard deviation of the X coordinate between the point clouds derived by the three different SfM processing software packages.

Table 3, A comparison of the minimum, maximum, mean and standard deviation of the Y coordinate between the point clouds derived by the three different SfM processing software packages.

Table 4, A comparison of the minimum, maximum, mean and standard deviation of the Z coordinate between the point clouds derived by the three different SfM processing software packages.