Who Should Define DEM Quality and How?

Who Should Define DEM Quality and How?

Exploring a Variety of Quality Assessment Methods

The diversity of relief mapping methods and the existence of large elevation databases intended for multiple users raises the issue of DEM quality. Digital elevation model (DEM) accuracy is commonly assessed by comparing it to reference data. When no data is available, one can look for inconsistencies in the DEM since the land relief is not a completely random object. These quality criteria correspond to different needs, and producers must therefore take into account a variety of users. The question then arises as to who should specify the quality of DEMs between the producer and the user, and how. This article explores a variety of quality assessment methods, from digital elevation model to global altimetric database.

Like any industrial product, digital elevation models must present a quality that meets both the technical constraints of production and users’ requirements. A major difficulty of DEM quality assessment is that it is a product that serves a wide variety of users, depending on the information required from the DEM. In addition, due to our experience and our senses as humans, the relief is a familiar object and we place such high demands on its representation that any inconsistency may be unacceptable. However, users (scientific or operational) do not always know how to express their needs. In fact, users’ needs are often over-specified, which means that the requirements are disproportionate. This phenomenon is encouraged by the availability of very high-resolution DEMs. Beyond this risk of over-specification, it is not easy to define an explicit and quantitative indicator to measure the degree of agreement of the data with the information sought in terms that producers could easily take into account to optimize the production. This is even more critical for large altimetric databases, in which a single production method must satisfy a variety of users on a variety of landscapes in terms of relief, land cover, climate, etc. Therefore, who should define the quality of DEMs and how? 

Overcoming difficulties

DEM quality is often assessed in a rudimentary way, by analysing the statistical distribution of elevation discrepancies. The most common indicator is the root mean square error (RMSE). This combines the bias and the standard deviation to verify that the model is accurate, which means that it is never very far from reality. It is, however, a rather poor approach, because it does not consider the spatial variations of the altimetric error. Therefore, it is not sensitive to the quality of restitution of the landforms, which is essential for geomorphology and related studies. The respect of landforms depends on the accuracy of the slope, the aspect and the curvature, which are derivatives of the elevation. However, estimating the accuracy of slopes from reference data has to face another difficulty, which is that the slope value depends on the DEM scale (mesh size). To overcome these difficulties, DEM quality assessment can also rely on a so-called internal validation procedure which does not use reference data but instead looks for inconsistencies (improbable or even impossible landforms) in the DEM.

Diversity of quality criteria

Internal validation extends the principle of visual control, which makes it possible to search for implausibilities in a DEM visualized in a suitable way. For example, a mere shading makes it possible to highlight resampling artifacts like noise, lineage or excessive smoothing. The same principle can be implemented in a more quantitative way by analysing the compliance of the DEM to several general rules that the terrestrial relief is supposed to fulfil everywhere. Two kinds of rules can be distinguished. On the one hand, physical rules lead to strong requirements and a terrain that does not fulfil them is impossible. For instance, water always streams downwards, and wherever it is not the case (e.g. presence of sinks), a DEM error is identified locally. On the other hand, statistical rules lead to weak requirements, and a terrain that does not fulfil them is improbable although not impossible. For instance, as a consequence of the fractal behaviour of most hydrographic watersheds, the number of streams of a given Strahler order decreases linearly with the increase of the stream order (Horton’s law). Similarly, the statistical distribution of slopes, aspects, curvatures and also stream lengths or watershed areas can reveal inconsistencies in the DEM.

There is, therefore, a great diversity of quality criteria, which relate to absolute positions as well as to landforms. The same criteria can be used to define the quality of products derived from the DEM. For example, the hydrographic network can be affected by geometric errors (absolute location errors and unrealistic shapes) caused by the errors of the DEM from which it is extracted and by the network extraction algorithm. In the case of a vector network, topological errors can also be identified (Figure 1).

Figure 1: Main types of errors in the drainage network extracted from a DEM (blue line: real network; red line: network extracted from the DEM).

It should be noted that a quality criterion is only meaningful if the nominal terrain, i.e. the surface that the DEM is supposed to describe, has been clearly specified. For example, a digital terrain model (DTM) describes the ground while a digital surface model (DSM) describes the canopy in forest or the roofs in urban areas.

Whatever the criterion, the quality of a DEM must be evaluated on the basis of the requirements foreseen for the application. However, these requirements are not always easy to express with criteria useful to the DEM producers, particularly in the case of multi-user elevation databases that must reconcile different quality criteria. In practice, due to the difficulty for DEM users to express quality requirements, the quality of a DEM is often specified by the producer and the quality indications highlight the product or method rather than informing users about limitations of which they should be aware.

DEM quality prediction

Beyond a posteriori quality assessment of an existing DEM, producers must be able to predict the quality they may expect before the production of the DEM, depending on the restitution method used and on the characteristics of the area (climate, relief, land cover). The quality of the DEM can be predicted to a certain extent as it is affected by the parameters of image acquisition (the instrumental and orbital characteristics of the imaging sensor) and processing, based on theoretical equations than can predict quality indicators like the altimetric standard deviation. Although this approach can help to select the best set of acquisition and processing parameters, it remains limited because the output DEM quality also depends on the interpolation process and on the heterogeneous characteristics of each area. Moreover, some of the parameters in the theoretical equation have a well-known but unpredictable influence. This is the case of the baseline in repeat-pass SAR interferometry, which is impossible to predict before image acquisition, so that the DEM accuracy is impossible to predict before the production. Therefore, predicting the performance of a DEM production method requires experience based on the analysis of numerous DEMs.

Figure 2: Two common options for the nominal terrain.


Understanding the phenomena that degrade the quality of DEMs can also lead to improvement methods, either during the production or a posteriori. Reference data can be used to correct an absolute bias or to remove tree height in forested areas. When no reference data is available, artifacts can be removed to force the DEM to comply with some physical or statistical rules, e.g. by filling sinks to force rivers to stream downwards, or resampling the DEM to filter noise or inversely to create a realistic variability by kriging or fractal synthesis. These DEM improvement possibilities open up new opportunities to reduce costs by using average-quality DEMs that can be improved in accordance with user requirements based on reference data or geomorphological assumptions.

Further reading

Polidori L. & El Hage M. (2020) Digital elevation model quality assessment methods: a critical review. Remote Sensing, 12, 3522, DOI : 10.3390/rs12213522.

Make your inbox more interesting.Add some geo.

Keep abreast of news, developments and technological advancement in the geomatics industry.

Sign up for free

Keep abreast of developments in the geospatial industry

We encourage you to subscribe to our weekly newsletter. Subscribers also receive a digital copy of our bi-monthly magazine.