The remarkable capabilities of airborne Lidar continue to improve, with scanners now able to emit more than a million pulses every second. Although it has never been easier to acquire high-quality laser data, storing and managing the huge volume of Lidar data collected can prove to be a challenge. Combined with associated metadata such as survey information, environmental conditions and flight lines, data management has now become a critical priority and focus. This article outlines a new initiative called LMAP that provides an automated workflow for uploading and storing Lidar and metadata into GIS.
Developed in a partnership between RIEGL of Austria, a manufacturer of ultra-high-performance Lidar scanners, and the USA’s Esri, creator of the ArcGIS platform, the Lidar Management and Analytical Processing (LMAP) initiative provides a new, automated workflow to upload and store Lidar and metadata into GIS. Once organised and managed using standard GIS functionality, LMAP utilises web applications to visualise the information on a map and perform a range of analysis. As a user of both RIEGL and Esri technologies, the North American geospatial-only solutions provider Quantum Spatial Inc. (QSI) decided to employ an LMAP implementation to improve project data management and provide a venue for clients to quickly and easily provide feedback on Lidar collection quality and completeness.
Lidar is a survey measurement technique that uses light in the form of emitted laser pulses to measure ranges to the Earth, resulting in accurate three-dimensional (3D) models of Earth. These models have a wide variety of uses and applications, including in engineering, town planning, mining, archaeology, computer vision and environmental monitoring. Bathymetric Lidar is a Lidar scanning technology that penetrates the water column to measure seafloor depths. To map Chesapeake Bay in the USA for a client, QSI selected RIEGL’s VQ-880-G, a fully integrated airborne scanning system for combined hydrographic and topographic surveying. Offered with an integrated GNSS/IMU system and camera, the VQ-880-G also houses both a green laser that penetrates shallow water for seafloor measurements and an infrared laser for improved capture of the water surface. The result is millions of Lidar points collected of the ground, vegetation, water surface and seafloor.
Results of a bathymetric Lidar survey are dramatically impacted by the environmental conditions at the time of acquisition, specifically water quality and turbidity. To record these conditions and to validate the Lidar, field personnel take measurements during the collection and monitor water-quality data transmitted from nearby buoys.
Timely client feedback
In February 2018, QSI collected topo-bathymetric Lidar and aerial photography over the extensive river delta regions of Chesapeake Bay within the states of Virginia and Maryland to generate improved shoreline and bathymetry data. This data provides highly accurate information of the bay’s geographic features for official shoreline characterisation, nautical charting, geodesy and marine resource management assessments. Environmental factors such as water quality and weather were closely monitored to ensure data collection occurred during peak windows to guarantee the collection of the highest-quality data possible. Additional information was also collected, such as aircraft flight lines, survey ground control locations and photographs from the flight. All of this information was shared with project stakeholders in different locations using a variety of computing devices.
Getting timely client feedback on the quality and acceptance of the Lidar data collected is a critical success factor in these types of projects. Long turnaround times can quickly lead to inefficient use of field teams and wasted money. Traditionally, QSI has relied on a myriad of technologies such as email, File Transfer Protocol (FTP), blogs, PDF reports, project status software and public websites in order to share data and receive client feedback. Each of these technologies have constraints and limitations including file size, data formats and bandwidth. Big data needs present a growing challenge for mapping companies such as QSI and the clients it serves, especially as advances in technology generate more and more data. To address these challenges, QSI partnered with RIEGL and Esri using the Chesapeake Bay project data and workflows to discover new ways to efficiently manage large volumes of geospatial data.
Web portal prototype
By leveraging the Esri platform and RIEGL’s LMAP concept, QSI prototyped a mapping web application known as the QuickLookViewer to enable project stakeholders to make and share decisions about the quality of the collected Lidar data during the acquisition phase of projects. This single web ‘portal’ provides access to preliminary Lidar data, derivatives such as the digital terrain model (DTM) and digital surface model (DSM) and all associated environmental data needed to support actionable decisions during acquisition. This has allowed QSI to replace numerous cumbersome workflows and methods.
Several products in the Esri platform were used to develop the QuickLookViewer (QLV). For example, ArcGIS PRO was used to create the Lidar-derived raster images as well as the scene layer packages, which were then organised as mosaic datasets. The project map was developed in ArcGIS Online (AGOL) and combined data layers from Collector for ArcGIS with geotagged oblique photos taken from the aircraft, project flight lines, environmental data from public websites and mosaic datasets. ArcGIS Server was used to host and publish the mosaic dataset. Finally, Web App Builder was configured to combine the AGOL project map with tools for viewing, analysing and inserting markups for feedback. Lidar point clouds were published as scene layer packages to ArcGIS Online and embedded as hyperlinks for 3D viewing in a web browser.
The complete workflow from source data to QuickLookViewer is shown in Figure 3.
Esri’s ArcGIS platform offers a variety of options to display and visualise 2D and 3D data including the key derivative terrain and surface models created from the bathymetric Lidar:
- ‘Low Hit’ or digital terrain model (DTM)
- ‘High Hi’ or digital surface model (DSM)
- Depth (created by subtracting the Low Hit from the High Hit) is used to represent bathymetry coverage and provide a visual representation enabling the identification of data gaps as well as a quantifiable representation of approximate depth extent.
Since these data layers are of significant importance to the client, they were made easily accessible in the QLV LMAP map interface. In addition to displaying each layer separately, a tool allowing the user to ‘swipe’ between any two layers created a simple way to do comparisons between layers. Since each data layer retains its intelligence (i.e. each pixel has a ‘z’ value), the user can click on any location and determine the depth or height above ground.
In addition to the swipe tool, several other tools were added to the web application to help the client provide feedback to QSI. These tools included:
- Draw (or mark up): The ability to mark up the map with text or symbols
- Measure: Perform both linear and area measurements, enabling easy quantification of areas of interest on the map
- Bookmarks: Enable the saving of an exact location and zoom level on the map, providing others with an easy way to revisit a view.
To enable the client to visualise the Lidar point cloud in 3D, QSI leveraged one of Esri’s newest technologies: the scene layer package (SLPK). SLPK utilises the ‘Indexed 3D Scene’ layers (I3S) specification, an OGC community standard, and is designed to allow large 3D datasets for use in mobile, desktop and server-based workflows to be accessed over the web or as local files.
QuickLookViewer was developed by one senior GIS analyst with minimal experience in programming. The part-time development occurred over a ten-week period, with the majority of the time spent understanding the source data and required workflows to implement the final product. The development caused no disruptions to production as the majority of the input data was obtained from outputs of existing workflows.
The development of the Esri/RIEGL LMAP web application prototype demonstrates the viability of leveraging the ArcGIS platform to store, manage, analyse and visualise the many diverse datasets required for a bathymetric Lidar project, as demonstrated in this specific case study with QSI. Utilising a single map interface gives users the ability to easily compare and interrogate the derivatives of the Lidar data such as a depth map, review photos and water-quality information and provide collection-quality feedback to the vendor in a timely manner. The next step will be to work with the client to refine the QLV LMAP implementation and to begin deployment for day-to-day operations.