The rise in UAV inspections for civil infrastructure

UAVs with a higher level of autonomy can make civil engineering inspections safer, faster and more accurate, plus they support the monitoring of infrastructural changes over time.

The increasing development and production of unmanned aerial vehicles (UAVs or ‘drones’) with a higher level of autonomy can reduce human error and assist in safer, faster and more accurate inspections. Thanks to their ability to perform repeated inspections, even in challenging and dynamic environments, they make it easier to monitor infrastructural changes over time. This will benefit several potential applications, such as civil engineering inspections.

To ensure sufficient inspection outputs, a conventional UAV mission needs to be controlled by a highly skilled pilot or operator. This professional should be aware of what kind of information the inspectors need and adapt to any on-site difficulties such as obstacles, time of day and weather conditions. After the UAV flight mission is finished, a large number of acquired images are processed. Whether the images are used in visual inspection or 3D model generation, both cases require significant computing power and experience. The recent development towards autonomous inspections using conventional UAVs equipped with either image-based or Lidar-based tools can certainly reduce human error and assist in a safer, faster, more accurate inspection generating data that is accessible online. Furthermore, UAVs can perform repeated inspections even in challenging and dynamic environments, making it easier to monitor infrastructural changes over time.

UAV autonomy levels

It is recognized that automation can improve efficiency, and efficiency can improve success. There is a strong correlation between automation and autonomy. As shown in Figure 1, the levels of autonomy for UAVs can be defined in a similar way as the levels for autonomous cars as suggested by the National Highway Traffic Safety Administration:

Level 0 – No autonomy: Manual flight, with a pilot controlling all the UAV flight functions from take-off to landing. Today, it is rare to see UAVs without some automation functions.

Level 1 – Low autonomy: Some functions are applied autonomously, such as autonomous stabilization, although the pilot maintains full control of the mission. 

Level 2 – Partial autonomy: Several functions are running autonomously but the pilot can have full control of the mission. Many UAVs currently offered in the market can be classified at this level. They use various inertial measurement units (IMUs) and global navigation satellite system (GNSS) receivers.

Level 3 – Conditional autonomy: The majority of flight functions can be run autonomously, such as collision avoidance, while the pilot only selects the object of interest and monitors the flight. However, the UAV has no decision-making ability at this level of autonomy. 

Level 4 – High autonomy: The flight functions are running autonomously but the pilot can intervene if necessary. The UAVs at this level of autonomy have the functionality to identify and track features using artificial intelligence (AI), avoid obstacles and plan the best flight path. Some pioneering UAV companies are currently competing at this level.

Level 5 – Full autonomy: This is the highest level of autonomy with no pilot intervention. UAVs are expected to make decisions independently and learn from the observations they make.

In general, it can be concluded that the technology is currently between Level 3 and Level 4, with a significant level of pilot control still required. The development of miniaturized processing units is driving higher automation by increasing the capacity for data to be processed on board the platform. However, Level 5 autonomy has not yet been reached for UAVs and it is associated with many discussions and concerns about regulations, traffic conditions, security, safety and privacy.

Key technologies enabling more autonomous inspections

Recently, a lot of research work has been published relating to autonomous inspections based on computer vision and AI. This can be split into two major lines of research: a) the design of autonomous flight plans, and b) automated data extraction. Figure 2 shows the technical factors driving this development, including:

• Automated path planning
• Collision avoidance
• Simultaneous localization and mapping (SLAM)
• Onboard processing units able to process the data in near real time
• Advancements in deep learning.

Accordingly, in autonomous inspections using UAVs, the software can plan an optimal flight and can then process the data, identify any problems in specific target objects such as wind turbines, and further build precise 3D models of those objects.

Path planning: Several algorithms from the robotics domain have been adopted to plan flights in complex environments. They calculate the most efficient way to optimize the flight plan considering a specific target object to be surveyed or a destination point to be reached. The traditional flight planning algorithms have been recently supplemented by deep learning solutions too.

Collision avoidance: More reliable collision avoidance techniques have been adopted on several commercial drones allowing them to sense and avoid static and dynamic obstacles. Currently, consumer-grade compact Lidar, ultrasound sensors are installed onboard for this task. In some solutions, rough maps are computed onboard using SLAM to explore and navigate the environments using monocular, stereo-cameras as well as Lidar.

Artificial intelligence: More specifically, deep learning is the workhorse inside the rapidly growing topic of AI. It is opening the door for advanced applications of convolutional neural networks (CNN) to gain a semantic understanding of the scene from 2D images or 3D point clouds. Many deep learning algorithms are nowadays embedded on drones to perform collision avoidance and object tracking (e.g. follow-me options).

The benefits of UAV inspections for civil infrastructure

Significant improvements have been achieved in infrastructure inspection operations thanks to the addition of UAVs with more autonomy over the past two years. Some companies have gone even further by deploying multiple UAVs in order to reduce the time spent on data collection in the field. Generally, inspection using UAVs enables reduction of the inspection time, evaluation of the damage risks in advance, repeatability and measurement reliability, the automatic location of potential damage, and increased situational awareness. The main advantages are as follows:

• Time: Inspection using UAVs can efficiently reduce the traditional inspection time. Additionally, there will be no need to visually inspect damage or potential problems of the investigated structures since there are powerful AI-based detection and identification functions.
• Safety:  To inspect structures in a complex environment, the inspectors often need to access some difficult-to-climb spots. Using UAVs will improve safety and security by helping the inspectors to avoid being exposed to hazardous situations.
• Cost: Detecting and identifying structural problems such as cracks on wind turbine propellers in their early stages will certainly help the operators to avoid the cracks spreading, therefore saving money on larger repairs in the long run. With the use of UAVs, the machine learning techniques will enable automatic indication of such problems on the captured images and in the 3D space, thus reducing the workload and expense of experienced human inspectors.
• Reliability: UAV inspections reduce the factor of human error factor and facilitate inspection accuracy to reduce the structure's downtime. Furthermore, they deliver high-quality and accessible data on the object of interest. With a semi-autonomous flight plan, the UAV can get closer to a critical area on the body of the structure and capture high-resolution images with a guarantee of full-body coverage.
• Repeatability: UAVs make it possible to repeatedly conduct identical inspections in order to monitor the spread of damage over time and prepare a record of the structural health and repairs. Accordingly, comparable inspection data will support decision-making over the lifetime of the structure.
• Adaptation: Recent semi-autonomous UAVs have an important advantage over conventional UAVs in that they can fly smartly in static complex environments using advanced path planning techniques and can adapt to the inspection of different objects.

Applications

According to the American Association of State Highway and Transportation (AASHTO), infrastructure inspections are among the top five UAV applications. Most of the current autonomous UAVs are relying on camera sensors and 4K videos, but some emerging solutions are relying on Lidar too. Various sectors either currently use or intend to use autonomous UAV inspection technology for monitoring, inspection and repairs, including:

• Oil and gas: pipeline leak detection, industrial tanks
• Industry: vessels and aircraft manufacturing
• Energy: wind turbine blades, power transmission lines, topside marine infrastructure
• Construction: inspections of cranes, towers, bridges, roads and railroads.

Some applications of advanced UAVs in civil infrastructure include:

• Wind turbine inspection: The maintenance work on wind turbines, including the required time for repairs, costs a lot each year. Currently, a few companies in the world are using advanced UAV inspections of wind turbine blades to generate a 3D model of the blades and spot defects in real time. The data is shared on cloud-based platforms.
• Aircraft inspection: Throughout aviation history, human errors have caused several aircraft accidents. Therefore, possible errors by technicians and inspectors – such as omitting necessary checks or making unreliable measurements – present a risk. Conventional aircraft inspection is a time-consuming and expensive procedure involving a crew of inspectors, some of whom have to be elevated to access the upper areas of the aircraft. Therefore some companies ship their aircraft to lower-cost regions to reduce the expense of inspections, but the downside of this is that the aircraft remains out of service for a longer time.
• Industrial tank inspection: If tanks are left to age without maintenance, they will end up with leaks which may cause costly repairs or tank collapse. Moreover, a tank leak can cause serious environmental pollution and workplace damage. Accordingly, routine tank inspections are important in identifying and preventing potentially catastrophic tank defects. Rather than conventional inspection procedures, UAVs are currently used by different companies for inspecting cargo and ballast tanks for ships, tanks for oil and gas, water and wastewater treatment plants, power plants, warehouses, etc. (see Figure 3).

• Crane inspection: Crane inspections are critical to worker safety at construction sites, especially in the case of tower cranes which are commonly used when constructing tall buildings. In some countries, frequent inspections of the operating mechanisms, hydraulic systems and hooks and hoist chains are a legal requirement that must be performed periodically (either quarterly or annually). Conventionally, the inspection is visually performed by inspectors, with all controls, mechanisms, operating systems and brakes being checked in accordance with regulations. This conventional inspection is obviously a long manual procedure with several safety issues. A few companies are using UAVs to fly around the crane and perform a full scan within half an hour and the captured image data is then uploaded to the cloud. This ensures that inspectors can accomplish the job with safety, speed, and accuracy.
• Tower inspection: UAV inspection can be applied to different types of towers such as monopole, self-support lattice, guy cable or rooftop towers. In telecommunication, business in the cell tower industry is continuing to grow all over the world, especially with the development of the 5G network technology. However, some unintentional divergences between the designed and constructed tower components may represent a major problem for the development company. Needless to say, accurate information about the tower’s real condition and maintenance status has a range of financial and operational consequences. UAV inspection of towers is useful since it can be applied at any location and deliver results quickly. The UAVs used are at Level 3 of autonomy, meaning that a pilot with limited skills is required on site to execute the survey. The data is then uploaded and processed in the cloud where it can be shared with stakeholders. Deep learning software can be applied for automated analysis to collect information about the type, location and orientation of each tower component, identifying any defects and corrosion for future maintenance.
• Bridge inspection: Conventional bridge inspection is normally performed using a snooper truck (Figure 4), which is a large and expensive truck that carries the inspection crew but poses critical safety risks to their lives. These traditional bridge inspections are costly, and those costs can increase significantly if interruptions occur because of problems with lane closure on the bridge. Currently, UAVs are used by many companies and contractors for bridge inspections because they are up to 75% cheaper to operate, unaffected by traffic levels and more environmentally friendly.

Autonomous UAV inspection challenges

Autonomous inspections are subject to several challenges, including:

• Weather: Difficult weather conditions like high wind speed or rain are still a challenge for UAV inspection missions. Furthermore, image-based inspection missions are difficult in dark places or at nighttime.
• Accessibility: In remote areas, the lack of internet connectivity can limit the use ofservices based on cloud computing or fast streaming of the data. Additionally, the presence of radio interference can create unsafe flight conditions.
• Weight: Building an autonomous UAV requires various onboard hardware components such as the computer motherboard, sensors and GNSS devices. Therefore, the UAV must be designed carefully and with consideration of the payload weight to preserve the battery life and maximize the time of flight.
• Regulations: Although rules are changing all the time, fully autonomous UAVs are still illegal in most countries of the world; a certified pilot must always be present. Furthermore, if a cooperative multi-UAV inspection mission is planned, there should be one pilot for each UAV involved. In 2020 there were some positive changes in drone regulations, and certifications were awarded to several different worldwide drone companies. The certification includes package delivery and more complex operations like extended visual line of sight (EVLOS), and beyond visual line of sight (BVLOS) which will push the development of new solutions and open up new business cases for companies around the world.

Future of autonomous UAV inspection

Various trends are shaping the future of autonomous UAV inspection. One development will be the use of UAV swarms for inspection tasks, in which UAVs will be coordinated dynamically to interact during the flight. Furthermore, we can expect to see an improvement in the performance of UAVs that can navigate autonomously in GNSS-denied environments. This will open new opportunities in the industrial domain. New business opportunities will also come from the improvement of exploration techniques and developments in automated scene understanding, powered by the deep learning techniques applied either onboard or in the cloud. Another potential development will be driven by the use of augmented and virtual reality in autonomous inspections.