Smart Cities Benefit from Infrastructure for Autonomous Vehicles

'GIM International' spoke to Blickfeld, a developer of Lidar sensors and software, to hear about other potentially relevant applications in smart cities, besides for autonomous driving.

Despite the name, a self-driving car cannot actually drive itself. Most manufacturers of autonomous and driver-assisted cars are keen to use Lidar sensors to provide the necessary guidance during the journey, but this market has been slow to evolve so far. GIM International spoke to Blickfeld, a relatively young Lidar manufacturer that is attracting significant global attention for its innovative and high-tech work on new sensors. It appears that there are many more relevant applications on the horizon beyond the automotive segment, such as monitoring the flow of all kinds of ‘traffic’ both indoors and outdoors.

Autonomous vehicles need to be able to perceive their environment reliably within smart infrastructures. The most commonly used sensors for this task are Lidar sensors, which deliver high-resolution and highly accurate 3D measurements of the surroundings under various weather conditions. So far, such sensors have been too large, technologically complex and expensive to be suitable for broad adoption. Now, however, new technologies are entering the market, plus key components such as detectors are becoming smaller, better and cheaper.

Crucial Improvements

“Devices used to cost a five-digit figure just a few years ago, but a fragile sensor costing US$50,000 was too expensive to integrate into autonomous driving or smart city applications. Nowadays, the price range starts at the three digits,” says Florian Petit, chief experience officer at Blickfeld, a fast-growing German-based Lidar software and hardware manufacturer that was founded four years ago. A second crucial development is that the technology no longer requires specialist personnel to operate it. New software platforms have strongly improved the accessibility of Lidar for smart city applications, for example.

Blickfeld has chosen to focus on large market adoption possibilities. That has led not only to more ease of use, but also to an innovative system architecture and the company has already received numerous awards for innovation. Its software enables on-device motion detection through background subtraction; the static background within the point cloud is removed so that the device transmits only motion information. Dynamic objects, such as people and vehicles, are highlighted along with their movements. Blickfeld is the first in the industry to include on-device processing features in its sensors to reduce the amount of data that is transmitted – mostly by more than 95% – while enabling immediate object detection.

A third key improvement concerns the size of the hardware, as Petit explains: “Our sensors stand out because of their extremely small size. That’s a big plus for integration, as the design aspect is often important – especially for passenger cars.” Blickfeld’s mid-range sensor measures just 5cm³. It can be incorporated into side mirrors, lights and pillars to achieve a 360° panoramic view. It has a field of view of up to 107° and a range of up to 150 metres. Meanwhile, the long-range sensor – designed for use at the front and rear of a vehicle – detects objects at distances of up to 200 metres.

Safer Testing in Real Life

But the widespread adoption of autonomous vehicles depends on more than just technology alone. For example, the European Commission has recently forbidden the activation of autonomous driving assistance (lane keeping, parking, etc.) in new cars in the EU because it is still considered too dangerous. Petit is convinced that autonomous or semi-autonomous driving will become mainstream eventually, but that it will happen gradually rather than overnight: “Autonomous guided vehicles for large warehouses or ports are less complicated. Equipped with sensors, the systems can avoid collisions and transport goods from A to B by automatically determining their position. Likewise, autonomous tractors or other self-driving agricultural equipment can assist farmers in automated precision farming. We even work together with Elbflorace team from TU Dresden that builds racing cars to participate in Formula Student competitions. This year, their car will compete in both disciplines: both with a driver and autonomously. But autonomous cars on public roads are a different matter. There are many issues that still need to be addressed, especially in terms of safety and regulation, but also related to sensor fusion, connectivity and data size. That’s why projects like IN2Lab are so important.”

IN2Lab is a three-year project currently running in Ingolstadt, close to the German city of Munich. It is being led by Ingolstadt University of Technology with the aim of developing a safety system for testing automated driving functions in real-life environments. “Only real-life scenarios can cover the outlying cases, i.e. unknown and spontaneous situations,” states Petit. The IN2Lab research partners have integrated additional safety features into the road infrastructure – including roundabouts, crossroads, bus stops, bicycle lanes, traffic lights and parking spaces – in one part of the town. This enables the entire road traffic environment, including the manoeuvres of autonomous vehicles, to be monitored externally using sensors, 5G communication modules and artificial intelligence tools. Blickfeld is responsible for the environment perception sensors integrated into the infrastructure. The sensors collect detailed real-time information, and the detection software analyses the speed and direction of movement of each road user. The Blickfeld sensors are based on MEMS technology, featuring a particularly large aperture, wide-beam deflection angles and high optical power.

Smart Cities

In Petit’s experience, local authorities are keen to use smart city applications to optimize vehicle flows in their cities. Equipping the roadside infrastructure with Lidar sensors will speed up their adoption for traffic monitoring. The information can be used for near-real-time traffic flow control by adjusting traffic lights or road signs taking the movement of all road users into account. Historic analysis can be input for urban planning purposes.

Various municipalities are also preparing parking-space detection and guidance to save drivers the hassle of searching for somewhere to park. Mounted on lamp posts or traffic lights, the sensors monitor parking spaces and the integrated geometric analysis software determines whether each parking space is occupied, as well as its size. This application is aimed at reducing unnecessary driving and therefore pollution in urban areas, although some pessimists fear that making life easier for drivers in cities will actually stimulate traffic growth.

Meanwhile, Blickfeld is developing an algorithm library that includes a variety of pre-processing features in order to enable the on-device data analysis being implemented in many other use cases, such as security. For example, in the case of an intrusion, a sensor first measures the precise dimensions of the detected object (even in low-visibility conditions); the alarm is only triggered if the software identifies a potential (human) trespasser. The public and private sector alike are becoming more interested in better perimeter security – resulting in fewer false alarms and absolute privacy protection – without the need for more human resources or external hardware.

People Tracking

One market segment in particular is growing rapidly, comments Petit: “In recent years, we have identified very strong demand for people tracking, and not only during the current pandemic. There are numerous situations in which institutions or organizations need to know how many people are present in a certain area and how they are moving around. Lidar sensors are ideally suited for this; with a large field of view, they usually cover bigger areas than cameras and they collect data independent of lighting conditions. We currently have a project with Frankfurt Airport, the fourth-largest airport in Europe, to track and analyse passenger flows. The sensors help to predict the passenger volume in the security check area so the authorities can take appropriate measures in advance.”

It is sometimes necessary to know how many people are on the premises and how the flow is moving in other places like museums, event venues, stadiums, pedestrian zones or on railway platforms, and new, smart Lidar applications are emerging for such uses too. Needless to say, what works for flows of people also works for objects, according to Petit. A Lidar device can immediately evaluate 3D-sensed data to provide information on occupancy and fill levels, so volume measurement and level sensing can enable companies to keep track of their inventory and the available space in warehouses, trucks or containers.

About Blickfeld

Before founding Blickfeld in 2017, Dr Florian Petit worked on robotic control as part of his scientific work at the Technical University of Munich, Stanford University, the German Aerospace Center and ETH Zürich. He obtained his PhD in the field of human-machine collaboration. Today, Petit is Blickfeld’s chief experience officer.

Blickfeld is based in Munich, Germany, and currently employs just over 100 people. It has received ten innovation awards. Blickfeld introduced industry-first functions for on-device pre-processing of Lidar data: a system-on-chip computing chip set enables the execution of algorithms on the sensors themselves. Blickfeld is expanding its algorithm library with a variety of pre-processing features to be used in applications for smart cities, security, agriculture and industry. www.blickfeld.com