Mapping Indoor Spaces with an Advanced Trolley

Few museums, shopping centres, airports and other indoor spaces have been mapped, although the demand for detailed 3D models of such spaces is accelerating rapidly. Here, the authors present a trolley-based system equipped with laser scanners, cameras and advanced software aimed at creating, visualising, navigating through and exploring detailed and accurate 3D models of indoor spaces. The system can capture up to 50,000m² daily. To date, ten mapping trolleys are operational; two are operated by NavVis and eight by other companies. The potential is huge: according to estimates, at least 50 billion square metres of interior spaces will be mapped within the next five years.

3D Mapping Trolley

The heart of the system is the proprietary, patent-pending 3D Mapping Trolley M3 equipped with three laser scanners, six 16-megapixel cameras and an inertial measurement unit (IMU) (Figure 1). Additional sensors record Wi-Fi, Bluetooth, and magnetic fields. The cameras capture high-quality images even under bad light conditions. The maximum range of the laser scanners is 30m and their accuracy is 2cm. The cameras and scanners capture space horizontally and vertically in 360 degrees, i.e. all the way round. The lightweight trolley can go up and down easily, its operation does not require specific knowledge or skills, and it takes just a few minutes to assemble.

Trolley Software

Combining laser data, IMU data and images through a 2.5D simultaneous localisation and mapping (SLAM) algorithm, the trolley constantly orients itself in the space (see [1] for a brief explanation of SLAM). Software stitches the images together into panoramic views. The trolley software post-processes the raw data fully automatically to create products suitable for publication on the web. The results are available within hours instead of the weeks or months required in the case of conventional mapping systems, thus enabling new applications – such as mapping a trade show the night before its opening so that visitors can explore the 3D model the very next day. The viewer is built on HTML5 and WebGL and does not require any additional software or plug-ins. It can therefore be easily integrated into a website and allows the virtual models to be displayed on any device – computer, tablet or smartphone – at any time. Text, audios, videos, live chats or other content can be added to the 3D model as points of interests (POIs). Clicking on a POI causes its content to pop up. A search bar helps users to find specific POIs quickly. For monitoring purposes, heat maps can be created which indicate how long and how often users look at objects or POIs. Detailed photorealistic 3D models are created by combining panoramas and laser point clouds. The user can look around, navigate to desired locations, interact with POIs and, by clicking on points in the panoramas, conduct accurate linear measurements using the underlying point cloud. An open API enables partners to integrate or link the IndoorViewer, which is part of the system software, to their e-commerce or ERP systems. Typical applications in facility and building management are documentation, inventory, construction progress monitoring, factory planning, path finding and task management.

Munich Airport

The indoor space of Munich Airport, Germany’s second-largest airport, covers over 20ha of public and non-public areas. To serve the airport’s customers, passengers and facility managers, the authors created photorealistic 3D models of the huge indoor space based on recordings performed by a team of airport employees using one of the authors’ trolleys. The 3D models enable travellers to explore the airport ‘virtually’ and to plan the route to their gate from the comfort of their own home, for example. Facility managers and external maintenance personnel who are unfamiliar with the layout of the airport are able to find their location quickly and can even prepare for repair work in advance, saving time and money on job-site inspections. To keep the 3D model updated, the airport bought its own Mapping Trolley for operation by a team that recaptures areas whenever changes occur.

Deutsches Museum

The Deutsches Museum in Munich is the world’s largest museum of science and technology. As a first step towards creating 3D models of the interior, the ‘Shipping’ section was recorded (Figure 2). Visitors can now take a visual tour on the museum’s website [1], and the many POIs give access to such a wealth of information that users almost feel as if they are actually walking through the shipping exhibition itself. For example, there are audio files about the Santa Maria, the small ship that Christopher Columbus brought to America in 1492, and about the luxurious amenities in the express steamer owned by Kaiser Wilhelm II in 1903. On site, visitors can use their own smartphones or tablets to access an audio guide. Data ownership remains with the Deutsches Museum, as stipulated in the contract.

Construction Sites

Monitoring the construction progress of factories, office buildings and other objects poses several challenges. For example, the contractor’s office may be located a considerable distance away from the site. To save time and money, photos of the site – sometimes taken using smartphones – are regularly sent to the experts in the office. However, it is more efficient to create 3D models of the site at various stages of construction showing where pipes are situated, which materials are being used or how the wiring has been done. Then, if an element breaks or if a warranty claim arises after completion, it is possible to retrace what has been done and how.

Smartphone App

The next generation of the system will enable navigation through indoor spaces via a smartphone app. Unlike most indoor navigation systems, and similar to human orientation, this visual positioning technology does not require any additional infrastructure such as Wi-Fi hotspots, RFID or Bluetooth to locate the user. Instead, the user is located autonomously with an accuracy of one metre by comparing the ‘visual fingerprint’ of their smartphone camera with the data captured earlier. The user’s viewing direction is also determined. If available, the app will also register Bluetooth and Wi-Fi signals and use further smartphone sensors such magnetometers to improve accuracy.

Partnering

Solutions tailored for specific user cases are currently being developed together with partners. For a seamless integration of indoor and outdoor mapping and navigation, the authors teamed up with Esri. The first result is Campus Navigator, a browser-based app which allows visitors to navigate to rooms and find people on a campus. Cooperation with Intershop will result in a solution which enables users to navigate through a physical shop and select products by clicking on POIs of products on the shelves: an easy-to-use solution for retailers that want to combine the advantages of online and offline shopping. The authors are also looking for partners active in computer-aided facility management (CAFM) systems, since the integration of the IndoorViewer into a CAFM system will make it much easier, quicker and more fault-resistant to locate a defect component.

Figure 1, 3D Mapping Trolley.

Figure 2, Point cloud overlaid with images of the shipping exhibition at the Deutsches Museum.

Biographies of the Authors

Felix Reinshagen, CEO, holds an MSc degree in computer science and a PhD in economics. After six years at McKinsey, an award-winning project resulted in him co-founding NavVis, a spin-off of the Technical University of Munich (TUM), Germany, which focuses on mapping, visualisation and navigation of indoor spaces. Since being founded in 2013, the company has grown into an international team comprising 45 highly qualified scientists, engineers and business builders. felix.reinshagen@navvis.com

Georg Schroth, co-founder and managing director, holds a PhD in electrical engineering from TUM and graduated with an honours degree from the master-level graduate programme of the CDTM Munich. He joined the GPS Lab in 2007 and the Information Systems Laboratory (ISL) at Stanford University, USA, in 2010. georg.schroth@navvis.com

Lars Schmitz is employed by Esri, where he previously worked as a project and product manager. He is now leading Esri’s initiative to collaborate with start-ups to unlock innovation potential. l.schmitz@esri.de