Smart Factories Need Space and Time Anchors

Spatiotemporal referencing can be used to enhance contextualisation of Industrial Internet of Things (IIoT) data and information flows within ‘smart’ factories, according to Studio iSPACE in Austria. Its five-step approach models an adaptive workflow to organise large-scale 3D data of a production facility. The goal is to support the extended personnel-machine interaction for indoor positioning, production asset monitoring and location-based management.

In the event of problems with a production line, every minute counts for the factory’s managers. Service staff must be able to respond as efficiently as possible. All necessary information should be directly available and location-based. Studio iSPACE, part of Research Studios Austria Forschungsgesellschaft mbH, recently transformed its experiences with spatial data infrastructures (SDIs) into an approach called OLS3D (Organising Large-Scale 3D Data). It tackles the use of building information modelling (BIM) for production environments and smart factories in the context of Industry 4.0. With the help of tablets or smart glasses and a central software system in the background, the knowledge is brought exactly on time to where it’s needed – to the service technician on site. Studies of these tasks for production environments are new; as far as iSPACE knows, the prototype it developed is still unique. Some parts overlap with facility management, where the integration of CAD, BIM and GIS systems is an important topic. A major issue proved to be the lack of indoor inventory definitions in geoinformation models. In factories, the indoor inventory is the dynamic layer where tasks are performed, data is created and which has a high optimisation potential.

Cyber-physical production system

Industry 4.0 relates to facilities and production environments where humans and machines can directly interact to solve problems together. This is also known as the Industrial Internet of Things (IIoT). Industry 4.0 is aimed at achieving full integration of processes, machines and products to optimise availability, costs and resource consumption in smart factories. Objects only can become smart if they can be linked with data and other digital resources. “By using space and time as anchor points, digital resources such as documents for maintenance as well as real-time sensor data for automation and facilitation of work processes can be linked to the real world. A spatiotemporal ‘anchor’ acts as a uniquely identifying referencing element,” is how Laura Knoth, researcher at Studio iSPACE, outlines the heart of the approach she developed together with her colleagues Bernhard Vockner and Manfred Mittlboeck. She gives an example: “In a cyber-physical production system, when a machine needs to be serviced, the authorised worker gets a signal on his mobile terminal about its priority or planning, a report on the machine’s status, where it is, and all related manuals.”

An Enterprise SDI should contain geodata on the outdoor and indoor building/facility infrastructures. A production environment has specific elements that need to be modelled and integrated to obtain the big picture of all processes. These elements include the machines and corresponding sensors as well as associated data in the shape of digital resources such as maintenance reports. This is mostly data in three dimensions – not only because buildings consist of several floors, but also because machines with complex structures and overlapping parts can only be meaningfully displayed using 3D. The time component is addressed as being able to look at the past with versioning and archiving of data, as well as being able to use this data to ‘look into the future’ through extrapolations and prognoses. The linking of data follows the Linked Open Data approach. In terms of sensor data, the information can be linked directly; other less structured data is harder to integrate. This is especially the case for maintenance documents, such as documentation prepared by a maintenance employee who worked on the machine earlier. To link such data, machine names can be used as attributes in the spatial data model and PDFs can also be used for linking documents to objects.

Five steps

The OLS3D approach consists of five steps. Firstly, a requirements analysis has to be performed to identify the structure of the organisation and its needs. The second step is to ensure that the quality of the data stays well organised. Modelling of the business process should take into account the whole process from data acquisition to use and updating, resulting in workflow models that automatically push changes – in infrastructure, inventory, all moving assets – through the entire pipeline. “In order to leverage the information in an interoperable manner we need mapping schemes between BIM, CAD and GIS. Per specific production chain there are also probably additional semantically harmonised ‘dictionaries’ for assets and machinery needed,” states Ms Knoth, speaking from experience. For an industrial facility building model, the source data models are mostly from BIM and CAD systems. When additional infrastructure information is needed, scanning in 3D is the solution – which becomes cheaper and easier with photo capture techniques. The target models are from the GIS world, since the end use takes place in GI systems. Open Geospatial Consortium’s CityGML/IndoorGML model and the INSPIRE Building model are the main standards that need to be considered. In CityGML, each object has at least three attributes. For ‘Building furniture’, the list contains 19 possible class attribute values. An additional code list contains descriptions and numbers for the ‘function’ and the ‘usage’. While CityGML is an ubiquitous standard, issues have been raised about the inconsistency of the levels of detail (LoDs) as well as the size of the files. The INSPIRE Building model (coming from the INSPIRE Directive: Infrastructure for Spatial Information in the European Union) also has a data specification on buildings. It includes a mapping scheme to exchange data with CityGML. While CityGML provides the possibility of adding movable furniture elements, such elements do not exist in the INSPIRE Building model.

The existing models will probably need to be adapted to fit the use case; that is the third step in the Austrian approach. It might include the definition of a CityGML ADE or a new intermediate model as transitioning step for the harmonisation tasks. Afterwards (step four), the transition process can be automated by using a spatial ETL tool such as Safe FME or OpenSource HALE. This opens up the possibility to integrate quality assurance mechanisms during processing.

Pilot

The implementation of a prototype of the new framework is the last phase in the Organising Large Scale 3D Data approach. To guarantee smooth development and the inclusion of all important aspects, it is necessary to consider not only the technological viewpoint, but also the organisational one. Referring to successes in the field of SDIs, Laura Knoth says: “The main differences between a 3D model and a 3D SDI are made by ‘organisation’ and ‘people’. Every stakeholder must want to share in all phases.”

Studio iSPACE implemented a prototype of a web-based 3D building representation that includes data streams of real-time indoor positioning in alignment with the presented standards. It has been implemented with the Esri JavaScript API 4.0. This representation can be highly customised in order to provide the best possible view depending on the user’s context: not distracting, no information overload, depending on the role, etc.

The building model was acquired using CAD plans that were transformed with Safe FME Desktop into a geodatabase, which was then extracted and uploaded as an Esri web service. The indoor positions were generated by a smartphone app developed by iSPACE. Knoth explains: “In order to be independent of specific companies for indoor mapping, we propose VGI as a way to capture building information: geographic data provided voluntarily by people working in a certain building. With tools from OpenStreetMap, for instance, the amateurs can model what they need in a simple manner which makes it possible for them to contribute their expert knowledge.”

The goal of all this is to support human-machine interaction in smart production environments for indoor positioning, production asset monitoring, orientation, location-based management, operational safety as well as situational perception of personnel and machines. According to the users’ positions, contextual information is provided on either tablets or, if both hands need to remain free, smart glasses. When an authorised maintenance employee approaches a machine, the employee’s smart device displays a website showing information and the status of the machine as well as links to maintenance documents. This approach can also be used for remote maintenance; a worker on site can be supported by a second employee at a different location based on live information. Additionally, using virtual data (‘digital twin’), maintenance personnel are able to get an overview of a machine and its environment before they even enter the location. Any photos, videos, text or voice memos created are provided as support for similar cases in the future. Needless to say, the big picture provided by the map and the current status of the production line supports management decision-making. Managers obtain location-based status information about the current maintenance visits and are supported in their planning and inspection tasks, e.g. monitoring, diagnostics and prognostics.

About iSPACE GI Research

Research Studios Austria Forschungsgesellschaft mbH manages the transfer of innovation and knowledge between the market and the universities of Vienna, Linz, Salzburg and Innsbruck. It currently has six research studios with different specialisations in IT-related research. Studio iSPACE is dedicated to geoinformatics and GIScience. The prototype in this article has been developed within the context of ‘Assist 4.0’, a research project that leading Austrian industrial, commercial and research companies have been working on since 2014 (budget of €3 million, with €1.8 million donated by the Austrian Research Promotion Agency). Production staff are supported during troubleshooting, servicing and remote maintenance with context-sensitive information. The industry dissertation project OLS3D became a spin-off concerning the use of building information models for production environments. Testing of Assist 4.0 has been done, for example, in the semiconductor production clean room at Infineon Technologies Austria and in the automotive industry. The application is controlled multimodally in line with the individual requirements of each case through touch, voice commands and, in the future, gestures.