Streaming Massive City Datasets to Mobile AR and VR

Umbra, a company specialised in creating 3D content in real time, has partnered with the City of Helsinki, Finland, to bring a massive, real-time streaming 3D model of the entire city online and make it viewable on mobile devices and web browsers. The reality mesh model of Helsinki is based on aerial photographs of the city taken in the summer of 2015. Helsinki generated an enormous point cloud, representing a 50-square-kilometre area of the city, then processed that data into a 700GB texture-mapped 3D mesh.

The city of Helsinki’s goal was to make this open dataset available for anyone to use, but the sheer size and complexity of the 3D data presented an insurmountable challenge. At that point the city spoke with Umbra about using its automated optimisation on the dataset to make it easily deliverable. Umbra optimised the mesh using its fully-automated cloud platform that can now stream the entire dataset to AR or VR-capable mobile platforms such as phones, tablets, headsets and even web browsers.

Combining terrain model and surface model

The Helsinki city model’s buildings were placed at the right elevation by transferring the buildings’ footprints from the city plan’s base map to the digital terrain model. The buildings were then converted into 3D models by combining the terrain model with a surface model that included the shapes of the buildings’ walls and roofs making a digital twin of the city.

This was really a new and unique project collaboration between Umbra and the City of Helsinki – combining game engines and the city’s own technology into the next-generation of city modelling, said Jarmo Suomisto, project manager at the City of Helsinki – 3D City Information Model. This collaboration brings together so many stakeholders and technical standards, such as the open data policy and manufacturing technology, as well as the public and private sector. Through this cooperation, this whole project has been made possible and Helsinki appreciates Umbra's willingness to develop new technologies that truly push the envelope.

3D streaming

Based on a novel and patented approach to 3D streaming, Umbra’s new solution enables photo-realistic, multi-billion polygon models to be streamed and manipulated in real-time on mobile, untethered devices including phones, tablets, and VR and AR headsets. Umbra also makes it possible to integrate these highly-optimised models directly into commonly used 3D development environments like Unity or Unreal.

Umbra’s new cloud-based optimization and delivery solution operates similar to Google Earth, but has the potential to render and stream significantly higher resolution 3D content, even down to sub-millimetre accuracy depending on the source scan, said Shawn Adamek, chief strategy officer at Umbra. In addition, Umbra is democratising city-scale 3D scanning, giving cities and developers the opportunity to take ownership of their own datasets, rather than using and licensing existing mapping data.

To access the Helsinki 3D dataset, see here.