Crowdsourcing the Reconstruction of Lost Heritage

Cultural heritage stands as an important testament to how we perceive the past. Unfortunately its preservation cannot always be assured, due to destruction by human or natural means. In February 2015, a video was released which showed the destruction of artefacts in the Cultural Museum of Mosul, Iraq. A volunteer initiative (initially called ‘Project Mosul’) was launched about a week later with the aim of using crowdsourced images and computer vision to create 3D representations of the destroyed artefacts. Project Mosul was initiated by two PhD students from the University of Murcia in Spain and the University of Stuttgart in Germany, and soon expanded to other geographical regions. The project was therefore renamed ‘Rekrei’, which is Esperanto for ‘recreate’. Read on to learn how Project Rekrei uses crowdsourcing and photogrammetry to recreate the past.

(By Matthew Vincent, PhD candidate at the University of Murcia, Spain, and Chance Coughenor, PhD candidate in the Institute for Photogrammetry at the University of Stuttgart, Germany)

An early version of the concept – the reconstruction of lost heritage based on existing photographs – was applied to the case of the Bamiyan Buddhas in Afghanistan by Armin Gruen, Fabio Remondino and Li Zhang, after they were destroyed in 2001. Thanks to the subsequent advances in computer vision, web development and digital cameras, Rekrei was able to successfully reconstruct some of the Cultural Museum of Mosul’s objects using the initial images provided by Suzanne E. Bott, Diane Siebrandt and Col. Mary Prophit. The further potential of this was quickly recognised as news emerged that the nearby sites of Hatra, Nimrud and Nineveh had suffered similar fates.

Expansion

As word about the project began to circulate, and thanks to the initial partnership with Sketchfab, more institutions, companies and volunteers began to join and contribute to the project’s development, including by donating images. Individuals who did not have images or experience with photogrammetric software offered their help by scouring the internet for architectural drawings, archaeological reports and other related documentation, or assisted with masking the objects in the uploaded images to improve the final step of 3D photogrammetric reconstruction. Since the challenge of cultural heritage preservation is not isolated to Northern Iraq nor caused by human destruction alone, the project’s volunteers began to spread their efforts to cases in Syria, Egypt, Yemen and, following the massive earthquake in April 2015, also Nepal.

Rekrei Online Platform

Rekrei’s work starts online. The online platform provides the primary area for organising and managing the digital images that are used for photogrammetric reconstructions. Online, there are three primary tasks that volunteers participate in. First, users identify locations around the globe where heritage has been lost, due to either natural causes or human intervention. These locations serve as the major divisions in organising the work. The second task is uploading images of these locations, which anyone can do as long as they have created an account on the website. The third and final task is organising the images of each location into relevant groups for reconstruction. Images can belong to more than one group, as an image may contain one or more parts that should be included for photogrammetric processing. Likewise, some monuments are too large for processing in photogrammetric software or are simply not fully covered in the digital imagery. Breaking these monuments down into smaller groups allows for parts to be reconstructed and eventually merged together if more imagery becomes available at a later date. In some cases, these monuments may be reconstructed manually by digital artists who base their work on the photographs. Since the above reconstructions are not valid for scientific analysis but rather serve primarily as visual representations of the lost heritage, any process for recovering lost heritage is considered to be valid.

Extending Functionality

As the platform continues to grow, new features will be incorporated to facilitate greater coverage and track the process by which models are created through the platform. One example is the integration of Flickr through its geolocation API. As each location on the Rekrei platform has a latitude and longitude, users will be presented with a section of the page that displays all the available images with non-restrictive Flickr licences that have been assigned to the same location. Users can adjust the search radius using a slider, extending it up to a kilometre away from the centre of the location that they are browsing. This allows for many more photos to be included by simply presenting the user with other possibilities on other platforms. Future development will attempt to do the same with similar online image repositories.

Tracking Contributions

It is important to track how models are produced using the Rekrei platform to ensure proper attribution. If a user wants to upload a photo but the image requires attribution, it is necessary that the metadata of that 3D model contains a complete list of the photographic sources that were used to produce that particular model. Therefore, further development is on implementing a metadata system on the platform that tracks users’ work, who uploaded which image, which licence that image has, who organised it into which group, and finally who did the processing to generate a 3D model. This workflow makes is possible to track how each volunteer contributes to the recovery of lost heritage via the Rekrei platform.

Achieving Some Goals, Setting Others

In November 2015, the initial dream of the project’s founders was realised during a short exhibition in Amsterdam, The Netherlands, thanks to The Economist magazine’s Media Lab. A virtual recreation of the museum was modelled and the 3D artefacts were ‘virtually’ returned to the museum, along with references to the volunteers who recreated them. The models were enriched further with an audio track featuring interviews from cultural-heritage specialists. In addition to providing a virtual reality (VR) experience to the public, three 3D-printed artefacts were also on display, providing a dramatic representation of preserving the memory of lost cultural heritage. In May 2016 the virtual museum was released to the public for Google Cardboard (iOS and Android) and YouTube 360, and a Samsung Gear VR version is coming soon. At the time of writing, RecoVR: Mosul had received almost 100,000 views.

Even with this milestone in place, Rekrei continues to look towards the future. It will continue to participate in retelling the story of lost heritage, such as partnering with the Dutch National Museum of Antiquities and its upcoming international exhibition on the Assyrian Empire’s city of Nineveh. Rekrei will also be participating in the Web Summit in Lisbon, Portugal, in November 2016. At the same time, the initiators will continue to push the platform’s tools forward, progressing with ambitious goals such as automated cloud processing of the photogrammetric sets and an overhaul of the user interface to streamline the process and make it easier to recover the world’s collective, lost heritage.

Further Reading 

Introducing RecoVR Mosul, The Economist’s first VR experience (20 May 2016)

Crowd-sourcing the 3D digital reconstructions of lost cultural heritage. Vincent, Matthew L., Mariano Flores Gutierrez, Chance Coughenour, Victor Manuel, Lopez-Menchero Bendicho, Fabio Remondino, and Dieter Fritsch. In 2015 Digital Heritage, vol. 1, pp. 171-172. IEEE, 2015.

Project Mosul: A Digital Future for Lost Heritage. Current World Archaeology (Edition 72) UK: Current Publishing

Authors

Matthew Vincent is a PhD candidate at the University of Murcia, Spain, where he is researching digital applications and new methodologies for archaeology. He has worked in Jordan for the last 12 years, where he has implemented new digital techniques in field work, co-directs the Balu’a Regional Archaeology Project, and generally looks for ways in which new technology can be of service to cultural heritage. (matthew@rekrei.org)

Chance Coughenour is a PhD candidate at the Institute for Photogrammetry at the University of Stuttgart, Germany. His main research interests are reality-based modelling techniques and virtual-reality dissemination strategies for archaeological investigation and cultural heritage preservation. (chance@rekrei.org)

Figure captions

Figure 1, Relief from Hatra, Iraq. (Pedro Máximo, rekrei.org)

Figure 2, Entrance of Nimrud, Iraq. (Pedro Máximo, rekrei.org)

Figure 3, Durbar Square, Kathmandu.(Konrad Wenzel, nframes.com)

Figure 4, RecoVR Mosul Exhibition, DocLab at IDFA Amsterdam. (Nichon Glerum)

Figure 5, 3D prints from crowdsourced data, RecoVR Mosul Exhibition. (Nichon Glerum)