How mapping experts built the biggest 3D dataset of Alcatraz Island

The world’s most complete digital twin of Alcatraz

The rugged 22.5-acre site of the notorious US island prison Alcatraz has been digitally captured in a world-leading project involving multiple mapping technologies. With the team camped in the jail’s infamous cells for three weeks during the cold San Francisco winter, this historically, biologically and architecturally important location was recorded in unprecedented 3D detail. We discussed the project with project head Pete Kelsey and team member Dr Jeremy Sofonia.

What was the Alcatraz project and who was it for?

Pete Kelsey: The Alcatraz project was conducted for the United States National Park Service, to obtain a baseline survey that can be used for change detection analysis. By mapping all the structures inside and out, a baseline could be established that future surveys can be measured against. Alcatraz – it’s a lot of things – but what most people don’t realize is that it’s a very dynamic environment. The Park Service was keen to understand the effects of climate change, sea-level rise and over one million visitors a year on the historic site. Alcatraz is located in San Francisco right on the San Andreas Fault, so it’s also seismically active. There are a lot of things going on that present challenges for what is arguably the most notorious prison on earth. As a result, the Park Service was very keen to get that baseline survey done. From there, the conversation changed to the creative ways in which this kind of reality capture digital twin type data could be applied.

The team was granted access to almost all areas of the facility, including restricted spaces such as the cells made famous in the 1962 escape

What pre-planning was involved?

PK: The planning took nine months. The first thing that comes to mind was the permit process as two permits were required: a scientific permit to do the work and, as no drones are allowed in US national parks, a permit to fly a drone. The plans included an airborne and a terrestrial component, and an interior and an exterior component. On the ground, we had Emesent Hovermap and terrestrial laser scanning. We also had
photogrammetry and we flew multispectral because some of the biologists were interested in the plant life on the island as well. Once those two permits were issued, things happened quickly. 
Jeremy Sofonia: I don’t think anybody truly anticipated what it was going to be like on Alcatraz. A typical day entailed waking up at 5 a.m. and leaving the cells we were sleeping in straight away, as we only had a couple of hours before the tourists and staff turned up. It was important that we captured the data before the crowds started coming in. We can capture data with people, but it can add time. to our post-processing to clean them out. Another reason was that the Park Service, and therefore we too, were very keen not to impact the tourist experience.

The team also had access to areas off limits to the public, including the famous lighthouse.

How did you organize the large volumes of captured data?

PK: I put together an amazing team who were well versed in the IT back office file management component of organizing data. Without a scan plan, a naming convention and some form of standardization, this wouldn’t have worked as it would have been chaos. We had a single airborne GNSS Lidar flight, a single multispectral flight and a single photogrammetric flight. Where it got challenging was with the volume of Emesent Hovermap scans to capture all the interiors and those outside not seen from an aerial perspective. Ultimately, we completed more than 100 scans, which I’ll bet is a first in this type of space. I knew that having Dr Jeremy Sofonia from Emesent onboard was going to be key, with the Hovermap work falling to him. He had to create a system that would make sense to all of us, but most importantly work for him.
And he did it, we’ve had no issues and it’s been fantastic.

Did you do any data processing on site?

JS: Absolutely…the sheer volume of data means it’s great to keep up with it so that you don’t have a massive backlog, but it’s also about quality assurance, quality control. We wanted to make sure that we captured what we came for as we collected it because of the relatively limited time we had on site for such a unique opportunity. It’s prudent to check that what you’ve got is true and correct as you go. All of the Hovermap data was processed in Emesent’s Aura software. With that initial processing, you can use the visualization tool to load the point cloud and can check for any issues. This allowed us to validate a scan and move forward in real time.

Cell block – Peter Kelsey (left), Jeremy Sofonia and other team members slept in the cell block for three weeks.

What were the project’s biggest challenges?

PK: Access was absolutely number one. Every room had a lock. Have we got the key, or who does? Do we know where they are on the island so that we can find them, borrow the key and then return it? We knew right away that this was going to be a problem, but the Park Service were great. I explained the problem to the facilities manager on Alcatraz and she didn’t blink, she just got up, went into a safe and handed me the keys to Alcatraz.
JS: My primary objective was to conduct handheld mapping of the interiors, which was Emesent Hovermap’s role in the project. Handheld mapping was used most of the time, but because many of the spaces were not well lit – or not lit at all – we had a special lighting rig that snapped onto my Hovermap unit. This illuminated the field of view of the GoPro that was there to obtain RGB true colour data. We also utilized the Hovermap’s versatility for those times when handheld wasn’t the best option. This included our polemount and even Boston Dynamics Spot robot. So, not only were we looking to capture Lidar data, but also in true colour so that people could have a more realistic experience when looking at the data. Adding that element definitely increased the challenge in many dark spaces; however, the lighting rig worked great and I think the results will speak for themselves. People don’t realize this, but rather than the prison guards living in San Francisco and commuting back and forth, their entire families lived on the island. Today, those structures are still there, some in ruin, others needing restoration. One of these, large, multi-story and complex, is closed to the public with signs on the door reading: Danger Lead, Danger Asbestos. This is, for example, where we used Hovermap on Spot. It would have taken us far longer to put the PPE on and take it off again than it did for us to do the whole job with Spot, let alone the health and safety approval process. This is a great example of how we can use robotic systems not just to keep ourselves safe, but also to minimize the red tape and time involved, to get the job done. 

The full version of this story is available at