The Revolution of Drone-carried Sensors

The world of professional unmanned aerial vehicles (UAVs or ‘drones’) is changing. It is a matter of ‘survival of the fittest’ in terms of the hardware aspect: UAV manufacturers. The interest is now visibly shifting towards payloads and software to capture, visualise and process the data; UAVs are the apparatus, but it’s the sensors and software that really count. GIM International decided to touch base with a company that serves as a good example of the current transformation, and spoke to Chris Anderson, CEO, and Daniel McKinnon, vice president, from 3D Robotics. Read the interview here.

(By Wim van Wegen, content manager, GIM International)

3D Robotics was founded back in 2009. What made you decide to enter the UAV business?

Chris: It was back in 2007 that I built my first drone at home with my kids. Not much later I created an online community, DIY Drones, where I met Jordi Muñoz, a 19-year-old college dropout and electronics wunderkind from Mexico, who demonstrated his talents by creating a fully functioning autopilot using circuitry lifted from a Nintendo Wii remote. In 2009 we set up our own company, 3D Robotics. We chose this name as it alludes to the third axis – up – where consumer robots hadn’t yet been. In late 2012 I quit my job as editor-in-chief of Wired to focus full-time on the drone business.

At that time, did you have any idea of the endless possibilities for UAVs, such as in surveying and mapping?

Chris: Yes and no. I certainly knew that there would be many applications for UAVs; as a matter of fact, I was more interested in the applications than I was in the drones themselves. When I first made drones in 2007, they were called ‘geocrawlers’ – like a web crawler but for the Earth, to measure the world. My vision has always been that there is an opportunity below the satellites, below manned aircraft, to do with the physical world what Google did for the web. So I was interested in putting sensors in the sky, and drones were just the best way to do it. I didn’t actually know what the specific applications would be, but I guessed they would probably be linked with agriculture, wildlife conservation, the environment, etc. I didn’t think very hard about it; I just assumed that experts in those domains would show me how to use the sensors in the sky and that I would learn from them – and that in fact turned out to be true. 

Chris Anderson, co-founder and CEO, 3D Robotics.

You’re moving away from manufacturing consumer drones. Why did you decide a new strategy was necessary?

Chris: That requires some explanation. The industry is actually going through three phases and they were predictable right from the very beginning, in the late 2000s. We knew there would first be the technology phase: the software, the autopilots, the sensors – the core ‘brains’ of a drone. We did very well in that phase. We knew the next phase would be the consumer phase, a ‘toy’ for want of a better word, thanks to the exemptions making recreational drone use legal – unlike commercial use. So the only real path to market would be via the consumer. We also knew that the endpoint in terms of our interest was the commercial use, i.e. less about the drone, more about the data. So this progression from technology to toy to tool was predictable from the very start. Unfortunately there was no way to just hop from technology to tool; we had to go through the toy phase, because that’s where the ease of use, the design and the overall package benefited from the demands of consumer acceptance. This phenomenon, the ‘consumerisation of the enterprise’, has held true ever since the iPhone. Rather than making complicated tools for professionals and easy tools for consumers, everybody wants easy tools – an iPhone equivalent. Whether a consumer is using a drone to film video or a construction worker is using the drone to scan, neither of them care about the technology or want to be trained as pilots, they just want a result.

However, the consumer phase is extremely expensive. You have to deal with retail, distribution, volume, customer support and so on. The consumer electronics market is tough at the best of times, and we happened to be competing with one of the best companies in the world: DJI. DJI represented a new era of Chinese companies that moved really fast, had extraordinary scale, great software, hardware, distribution, marketing…they really did everything right. It became clear very quickly that they could operate at a price point that we could not. We were fortunate to see the writing on the wall relatively early, so we actually decided to move out of the consumer hardware back in 2015 to focus entirely on the commercial applications. Today we’re seeing other companies, such as GoPro and Parrot, feeling exactly the same kind of pain. Now, we not only feel somewhat lucky to have been the first to discover how tough the market was and be the first to get out, but also that our plan had always been to get commercial as quickly as possible. The silver lining in the consumer boom is that it forced the regulators to accelerate making commercial use legal.

Instead of UAVs, you’re now focusing on the Site Scan software for capturing and analysing aerial data. How does Site Scan work exactly?

Chris: Site Scan is a three-part product: the ‘box’ (the drone, camera and batteries), the iPad app and the cloud service, in conjunction with Autodesk. You simply push the button on your iPad and the drone takes off, takes the necessary pictures with a very good camera – the Sony UMC-R10C – with all the right settings and then transfers those images to the cloud where they are converted into 2D orthophotos and 3D models using the Autodesk engine. You then have data that seamlessly integrates with the Autodesk software. The drone part is now the least interesting – it works great and is fully autonomous, but that’s true for all drones nowadays. The ‘secret sauce’ is the software – the ability to not only capture the right data but then to overlay it with CAD data from the Autodesk tools, to be able to do measurements, to be able to visualise it in new ways in the conventional tools in the AEC (Architecture, Engineering and Construction, Ed.) market, like BIM360, Civil3D, Revit and even AutoCAD. As part of the Autodesk portfolio, our job is to basically be the ‘capture’ part of people’s ‘reality capture’ strategy.

You claim that the georeferencing tool included in Site Scan Manager enables orthomosaics and digital elevation models to be generated that are accurate to their theoretical limit. Is this true?

Daniel: Typically, the generation of an orthomosaic will lose between one and three times the accuracy of the ground sampling distance (GSD), so the theoretical limit of the accuracy of an orthomosaic is one times its ground sampling distance. In UAV-based photogrammetry, the GSD of the camera – let’s say it’s 1cm per pixel – gets diminished once you create the mosaic because of the computer vision used to stitch the images together. In that particular case, we ended up with about 2cm per pixel. So for a 10-acre site with five ground control points, we would achieve close to the theoretical maximum accuracy.

Daniel McKinnon, vice president, 3D Robotics (Note: In July 2017, McKinnon left 3D Robotics to become director of product management, infrastructure, at Orbital Insight.)

In November 2016 you announced the launch of Building Information Modelling (BIM) capabilities within Site Scan. To what extent is this a game changer for the construction industry?

Daniel: The capture part of reality capture is only as good as what you can compare it with: the ‘as built’ compared with ‘as designed’. The ‘as designed’ is a bunch of CAD files that designers and engineers use to prescribe the construction of a building or a piece of infrastructure. The ‘as built’ is what we measure, closing the loop of the design-build cycle. So, for example, with Site Scan you can make sure concrete is laid down properly, eliminating things like having to jackhammer overpours. So yes, this truly is a game changer for the industry. Whereas previously you would only catch an error several steps beyond it, you can now do it in near real time.

Chris: The digitisation of the built world, of the construction industry, is only half completed. We design things digitally, but once they get built they become analogue. So we need to be able to essentially ‘redigitise’ the physical world and then merge it with the design to see if anything has changed. The promise of BIM, with a fully digital workflow, is that modelling buildings not only enables you to catch errors during construction but also allows you to feed any changes made in the real world back into the digital model to keep it up to date.

How will this stronger focus on construction affect your commitment to the mapping and surveying industry?

Chris: Those two industries are on a continuum, and they’re moving towards each other. The GIS world is the world of latitude and longitude – absolute frames of reference in the real world. Meanwhile, the construction world has typically been one of CAD, which is all relative dimensions. Traditionally, the Autodesks of the world work in coordinate systems that are relative to a building site, whereas the GIS companies often don’t get into that sort of building-level granularity. What we’re seeing now is two worlds starting to converge: the AEC world is starting to move outwards, from just the building to the building on the site and the site in the city – all part of the broader urban infrastructure. Meanwhile, as resolution continues to increase, the GIS world is starting to move into the actual building level. The building site is our starting point, because there’s a clear and present need in the construction industry, but as we build these tools we’re finding ourselves extending out into GIS more and more. So we started with Autodesk, but we’re now working very closely with Esri on the GIS side and I think will we see some commonalities, some harmonisation between these two industries on things like file formats, coordinate systems, databases and so on. We feel we have the potential to be a bridge between those two industries.

So 3D Robotics manufactures UAVs and software, but how about the payload? Who supplies the systems?

Chris: We manufacture the gimbal and the interfaces and all the other bits necessary to make the Sony R10C camera appropriate for use on a professional drone. That’s all we do right now on the hardware side, and over time the entire industry is becoming somehow agnostic about the drone. Companies like DJI and Yuneec make amazing drones. It’s like with smartphones; you can buy a Samsung or LG or HTC or whatever – their phones are so sophisticated that you can assume the hardware is good enough. We’re very close to that point with drones as well.

Will you be focusing on any other mapping technologies, such as Lidar?

Chris: Lidar and other laser ranging techniques are very hot right now. In the same way we are agnostic about the drones, we want to be agnostic about the sensors we use. Right now we use cameras just for photogrammetry, but the same tools we use – the Autodesk Recap engine and others – can actually take input from Lidar too, so you can combine point clouds from Lidar with point clouds from photogrammetry. Imagine a building – for some aspects it is better to survey it from the ground. Anything that involves looking up, behind obstacles or inside is better done with Lidar. Other things, especially things that are higher off the ground, are better done with a drone, with photogrammetry. The ability to combine aerial photogrammetry, ground-based Lidar and interior photogrammetry into one holistic point cloud is the workflow we’re working on right now with Autodesk.

Our recent readers’ survey indicated a rising demand for accurate and easy-to-use point cloud processing software for UAVs. How are you capitalising on this opportunity?

Chris: There are three elements in the point cloud processing software. One of them is the acquisition of the data in the first place, which is done with our mobile app. The second is the creation of the point cloud, which is done with the Autodesk Recap Engine. The third is the tool to visualise it, to combine it with other data, to annotate it, et cetera, that’s what we do with our web tools. You’re absolutely right when you say that today’s tools are still a little fragmented and are often hard to use. So we see a huge opportunity to simplify the workflow and make it easier to capture, process and visualise point clouds.

Is there anything else you want to share about the surveying and mapping industry and the applications of UAVs in general?

Chris: My message is simple: the sky is empty! It’s really amazing that there is still an opportunity that big and that untapped today. The industry has done extraordinary things with satellites, but two-thirds of the planet is covered by cloud at all times. Aeroplanes are useful but they are expensive and not available everywhere. With drones it is now possible to put a sensor in the sky pretty much anywhere, anytime, for almost no cost. I think this is as important as the origins of the satellite industry in the first place. Not only are drone-carried sensors cheaper, available nearly anywhere and anytime and able to capture data at higher resolution due to their lower altitude, but they are also connected to the internet. So these sensors benefit from cloud data processing, providing insight in near real time. In the same way that the internet democratised telecoms, I think drones will democratise geospatial information gathering and analysis. Everybody will soon have access to the tools that only satellite owners had just a few years ago.

Chris Anderson

Chris Anderson is CEO of 3D Robotics, founder and chairman of the Linux Foundation’s Dronecode Project, and founder of the DIY Drones and DIY Robocars communities, including the ArduPilot autopilot project. From 2001 to 2012 he was the editor-in-chief of Wired magazine. Before Wired he spent seven years with The Economist in London, Hong Kong and New York. Anderson is the author of the New York Times best-selling books The Long Tail and Free as well as, most recently, Makers: The New Industrial Revolution. He was listed among Time magazine’s 100 most influential people in the world in 2007 and in its Tech 40 as one of the most influential minds in technology in 2013, as well as one of Foreign Policy magazine’s Top 100 Global Thinkers in 2013. Chris Anderson has a background in science, starting with studying computational physics and doing research at Los Alamos and culminating in six years at the two leading scientific journals, Nature and Science.

Daniel McKinnon

Daniel McKinnon is vice president of product at 3DR. He has worked closely with the Autodesk ReCap team to build Site Scan, an intuitive application for autonomous drone scans that seamlessly integrates with the Autodesk software ecosystem. Site Scan enables Autodesk customers to generate aerial imagery, orthophotos, point clouds or meshes for infrastructure planning, construction monitoring and myriad other uses. McKinnon completed a bachelor’s degree in physics and doctorate in chemical engineering before co-founding Agribotix, an agricultural survey company, after being conscripted by the Denver Zoo to develop a drone for wildlife observation and capture. At Agribotix, he worked extensively with growers and agronomists to develop both a front-end data collection workflow and a back-end data processing solution that quickly and easily gets valuable actionable intelligence into their hands. These efforts have resulted in the establishment of several concrete value propositions for drones in agriculture as well as numerous lessons for what does and doesn’t work in the field.

Note: The interview was published in the April 2017issue of GIM International. A few months later,  Daniel McKinnon left 3D Robotics and became director of product management, infrastructure, at Orbital Insight.

More information: www.3dr.com.