Pioneers in Capturing Public Space

CycloMedia is a renowned mapping company specialised in large-scale and systematic visualisations of environments. Its headquarters are situated in The Netherlands, a country that is an excellent testing ground for continuous innovation in mobile mapping solutions. GIM International's Wim van Wegen took to the highway and headed south to the old town of Zaltbommel to talk with Frank Pauli, CEO of the developer of advanced camera and image processing techniques.

You were appointed in 2013 as CEO at CycloMedia. An important part of the company’s strategy was/is the focus on the international markets. Where are we today?

It certainly is part of our strategy; we are convinced that our technology is unique. Technology that is being developed in The Netherlands, but meeting needs and supporting use cases that exist in many markets worldwide. However, in growing internationally we are quite selective: our two main priorities now are Germany and the United States. We have established two subsidiaries; one in Berkeley, California, and the other one near Frankfurt, Germany. Our focus is on the city customer, which has also been the basis of our growth in The Netherlands. There are two reasons why cities and local governments are our main customers. The first one is that these customers are using our data intensively, so that is obviously interesting in a commercial way. The other reason is that a city is a relatively limited capture area. We can capture a typical city in a few weeks or a couple of months. In Germany, we have mapped cities such as Frankfurt, Stuttgart, Mannheim, Cologne and Duisburg, to mention but a few. In the US, we did a similar rollout - Washington DC was one of our first clients there. Since then, we have signed contracts with Columbus, Philadelphia, and one of our milestones is New York City. In late 2016, we concluded an agreement with the city of New York, and then we captured the entire city in the first half of this year.

What kind of solutions do you offer that they cannot find in their domestic market?

Our core product is our panoramic imagery – what we call the Cyclorama. The main differentiator of our panoramic imagery is our positional accuracy, and this is what distinguishes us from what other vendors have to offer. None of the local vendors offer the same accuracy, and nobody offers it on a scale that we offer. The use cases in these markets are very comparable; when we look at The Netherlands, our primary use case was always tax assessment. In the US, this is similar - tax systems, not so much managed by the city but by the county. In Germany, tax assessment does not exist, but there all the other use cases are similar to The Netherlands, for example, asset management, urban planning, utilities (network planning and asset management) and safety & security. The use cases are very comparable across the markets that we address.

Your company is well-known for its 360° panoramic imagery. What are the latest technological developments when it comes to capturing the images?

We migrated from standard resolution to high resolution, and we started this in 2015 – the 100 megapixel panorama imagery. Second is accuracy, and we are continuing to invest in this accuracy, which we call the CycloPositioner. After the imagery has been captured, we are able to further enhance the accuracy by applying processing technologies and if required by integrating ground control points. A third invention which I think is very important is that we have added Lidar to our collection system. Until two years ago, we were focusing entirely on imagery, but since then we have added Lidar data to this and we are rolling it out now. In the United States next year all our mobile mapping will include Lidar, and also in other countries we are seeing more and more customers who are interested in Lidar.

And when are customers particularly interested in the application of Lidar?

Something that all our customers are interested in is the further enhancement of measurement capability. Accuracy is most important in our imagery and, based on this accuracy, customers use the imagery not only to do analysis and modelling, but also to take measurements. These measurements have always been based on imagery, and when we collect Lidar we create depth panoramas which we put behind the imagery, on the basis of which you get a 3D representation. We have added a 3D cursor which moves with the surface that is in the image. Lidar makes measurements far simpler, particularly if you have multiple pixels of the same colour and structure next to each other. In this case, you need to select the right pixel using the 3D technology that we base on Lidar – that is an immediate integration in our imagery. Furthermore, I would say that application of the Lidar that we collect is comparable to other Lidar point clouds as it is typically used in engineering, simulation and asset management. The fact that we combine the two captures in one production process gives our point cloud two unique elements: the accuracy – not only for the imagery but also for the point clouds – and the combination of the imagery and the point clouds which enables us to do a better job of colouring the point clouds - taking the colours from the imagery and taking them into the point cloud.

A few years ago, CycloMedia upgraded its 3D processing technology based on point cloud and mesh processing methods. Can you tell us something about this?

This continues to be an important area in which we invest. From the technology point of view, we were quite early in developing technology that allowed us to create meshes. While talking about collecting imagery and Lidar we are looking at combining these data sources into 3D. Certainly in The Netherlands we are not only actively engaged in street level imagery but also in aerial imagery. We are looking more and more at how we can integrate street level imagery with aerial and, in particular, oblique imagery. Another area in which we are investing in 3D is on the business side of things. 3D is one of those areas where many companies are investing in moving the technology forward, which is great. However, you also need to have a clear idea about where you want to sell this kind of content. We invest quite a bit in educating the market, in working with partners and customers. It’s about how we see the future of 3D becoming a reality. Technically, a lot is already possible, but in terms of platforms, use cases and users, there is still quite a bit to be done. In this context, we are involved in an initiative in The Netherlands with Kadaster, Esri, and Delft University of Technology to work on possible national coverage on 3D. Obviously, we are employing our products and technology for this purpose, but we also find it important to invest in understanding the market and in building solutions that the market needs.

Your Lidar point cloud data can be imported into software from various companies, such as Autodesk, Bentley and Esri. How do you team up with these software suppliers to offer a joint workflow solution?

We see ourselves as a data provider, not as a software company. We have some software solutions ourselves, but that’s not our core focus. Our core focus is to collect data and create content, so we are interested in teaming up with all sorts of software companies that are relevant. The relevance is largely determined by our customers. For Lidar it’s a bit simpler, because Lidar has standard formats, so when you produce your Lidar in LAS or LAZ files, all the major engineering software companies support that format, so there is not that much to interact. For panoramic imagery, this is a bit different, since there is no worldwide format. Apart from our own software viewer solution we create our own API, which is available to every software company wanting to interact with us. We try to team up more closely with some software companies than with others. A good example of this is our US rollout, where we strongly cooperate with Esri. We focus on local and regional governments there and Esri is clearly the market leader, so it is also in our interest to make sure the interaction with Esri works. Apart from making our API available there, we also developed some add-ins to enhance the integration of our content with Esri, and we do co-marketing programs together with them as well.

Aerial imagery is also one of the pillars of your solutions. Are there any significant technical developments in the acquisition of this data?

It’s part of our solutions, but only in The Netherlands I would say. In The Netherlands we currently have a strong market position where we support many customers, and for these customers we create added value by providing both street level imagery and aerial imagery on an integrated basis. We host the data together, we have cross-referenced the data, and we have created a software solution which allows our customers to view a location in the public space both from a street level perspective as well as from an aerial perspective. So, I would say we are more of an integrator and it is actually more of a business proposition than a technical proposition. We have not created the aerial products we sell in The Netherlands ourselves. We partner with other companies and based on this we make the images available. There have not been that many technical developments in aerial imagery in the Dutch market in recent years.

Oblique airborne photogrammetry has evolved from a ‘nice to have’ towards an increasingly important element for the workflow of service providers. How is CycloMedia dealing with this trend?

In oblique imagery there is a lot of innovation going on. In The Netherlands, we do offer oblique imagery as part of our range. 2017 has been the third year in a row that we offer the whole country in oblique. We are seeing more and more customers using oblique imagery, so it is integrated in our viewer. In the United States, we don’t offer oblique imagery ourselves, but we partner with the leading oblique providers – companies such as Pictometry and Sanborn. We are seeing many technological advancements in oblique photogrammetry. Clearly, the resolution is continuing to grow there and we are seeing a number of initiatives where camera manufacturers are integrating Lidar in oblique. The ultimate vision that we have is combining high-resolution, high-accuracy street level capturing and high-resolution, high-accuracy oblique capture which gives you a total view of everything in the public space. This is the ideal basis for the further development of 3D.

You are also involved in 3D city modelling, which plays an important role in the concept of Smart Cities. What is your view on the future of Smart Cities?

The fact that cities are getting larger is a given if you just look at the urbanization data. If governments want to continue to manage these cities, they need to embrace the smart city concept, to which we contribute in multiple ways. Before we talk about 3D, our existing imagery also plays an important role in managing smart cities. I mentioned New York, which is probably the most challenging city in the United States. The fact that the City of New York selected us to collect the imagery for the upcoming years is a great example. New York has a number of use cases, such as tax assessment, asset management, urban planning, and safety & security. To me, that is very much a smart city at work. There is a lot of talk in the industry about smart cities - people are trying to get their heads around what they really are. I would say that when we provide our data to larger cities and when I see how often those cities are using our data, this is certainly an element of the smart city. I was recently with one of our German customers, looking at their usage of data last year, and we discovered that about 4,000 people working for the city had clicked on the data more than a million times. That means heavy usage and that the larger cities in the world where we are active are using our data to manage their city in a smarter way in our standard product. That standard product is then integrated in the software solutions we spoke about before.

In moving forward, we see three directions where our content can play a role in making cities smarter. The first one is 3D and we are actively engaged in converting our panoramic imagery into 3D, which will help to manage cities better. From our perspective, the difference between imagery and 3D is not that large, but it is a good next step. A second direction involves what we call information products. These are products where we create asset databases from imagery. These can be simple assets like traffic lights, traffic signs and pedestrian ramps. These are important to the management of cities and most cities we know don’t have an accurate and up-to-date database of these assets. We have developed a number of automated algorithms based on which we can create these databases. The cities either do the work themselves or we offer to do it. And as we capture our imagery on a high update frequency, typically every 12 months, we can also update these databases. The notion of what we call information products is a second element involving the development of our imagery to help cities be managed in a smarter way. The third area is what we call content collaboration, which is all about capturing data with more sensors. It is great that we can drive around the city once a year and publish new and accurate imagery, but there are more and more sensors with which you can capture imagery during the year. This can be done with different cameras but also with other sensors like the ones in connected passenger cars and those used to measure temperature, air quality etc. Content collaboration is all about enabling our customers to integrate their own data with our data. Our data has the advantage that it is correctly positioned, so when a city also sends its people out into the streets with mobile phones to take pictures, our newly developed technology - which we call the geometric framework - can be put to use. Pictures taken with your smartphone will not be so accurately positioned, but we match them against our imagery, allowing customers to get even that kind of data correctly positioned. In short, we believe our street level imagery is an important canvas to manage the city, and on top of that canvas we are developing three sets of technology to make it even more relevant: 3D, information products based on object detection and change detection and thirdly, content collaboration.

Technological developments such as Artificial Intelligence and Machine Learning are advancing rapidly. How do you incorporate these game-changing technologies in your solutions?

R&D was and is an important ingredient for the company. We started as a spin-off from the Delft University of Technology in 1980. We have a team of 35 R&D people in house, and the type of technologies that you mention are the basis of the products we spoke about in the previous answers. When we talk about object detection and change detection, the latest technologies in terms of machine learning and deep learning are integrated in this. Also for some of our basic products. We need to blur our images because imagery resolution is getting higher and privacy regulations are becoming stricter. We do this with deep learning in order to do it in an automated way. We make many millions  of pictures per year, so it is obviously not an option to do this by hand. In the Netherlands, we have built a database in which all approximately 3million traffic signs have been captured by using object detection technologies.

You are based in The Netherlands. What are the main advantages of this?

Expectations, norms and standards for geospatial information in The Netherlands are very high. The Kadaster (The Netherlands' Cadastre, Land Registry and Mapping Agency, editor) is considered to be one of the world’s leading national mapping agencies. The Netherlands has played a significant role in the development of companies like TomTom, Tele Atlas and Navteq, and it is all based on the fact that geospatial data is considered to be important in this country. Probably the fact that 40% of the country is situated below sea level helps there, and the fact that the country is densely populated. The Netherlands is an ideal base for a company that wants to be forward-looking and technology-driven in geospatial information. On top of that, there are many well-educated, talented people here, which helps us to get the right people for our R&D department. After having developed and tested our products in our home country, it is relatively simple to bring them to other markets.

Self-driving vehicles are becoming a reality. When do you think we will be seeing self-driving CycloMedia vehicles capturing imagery all over The Netherlands and other countries?

That’s a good question! We talk about it a lot, but we have no concrete plans in terms of when that will be. Other than using the technology ourselves, given the fact that our imagery is so accurate it could be one of the ingredients in self-driving vehicles. Although it’s not our core business, we are already doing some work on this. I personally believe that it will still take quite some time before one of our vehicles can drive really autonomously in the city centres. Doing that on the highway is not far away. In The Netherlands we capture 160,000km, only 9,000 of which is on the highway. I think here it will still take some time before we can capture the entire road network with self-driving cars.

The future of 3D modelling is something that is widely reported. In which direction do you think reality modelling is heading?

I think there are multiple answers, and it very much depends on the use case. One of the answers is that we believe that there are a number of use cases where the photo is still a better answer than the model. A model is good when you are at a higher abstraction level and looking at more macro trends. When you want to assess the value of an individual building, the original photo is still more accurate l than the 3D model. 3D, however, will be more and more important. The question is: is that more important in our business, where we support professionals, or is it more important to society as a whole. A good example of this in The Netherlands is the Environment and Planning Act, where the government has taken on the obligation to enable every citizen in the country to look at their own environment. I’m convinced that the only way to do this in a meaningful way is through 3D. Many people are not good at interpreting maps and complex computer interfaces are not user-friendly, so if you want to inform citizens about their environment, 3D is the answer. Not only for consumers but also for urban planning, simulation of water and sound, and the simulation of city modifications. All these kinds of use cases incorporate a very important role for 3D. And then there is the third answer: for engineering applications, where 3D models are purpose built for industrial sites that need to be re-engineered or maintained. This is also another very important market which will be based on integrating more static and more dynamic data. We take our images every 12 months, but if you want to reconstruct buildings, you need to make sure that you can also iterate images that were taken yesterday or today and not only six months ago.

Earlier in this interview you mentioned the importance of educating the business. A good topic for the final question! What is your view on this?

We are a commercial company, so we need to have a healthy financial situation. Half of our business is about R&D, pushing the envelope and investing in new technology. The other half is about making sure we can sell the technology. Certainly, when you are a pioneer and want to be leading-edge, it’s not always a matter of finding unique demand or a unique use case that has not yet been met in the market. It is also a matter of making the market, and 3D is a great example. There’s a lot of investment in 3D, and I’m sure 3D is happening and will become a very large market, but the question is which submarket will develop first? What use case will be developed first? From our perspective, we want to find that out for ourselves rather than wait for the market to come to us asking if we can make this or that. That’s what I mean by investing in educating the market, and that’s exactly what we do with 3D. We have now built a next generation of 3D texturing technology, which we are now offering to some municipalities in the Netherlands who are also more forward-looking with regard to 3D. We have agreed to do a pilot together to build the city in 3D and experiment with citizen participation, urban planning and other use cases. We try to be active on that side of the business chain as well.

About Frank Pauli

Frank Pauli is the CEO of CycloMedia since 2013. Prior to that he was with Navteq/HERE Maps for over 5 years where he was heading up the Mapping and Content business in the EMEA region. Before that he worked many years for Philips Electronics in various roles in business management and innovation, which included general manager of the DVD business unit and managing various innovation projects including Connected TV and Blu-Ray. Pauli has an MBA from Delft University of Technology and lives in The Netherlands.