Bridging the Gap between Aerial Imaging and Terrestrial Data Capture - 20/03/2019
The Challenges of Drone Use in Facade Mapping
There have been rapid advancements in unmanned aerial systems over recent years, yet there is widespread consensus that we have still only seen a fraction of how unmanned aerial vehicles (UAVs or ‘drones’) could ultimately improve efficiency for geospatial professionals and their customers. To get a glimpse of the future, 'GIM International' spoke to Mark Nicolai, co-founder of industry start-up Aeroscan. Here, he shares his vision of how drones could form the missing link between aerial imaging and terrestrial data capture.
According to independent research by the likes of PwC and Goldman Sachs, the construction industry tops the list in terms of commercial drone use and is by far the fastest growing sector. UAVs are becoming increasingly commonplace on building sites, enabling construction companies to monitor progress with ever-greater accuracy, efficiency and safety. The use of data captured by an unmanned aerial system (UAS) is also on the rise in 3D models for construction planning and design purposes. Drone deployment has reached a sufficient level of maturity that activities such as UAV-based roof inspections have become fairly routine. Now, a Dutch startup called Aeroscan is taking things a step further and exploring the use of drones for facade mapping. “If this can be achieved, it will have major implications – not only for construction and real estate but also for infrastructure, law enforcement, emergency services, local governments and so on. It relates to every business case already using aerial data, plus it will unlock countless new business cases by providing higher-quality data and more detailed information about the local environment,” states Mark Nicolai, one of the two co-founders of Aeroscan.
Founded in mid-2017, Aeroscan is actually the result of three businesses: a leading Dutch construction company, an IT company specialized in the development of cloud software for construction management, and Nicolai’s visual media company which he started in 2008 and which produced promotional videos and photos of building sites for construction companies, including using drones. “Our photos were mainly used for marketing purposes, but technical departments started to ask whether they could use the photos for calculation purposes too,” recalls Nicolai. “So we increasingly moved into drone-based imaging for construction firms and social housing associations. We have the necessary equipment, certificates and a dedicated drone crew to collect data. We also have customers who are already using our software and 3D models to obtain objective data as the basis for reports, calculations, estimates for repair, and so on. But we’ve noticed that large property owners have a need for real-world insights into the condition of their properties, including their facades, so we decided to take a risk and investigate whether UAVs could be used for this.”
Helping to shape EU drone regulations
“Together with two other partners, VMRG and Octo, we received €1.6 million of EU (EFRO) subsidy from the ‘Kansen voor West II’ programme as part of a three-year project called the Facade Service Application (FaSA). The project involves a consortium of dozens of contractors and multinationals from the entire chain of producing, installing and maintaining facades. We needed a drone operating licence for the built environment, so we applied to the Dutch aviation authority (ILT). When the people at ILT heard about the FaSA project, they invited Aeroscan to work within the framework of a Specific Operations-based Risk Assessment (SORA) for drones and to cooperate with them on the JARUS project which is helping to shape the future of EU drone regulations,” he continues. The Joint Authorities for the Rulemaking of Unmanned Systems (JARUS) is a group of experts from the national aviation authorities (NAAs) and regional aviation safety organizations working to recommend a single set of technical, safety and operational requirements for the certification and safe integration of unmanned aircraft systems (UAS) into airspace and at aerodromes. Presently involving 59 countries, as well as the European Aviation Safety Agency (EASA) and EUROCONTROL, the objective of JARUS is to facilitate each authority to write their own requirements. “In the JARUS project the focus is on presenting a real-life scenario and operation-based risk assessment and proving that your business case is safe. So this was a unique opportunity for Aeroscan to work on this conceptual new framework for future EU regulations based on our actual business plan,” explains Nicolai. “After a process of 16 months, we finally gained approval for our developed scenario in late December 2018 and we now have the most complete certificate that’s possible today: we are flying our drones on a kind of ‘temporary exemption’ based on future EU regulations that are still being developed!” he adds. As a result, Aeroscan became the first company in the history of the Dutch UAV industry to receive an official permit for flying drones in urban areas. “We’re gaining valuable knowledge and insights about how to investigate measures and risks in the field, which we share with ILT. By continuing to work with the authorities we hope to help expand the possibilities of flying in urban areas for the industry as a whole.”
Field pilots for housing associations
For its customers in the housing sector, Aeroscan’s long-term aim is to offer UAV city mapping in combination with photogrammetry/data processing, and to combine automatic image and object recognition techniques with UAV data to find flaws in real estate such as cracks in facades or window frames that need repainting. The company ultimately hopes to generate maintenance insights and data visualizations based on the analysis. So far, the main focus is on the operational aspect, i.e. data acquisition. “Since gaining the licence, we have conducted several pilot projects for social housing firms and property companies using existing off-the-shelf technology. In fact, our first pilot – mapping a row of terraced houses in a small Dutch city – was the country’s first ever official flight of a UAV over an urban area to inspect residential properties. As it was our first time in the field, we just wanted to explore the drone’s performance when exposed to environmental factors such as Wi-Fi signals, GPS, jamming, blocking, reflections, turbulence and gusting around buildings, for example.” The drone hardware Aeroscan uses is an Intel Falcon 8+, which Nicolai considers perfectly suited for inspection flights in high-risk urban environments due to its small size, low weight (just 2.5kg) and eight engines giving triple redundancy (i.e. it can still land safely even if two engines fail). In terms of the imaging sensor, the aircraft is fitted with the high-end Sony A7R full-frame camera to capture images in high resolution. As it turned out, there were no interference problems and the drone data was successfully post-processed into accurate and georeferenced nadir maps of the rooftops.
Simulating a multi-camera rig
In subsequent pilots Aeroscan has increasingly explored how to capture images of facades, because that is where the real added value lies for housing associations. “To tackle the challenge of vertical mapping, we took inspiration from the manned aircraft approach. We started to create flight plans to generate relevant datasets based on the workflow in Bentley’s ContextCapture, considering overlap rules, angle shifts between cameras and so on. In our most recent pilot for a social housing association in a densely populated urban area of a major Dutch city, we worked with an acquisition plan to simulate a multi-camera rig you would commonly find on a manned aircraft. So we designed a setup in which the drone passed each point multiple times, with the camera pointing in a different direction and at a different angle each time. Of course there’s only one camera on the drone, so we had to conduct multiple flights, which means it’s not really such an efficient approach right now.”
The Aeroscan team are undeterred by this inefficiency and see various ways to improve it. “The off-the-shelf hardware solutions aren’t based on a UAV city mapping scenario because the current legislative situation means there is no demand for them; so far, manned aircraft have been the only aerial option. We really need a multi-camera sensor that is lightweight enough to be carried by a small UAV but offers sufficient calibration to ensure fixed overlaps and georeferencing accuracy. But I don’t see it as being a huge problem to combine and refine certain types of existing hardware,” states Nicolai. “And the next-generation ‘beetle’ drones – which weigh just 10 to 20kg and combine vertical take-off capability with the range of a fixed-wing drone – could get close to matching the efficiency of a manned aircraft, for example. Having said that, their flight range of approximately 100km in an hour raises other issues, such as which safety standards will need to be in place to allow 100km beyond visual line of sight (BVLOS) flight missions on a predefined grid. So maybe UAV deployment in city mapping will remain limited to small-area projects for the time being, but even so it can still be an interesting alternative to the use of manned aircraft or terrestrial mapping.”
Conducting low-altitude flights in highly populated and congested urban areas raises not only safety issues but also privacy concerns. Nicolai explains that most clients are excited about working with UAV technology because drones are generally considered “cool and interesting”, but on a practical level the housing associations’ tenants are not always immediately keen to have a drone flying over their backyard or balcony. Aeroscan tackles these concerns from two angles: legislation and social acceptance. “On the legal side, we work closely with our housing association customers to ensure their rental agreements with tenants actually give them legal grounds to use drones for conducting inspections, for example. But more important to us at Aeroscan is social perception and ensuring we don’t invade anyone’s privacy, so we’ve developed an automated workflow to anonymize sensitive data such as faces or number plates,” comments Nicolai. “In the pilots, we’ve made it a priority to inform all the households that have been involved, including through panel interviews and information events afterwards to evaluate people’s experiences. We try to involve as many tenants a possible. After all, a drone is just another tool. In reality, it can be less invasive than the current manual inspection method in which inspectors physically visit premises and sometimes need access inside people’s homes and gardens.”
Now that the operational approach has been proven to work in practice, Aeroscan is advancing to the next phase of developing an end-to-end solution: processing. “During the pilot projects, the datasets were modelled in a pipeline created with ContextCapture. We then supplied those models to our clients as raw data. But we are about to launch our own cloud-based platform with enhanced visualization functionalities so that customers can gain access to the deliverables in their preferred data format, such as high-resolution photography, meshes, orthos or whatever,” he continues. “Then the next step will be to develop automated analysis to truly unburden our clients. That’s our aim: to take an industry-specific and customer-specific approach in order to provide the insights that social housing associations need. We’re partnering with OCTO – a specialist in smart asset management and big data analysis – within the FASA project, for example. By the end of this year, we hope to be conducting tailormade drone missions for property companies based on their predefined business cases: the defects or objects they want to count, measure or find automatically within an area. The information our solution provides will enable them to focus on their core business: planning and performing the necessary maintenance of their properties to ultimately make life better for their tenants.”
Although Aeroscan still has some development work to do, the company is clearly on the right track towards developing an industry-specific end-to-end solution that adds real value for its clients in the Dutch social housing sector. If this is a success, there is no reason why it cannot be rolled out internationally and/or used for other smart-city applications. It remains to be seen how quickly the industry will ‘catch up’ in terms of developing the miniaturized hardware that is required to expand the horizons of drone-based city mapping beyond the nadir view. The topic of legislation and flying permits remains a crucial issue, of course, but the Aeroscan team are certainly doing everything they can to ensure the upcoming EU-wide legislation works in practice so that professional users can truly exploit the full potential of drones for city mapping in the near future.
Last updated: 28/08/2020