Photogrammetry Method Measures Wings During Flight

One of the most important goals of the aviation industry is to make aircraft more fuel efficient. To this end, the Fraunhofer Institute of Optronics, System Technologies, and Image Exploitation (Fraunhofer IOSB) is working with Lufthansa Technik to conduct flow simulations that take the shape of the aircraft’s wings into account. Lufthansa Technik uses AeroShark, a coating technology inspired by sharkskin, which significantly reduces frictional drag and, therefore, emissions.

To conduct these simulations, engineers perform flow analyses with 3D models of the aircraft. A classical 3D model of the stationary aircraft can be created in the hangar using established metrological methods. However, this is not sufficient to understand how the wings behave during flight since part of the tank load is in the wings and is consumed during flight.

The black measurement marks allow distances to be measured using just one camera. Courtesy of Lufthansa Technik.

During flight, the aerodynamic uplift and the change in tank load influence the curvature of the wings. The wings deflect upward by several meters depending on the flight section. Using models that reflect the actual aircraft shape during flight, computer-aided flow simulations (computational fluid dynamics, or CFD for short) of the sharkskin technology can be carried out to determine its optimum position and orientation.

“Usually, at least two cameras are needed to create such a 3D model,” said Karsten Schulz, head of scene analysis at Fraunhofer IOSB. “However, the conditions for stereophotogrammetry are often not met, for example because it is not possible in the passenger cabin to correctly align multiple cameras with a view onto a wing. Our new measurement method enables accurate measurement with only one camera. This is an innovation for model creation in difficult use cases.”

One challenge of in-flight measurement is that it has to take place during scheduled flight operations in order to save costs and resources. The measurement equipment has to be placed in the cabin and measurements have to be taken through windowpanes, the effect of which is not always known in advance. Since distances cannot be determined with a single camera, existing solutions envision the use of multicamera systems. However, the installation of such complex systems in wide-bodied aircraft on scheduled flights with passengers is an impossible undertaking due to the required space.

The innovative photogrammetry process allows the creation of 3D models based on images from a single camera. Courtesy of Reto Hoffmann.

Fraunhofer IOSB solved this issue with a monocular approach. The missing distance information is obtained with additional measurements on the ground. For this purpose, the upper surface of the wing is covered with numerous measurement marks whose positions are measured using a tachymeter. A single camera is permanently mounted in the aircraft cabin where it records and locates these marks several times an hour under different flight conditions. The rest is handled through mathematics. A model for wing deflection is based on the assumption that arc lengths do not change during bending; at the same time, length changes due to temperature fluctuations can be taken into account mathematically. The positions of the measurement marks can be determined in the measurement images and converted into spatial coordinates. On-site camera calibration takes into account the influence of the window panes; the distance between the camera and the target marks is finally obtained by solving an optimization problem.

“We were able to successfully apply our patent-pending photogrammetry method with the support of an expert from the Karlsruhe Institute of Technology,” said Jochen Meidow, head of on-site surveying and a scientist at Fraunhofer IOSB. “On a regular flight from Zurich to San Francisco and back, we were able to obtain the 3D model of the wing of a Boeing 777-300ER and make it available to Lufthansa Technik.”

The method, Meidow added, can be used in other contexts where exact measurement is difficult.

Details of the photogrammetric approach were published in the Journal of Photogrammetry, Remote Sensing, and Geoinformation Science (www.doi.org/10.1007/s41064-022-00230-y).