Wheat is the world’s most planted crop, accounting for more
than 40% of global protein intake.
Helping wheat growers to increase yields, then, is a logical focus to ensure that the world’s population has consistent access to food and essential nutrients.
On one side of the
scanning device, laser light projects the patterns. On the other side, a camera captures the reflected light.
Gathering this intelligence data has traditionally involved rulers, hand-held cameras, and manual measurements. For centuries, agronomists have relied on manual methods that have provided precise data. But these approaches are labor-intensive and time-consuming.
Automating measurements
A team at National Physical Lab (NPL) in England is combating these shortcomings using 3D scanning analysis. The team constructed a wheeled rig, to which it mounted four Photoneo PhoXi 3D XL scanners (distributed by Multipix Imaging). The vehicle moves around plots within a field to capture complete 3D scenes.
The laser scanners use structured light to create high-resolution, millimeter-precision 3D point clouds in less than a second. The technology illuminates an area in a pattern formation. On one side of the scanning device, laser light pro-jects the patterns. On the other side, a camera captures the reflected light.
The National Physical Lab (NPL) mobile 3D imaging demonstrator platform, equipped with four Photoneo scanners and a control laptop, performs
a test prior to its deployment on a farm vehicle. Courtesy of Richard Dudley/NPL.
The system scans for deformities, delivering information on depth, or the distance between points imaged by the projections. Out of these points, the scanner creates a triangle to capture and deliver visual information in image format. The scanners then output the image data, which reveals plant traits at a level of precision that allows breeders to develop a better seed.
“If you talk to seed breeders, they have a list as long as your arm — we are talking maybe 100 different traits of the crop that they want to look for when they are growing,” said Richard Dudley, NPL’s science area leader for electromagnetics and precision agriculture. “We focused on some of the dimensional ones that we could see.”
Merging methods
The NPL system incorporates the benefits of multiple imaging techniques. Though each technique offers distinct benefits, each on its own is insufficient to provide the speed, resolution, and adaptability that the Photoneo 3D
scanning technology brings to the
application.
The capture of a single, complete outdoor wheat trial plot measuring approximately 1.5 × 3 m. Clustering algorithms are used to highlight the wheat ears. Courtesy of Richard Dudley/NPL.
Time-of-flight imaging, for example, does not produce the necessary
resolution, especially in three dimensions. Even 2 cm of spatial inconsistency can be consequential when precise measurements of a small seed are needed, said Peter Matus, a 3D vision consultant at Photoneo.
And conventional and hyperspectral cameras require high computational power to ultimately deliver lower-quality images than those from the Photoneo scanners, according to Dudley. Hyperspectral imaging provides valuable information on plant coloration, such as amounts of chlorophyll, but it takes multiple hyperspectral cameras working together to produce useful 3D representation.
The parallel structured light technique is an effective 3D imaging method developed by Photoneo that currently powers its MotionCam 3D camera. The system illuminates a whole area with a single laser sweep, Matus said. The camera sensor features a mosaic shutter that enables the programming of individual groups of pixels to produce a specific, structured pattern.
The Photoneo technology enabled the NPL team to capture visual details
down to the individual ear of wheat. The information is used to determine the health of existing and future wheat crops. Courtesy of Richard Dudley/NPL.
NPL used the PhoXi 3D Scanner, however, which uses structured light technology to capture the complex and nonuniform shapes and angles of wheat crops, to produce images at high resolution and with a high degree of accuracy.
Optimizing performance in the field
While system illumination must be considered, natural light is especially important because wheat measurements are taken during the day. Any
3D scanner performing in the daytime will suffer from issues caused by light saturation, but, according to Dudley, the Photoneo scanners are particularly adept at overcoming high levels of saturation without compromising speed and resolution.
“How you operate it in the lighting
conditions is key for us,” he said. “We
spent a lot of time optimizing the settings in the scanner — how we mounted them, how we illuminated them, if we needed to at nighttime, how we shielded them if we needed to — so, that was quite a big challenge.”
Because the system relies on multiple scanners working at once, it was difficult to align all the captured images in a way that was less time-consuming yet also accurate, Dudley said.
“If you take the pull-out capture
from four angles and imagine it around a square, you need to be able to
merge those four independent point clouds together with submillimeter resolution. Otherwise, it looks very blurry, and you’ve calculated them incorrectly.”
Photoneo provides an alignment chart, along with built-in software that helps the scanner perform automatic alignment.
“And we really pushed the bounds of that,” Dudley said. “We looked at how we could change those charts, change the colors, make them larger, and make them in three dimensions.”
Advancing the application
PhoXi 3D scanning technology
delivers resolution of up to 3.2 MP in each scanned 3D image. It is also capable of imaging at various fields of view, depending on the density of the point cloud that the scanner is acquiring. NPL’s application required these capabilities and others.
Working with 3D point clouds of wheat data taken from a single scanner is challenging because the shape of each wheat plant is different and the plants are imaged against a complex
background rather than a clear,
contrasting one. So the team used 3D point cloud data taken from four scanners, which obtained images from different directions and delivered point cloud shapes that were then merged into more complex data clouds to provide a full representation of a wheat plot.
Refinements to the Photoneo technology have allowed the work at NPL
to progress. Software improvements that enable various calibrations, such as thermal calibration, now support scanning outside of neutral settings.
Other advancements to the software
enable changing the degree of centi-
grade and adjusting dimensional properties. Both of these are important for conducting measurements, Dudley said. The software provides users with information based on real number
values, as opposed to visual point clouds in a relative space. The values add precision to an evolving application that must be as exact as possible.
Looking forward
NPL’s mobile system completed
its first commercial run in 2020, and the system was deployed again this summer.
Following the data collection stage, the team uses custom-developed
computer vision, machine learning,
and image processing algorithms to further scrutinize images to determine the object (wheat) and its individual components (such as stems and grains) and separate them out. Because the images and therefore the useful data are provided in 3D, the process is more challenging than it would be with 2D images.
In the future, the NPL team would like to scale up its method, in line with the needs of seed growers and farmers, by increasing throughput and the volume of measurements that it is able to obtain.
“These technologies are new in
these specific types of environments,” Photoneo’s Matus said. “They usually take a lot of computational power.” For many applications, this level of power is not yet available to those who need it.
“In some cases,” he said, “AI-based programs could help out, just to
simplify the sheer volume of computations required.”