ELENA BELETKAIA AND JOSE POZO, EUROPEAN PHOTONICS INDUSTRY CONSORTIUM (EPIC)
Lidars provide point cloud distribution — to represent the 3D environment — using the backscattering of a laser beam from the target surface. They have long been used for purposes of mapping and atmospheric sensing. Lidar is no longer a technology to consider for the future; it is becoming part of our daily lives.
Lidars are used in luggage handling, parcel delivery, robotics, and more. Interest in this technology for automotive applications also keeps increasing. The automotive industry demands much from lidar technology in terms of size, price, and reliability. The lidars need to be ready for placement in headlights, taillights, bumpers, roofs, and other locations in the car, at low cost and with high durability, low maintenance costs, and high stability. Using lidars in any of these locations has its own specific set of requirements.
Ibeo Automotive USA Inc. offers the whole processing chain, with over 20 years of experience in lidar, including developing the sensor and the processing of the point cloud, tracking algorithms, optic identification, car controls, and braking. The first lidar sensor the company developed with Valeo lacked in the parameters of size, price, and reliability. The sensor had a rotating mirror and four vertical layers, and the horizontal scanning was performed by rotating the mirror at a given rate. Such a configuration makes the device too bulky for many practical uses due to the space required for the rotating mirror. The motor that drives this mirror is an expensive component because it has to be reliable for the lifetimes of both the sensor and the car. Moreover, rotating mirrors have a limited vertical field of view.
Many of these challenges are addressed by the introduction of solid-state lidars.
Solid-state lidar
Solid-state lidar systems have no moving parts. Such a construction not only makes them more resilient to vibrations but it also allows them to be built smaller and enables more cost-efficient production. While it has many advantages, solid-state lidar in the automotive industry has to meet high-resolution requirements: 0.1° × 0.1°, horizontal × vertical (H×V).
The Beamagine lidar provides good resolution, not only along the horizontal axis but also along the vertical axis. Depending on the OEM specs, it can be customized to achieve an angular resolution down to 0.05°. This ensures it can detect low-height objects around which the vehicle cannot maneuver. Next, the lidar has to include a wide field of view with an opening angle of 120° horizontally and 20° vertically, and a long range of 300 m. It has to accommodate common reflectivity of 5% to 10% and an 80-m range, for when an obstacle has to be evaded, not driven over, because it is too high. To synchronize the camera with the lidar data, it has to function at 25 fps.
L3CAM lidar developed by Beamagine is built using a dual MEMS technology approach to provide a scanning system without mechanical moving elements, combined with a large entrance pupil diameter. This enables long-range detection within eye-safe power levels. The performance requirements include high resolution, wide field of view, long range, and high frame rate, which can be achieved with a particular scan mode. The scanning in solid-state lidar is achieved through optical phased array — that is, when optical emitters send out bursts of photons in specific patterns and phases to create directional emission. The focus and size of the scan can be adjusted without need for physical adjustment to the optical emitters.
A possibly fitting solution for solid-state lidar is a flash lidar, which has no rotating parts and just emits a big flash of light to illuminate the whole scene. But that requires a lot of optical power. For this reason, Ibeo developed a system based on VCSELs (Figure 1). Since VCSELs do not produce high power, the system has to be accompanied by higher-sensitivity detectors. One remedy is a single-photon avalanche diode (SPAD). SPADs are trigger devices with binary detection that can determine whether there is one photon or millions, or whether there is signal or noise that requires measurements to be repeated. A SPAD can take 1000 or 10,000 measurements of a distance of perhaps 10,000 or 20,000 pixels per second.
Figure 1. Ibeo’s system is based on VCSELs, to get a 2D array of emitters with an optical system in front of the array, and sequential flash or e-scanning. Courtesy of Ibeo.
To handle such data flow, arrays up to 1 MP can be built, which is beyond the requirements of the automotive industry. The data is then used to reconstruct a histogram with time-correlated single-photon counting (TCSPC). On the y axis, the number of times SPADs are triggered is plotted against time on the x axis. The yielding signal from the histogram is very similar to the analog signal, but with additional information, such as time of day or weather conditions, that can be extracted from this graph.
It has to be noted that in a measurement setting of 13 beams and 8 bits, for example, the flow of data will require the processing of 1.2 GB/s. Thus, another challenge will be to integrate a system on a chip, or to package the SPAD array on the same analytic chip as timed digital control. A control for the illuminator and digital processing would be required. In collaboration with CSMC Technologies Fab2 Co. Ltd., Fastree3D SA achieved development of the first megapixel chip with the first 3D stacked chip on multiple technologies.
Components and assembly
While many elements required for a lidar system are generally available on the market, not many can be used off the shelf. When considering a lidar for the automotive industry, it is critical to meet specifications at the early stages of manufacturing. Many challenges of the lidar system are related to optics. Weather, debris, pieces flying around, temperature fluctuations, and more can have a significant impact on the protective window and have to be mitigated while the highest transmission level is maintained. SCHOTT AG provides glass-based solutions combined with various coatings for the protective window. A hot air coating or heating layers, flex solutions, and hydrophobic solutions are scratch resistant and environmentally stable compared to polymer.
Additionally, for adhesion of multiple coatings, glass is better suited than polymer and glass lasts longer as well.
To account for each photon being measured through the optical pass, high optical precision, high quality, and thermal impact have to be considered (Figure 2). Various solutions can be found from SCHOTT, Research Fab Microelectronics Germany, Aixemtec, and others. A surplus of light entering the lidar detection system, such as direct sunlight, is another big challenge that needs to be solved along the optical pass. Patented technology developed by Surrey NanoSystems can mitigate this issue, improving stray light suppression in a wide angular space. This increases the signal-to-noise ratio and therefore increases detection range and provides better recognition of low-reflective objects.
Figure 2. The optical pass elements include glass filters, MEMS mirrors and mirror substrates, optical glass, and aspheric lenses. Courtesy of Fraunhofer IPT.
Other challenges are the balance between range and resolution, optical interference, parallax error in data fusion, and processing of the data.
The requirements for lidar miniaturization are producing a trend toward the integration and use of chips, photonic integrated circuits (PICs), and so on. LIGENTEC SA makes PICs from silicon nitride. Silicon nitride is a much more natural material to propagate light in a waveguide than silicon. A thick-film approach allows for much shorter bands
or tight turns. High yield is achieved due to small chip size. At the same time,
400 nm to 4 µm in transparency and power up to 10 W or more in one single waveguide can be handled.
So the all-nitride core PIC technology provides the benefits of classical thin-film silicon nitride with silicon photonics. Furthermore, use of PIC technology enables integration at the wafer level and production of hybrid PICs for scanning lidars (Figure 3). Hybrid scanning lidar uses a motor for vertical scanning and moves toward solid-state lidar while using VCSELs. The use of VCSELs requires additional attention to the system alignment.
Figure 3. An InGaAs wafer with photodetectors for the shortwave infrared (SWIR). Courtesy of Fraunhofer IAF.
Manufacturing solutions such as 2-photon polymerization (2PP) additive manufacturing from Nanoscribe GmbH and adhesive bonding offered by SCHOTT or Aixemtec enable assembly of lidars with high precision and resistance to vibration, shock, and other environmental factors. The assembly of a solid-state lidar is a complex multistep process. A machine made by Aixemtec can assemble the receiver and sender side relative to each other with a high precision in the micron range and can integrate UV curing for high bonding repeatability. A big advantage of the machine is that it can be reconfigured within 15 to 20 min to provide a different field of view.
Another approach is to assemble the system with the lasers, receptors, and electronics on it. DIOPTIC GmbH created a tool for the assembly of the imager where an exopod holds a laser. During production with an XY table, the whole imager is shifted, and once it is in the right position, it is glued, and the glue is cured with UV. Such assembly ensures that each unit is not misaligned.
Solving real-life problems
While autonomous driving is a leading focus of lidar technology development in the automotive industry, LuxC ApS has deployed technology focusing on emergency services such as police, fire, and ambulance. LuxC is building a holographic head-up display with night vision that comes from a lidar. The software interface includes arrival time, directions, emergency code, important objects (such as fire hydrants), and public buildings.
Considering that not all emergency vehicles are equipped the same way, such a system can offer live fleet management, redirecting the responders on the fly as an emergency is happening. Here lidar is used for building an augmented image and is integrated in a complementary system connected to screens in emergency vehicles. Lidars can include such critical information as the direction that surrounding cars or other moving objects are traveling, which protects emergency personnel and the people they are serving.
Meet the authors
Elena Beletkaia, Ph.D., is project leader at the European Photonics Industry Consortium (EPIC). She graduated from Lomonosov Moscow State University, where she specialized in biophysics. In 2015, Beletkaia received a doctorate from Leiden University, where her research focused on the biochemical and biophysical mechanisms underlying the metastasis of Ewing sarcoma. Later, in the collaborative setting of NKI (the Netherlands Cancer Institute) and the University of Twente, she investigated the application of noninvasive spectroscopic/multiphoton methods for intraoperative resection margins assessment. All of Beletkaia’s research projects have involved the understanding and use of multiple microscopic and spectroscopic techniques.
Jose Pozo, Ph.D., is the director of technology and innovation at the European Photonics Industry Consortium (EPIC), with close to two decades of experience in photonics technology. He has a doctorate in electrical engineering from the University of Bristol in England, and bachelor’s and master’s degrees in telecom engineering. Pozo is a member of the board of the IEEE Photonics Society Benelux.