On-Chip Lidar Sheds Cost, Size, and Power Use to Expand Adoption

GREG SMOLKA AND MICHAEL R. WATTS, ANALOG PHOTONICS

There are a number of applications for which 3D sensing is rapidly emerging as a critical sensing modality. 3D and 4D sensors are arising in autonomous vehicle applications because they are propelled by products that are being developed for advanced driver assistance systems (ADAS). Additionally, they are beginning to be deployed in volume for security, traffic control, industrial and factory automation, and a number of other applications.

Lidar is attracting great interest as a 3D sensor technology due to the point cloud data it generates, which offers rich information regarding objects in the environment, such as their range, intensity, and, in some cases, direct velocity. Though lidar sensors are not new (the first system was reportedly built in the 1960s), their size and cost have prevented their deployment for all but a few specialized applications.

The emergence of ADAS and autonomous vehicle applications has helped change this because companies poured billions of dollars into the development of sensors that they hoped could be built for low cost at scale, while delivering the rigorous performance that is required for automotive applications (Table 1).

Table 1.
Desired Specifications for Long-Range Automotive Lidar

Even with the enormous resources poured into the problem, the combination of requirements such as high performance, high reliability, low cost, small size, low power, and automotive environmental specifications has proved to be very difficult to solve.

To illustrate this problem, consider the initial Velodyne HDL-64E lidar units deployed on autonomous test vehicles during the aughts. Affectionately described by Aurora CEO and autonomous vehicle pioneer Chris Urmson as “spinning KFC buckets,” these units were used successfully during the DARPA Urban Challenge and demonstrated how valuable lidar data is for autonomous driving. But clearly, they were not suited for general automotive production. Substantial performance improvements as well as size and cost reductions were needed to get to automotive scale (Table 2).

Table 2.
Improvements Required from Velodyne HDL-64E Lidar

In addition, factors that are unique to the automotive world needed to be considered. The industry expends considerable engineering efforts to maximize vehicle efficiency, as well as to optimize the feel of the ride, reduce cabin noise, and electrify the fleet. But many automotive engineers would say that the studio, or the design team, is who really calls the shots. In the eyes of the design team, the ideal lidar sensor is so small and unobtrusive that it can be seamlessly embedded into the vehicle exterior without visually interrupting their well-crafted lines. Today’s typical autonomous vehicle performs exceptionally well, but the myriad of large sensors that surrounds the vehicle is the exact opposite of what the studio wants (Figure 1).

Figure 1. Frequency-modulated continuous wave optical phased array lidar data captured when operating indoors (top) and outdoors (middle). Velocity information is depicted as color changes in the point cloud. Some FMCW optical phased array systems can achieve a 100-m range performance without any moving parts (bottom). Courtesy of Analog Photonics.

While first-generation lidars made great strides in performance and size, they are still viewed as too large, unreliable, and costly for mass deployment. Limitations that are maintained by their technology and architecture prevent them from making anything but incremental improvements. To achieve the performance, size, and cost goals demanded by the automotive industry, a paradigm shift is necessary.

Fortunately, a new class of second-generation lidar sensors is emerging to meet this need. Borrowing from the technological progression in automotive radar, these new units are based on coherent and frequency-modulated continuous wave (FMCW) lidar detection that incorporates most or all of the optics onto photonic integrated circuits (PICs) and, in some cases, are completely solid-state — steering the beam with optical phased arrays (OPAs).

These fundamental changes to the system architecture result in high-performance chip-scale lidar sensors that can be built, tested, and burned in on large 300-mm wafers in production foundries. The resulting sensors are small, highly reliable and reproducible, and low cost.

Coherent detection

Most current lidar systems for ADAS or autonomous vehicles in operation use time-of-flight (ToF) detection. These systems emit a short pulse of light that bounces off objects in the field of view and reflects back to a detector in the sensor. The range to the target is determined by measuring the time it takes the pulse to travel to the target and back to the detector. These systems work well, but due to laser pulse widths between 1 and 2 ns and high peak power, they are poor candidates for chip-scale photonic integration.

Coherent detection offers an alternative measurement architecture that offers significant performance advantages and is much more amenable to chip-level integration.

Miniaturization and integration are key to the automotive industry’s aim to incorporate lidar into vehicles on a broader scale. Leveraging silicon photonics technology will be critical to that aim. Courtesy of Analog Photonics.

FMCW lidar uses low-power continuous-wave lasers that couple very efficiently to optical waveguides and components fabricated onto silicon wafers. This results in incredibly small devices that can be manufactured at semiconductor scale and with semiconductor reliability.

The advantages of FMCW lidar go well beyond the technology’s cost and manufacturing gains. In addition to providing range and intensity information, FMCW systems detect velocity with every pixel. This is invaluable information to perception teams who build algorithms not only to detect but also to classify all the various objects in the field of view.

For example, direct velocity measurements on some FMCW systems are sensitive enough to detect the turn of a pedestrian’s head or shoulder as they contemplate entering a street. Perception teams can use this information to make decisions and give vehicles much more time to safely react.

In addition, unlike ToF technology, FMCW system performance does not degrade in bright sunlight. These systems perform the same whether they are staring directly into the sun or immersed in total darkness. A coherent receiver allows the FMCW detection technique to act as a very narrow filter, cutting out all but the reflected signal from the target. Systems that use OPAs as steering mechanisms add a further layer of very tight optical filtration.

These characteristics allow the FMCW OPA sensor to be immune to bright sunlight or any other stray light that adversely affects ToF systems. And while immunity from bright sunlight is important, immunity from other lidar systems is critical.

FMCW lidar sensors can reject laser light from other lidar systems, and even other FMCW lidar systems. Thus, vehicles equipped with FMCW OPA lidar can operate in an area with hundreds of other lidar-equipped vehicles with absolutely no change in performance. The same cannot be said for ToF lidar.

Solid-state 2D beam steering

Though FMCW detection offers great advantages in performance and chip-scale integration, another key problem still needs to be solved. Lidar sensors “see” their environment by illuminating the scene with lasers.

Short-range lidar surrounds the vehicle and identifies any object in the immediate vicinity. Courtesy of Analog Photonics.

Covering a large field of view requires an abundance of lasers to operate at the same time (flash lidar) or beams that are scanned through the field of view (scanning lidar). Scanning lidar is favored for long-range applications due to the high resolution that is required to detect and classify small objects, such as debris on the road, at a long enough range for the vehicle to safely and comfortably react.

Many companies use scanning mirrors, either discrete or MEMS mirrors, to cover the field of view. Due to the size, cost, and reliability concerns regarding mechanical scanners, auto manufacturers strongly prefer a solid-state solution with absolutely no moving parts.

In addition to the obvious reliability improvement, an all-solid-state, chip-scale solution comes with fewer parts, which further simplifies the supply chain, reducing costs.

Another advantage with a true solid-state sensor is its silent operation — one of the preferred mounting locations for long-range lidar is in the cabin above the rear-view mirror. Vehicle designers work hard to reduce all exterior sounds from their cabins, so adding a noisy mechanically scanning sensor inside the cabin is problematic.

As lidar technology trends toward chip-scale solutions, advanced FMCW optical phased array lidar systems are eliminating external optics entirely. Light is emitted and detected directly from the chip with no moving parts or lenses. This results in extremely small form factors that automotive designers can discreetly position around the vehicle, maximizing performance while minimizing the impact on the vehicle design.

Analog Photonics, a company founded by ex-MIT professor Michael Watts, recently demonstrated the first all-solid-state, long-range FMCW lidar utilizing OPAs for beam steering. Phased arrays are a series of antennas that enable true solid-state beam steering through tuning the individual phase of each antenna. They were originally developed for radar and are currently utilized in most automotive radars. While the physics are similar, there were significant challenges regarding translating the technology from radar frequencies to optical frequencies. Analog Photonics has been working since 2016 to increase the scanning range and efficiency of the OPAs, while optimizing designs specifically for their tier 1 automotive partners.

One of the most unique attributes of Analog Photonics’ FMCW OPA lidar is that it requires no external optics at all. Light is emitted and detected directly from the chip with no moving parts or lenses.

This results in extremely small form factors that auto design studios can discretely position around the exterior of the vehicle, maximizing performance while minimizing the impact on the vehicle design.

PICs reduce size and cost

Several companies have now begun to design solid-state FMCW OPA lidar systems for the silicon photonics platform.

The Optical Fiber Communication Conference 2023 in San Diego included a panel session, titled “Lidar Systems and Technologies with Integrated Photonics,” that featured speakers from Analog Photonics, Valeo, Aurora, Samsung, and imec — all of whom discussed the advancements in chip-scale lidar for autonomous vehicles.

Christoph Parl from Valeo, a French tier 1 automotive supplier who has sold the most automotive lidar systems to date, argued that while Valeo’s current systems are based on ToF technology, FMCW is the future due to its comparative performance, size, and cost advantages. Christopher Poulton, vice president of Silicon Photonics at Analog Photonics, presented results from the company’s latest FMCW OPA lidar units, which demonstrated a wide field of view and long-range performance in an all-solid-state package. While technical approaches varied between the different panelists, they all agreed on the fundamental notion that for lidar technology to reach the scale that automotive applications require, then the miniaturization driven by chip-scale photonic integration is a mission-critical goal.

Miniaturization is critical in several ways. Not only does it help address the studio’s concerns regarding the look and feel of the vehicle, but it also helps address cost concerns. Simply put, smaller devices with fewer parts cost less, especially when all the key parts can be manufactured, tested, and burned in on a silicon wafer, at scale, in existing semiconductor foundries.

Sabbir Rangwala, an autonomous vehicle and lidar industry expert, wrote the cover story for the April 2023 edition of ADAS & Autonomous Vehicle International magazine, in which he argues that “the race to miniaturize automotive lidar is on¹.” Rangwala explains in his article that automotive radars and cameras are between 10 and 20× smaller than they were three decades ago and that the size shrinkage was accompanied by cost shrinkage. Clearly lidar is following this path. Rangwala describes some of the different approaches that companies are taking to “move closer to the holy grail of chip-scale automotive lidar.” He said, “eliminating moving parts in the lidar is a priority, accompanied by a significant degree of miniaturization which is critical for integration and styling in automotive applications.”

The silicon photonics connection

While the vision for small, solid-state, chip-scale lidar sensors is clear and compelling, the execution of this vision can be challenging and costly. Laser sources, optical amplifiers, waveguides, modulators, splitters, and OPAs must all be either designed directly onto or tightly integrated into the silicon photonics platform.

Each PIC design iteration or spin takes between 6 and 9 months and costs over $1 million. Unlike application-specific integrated circuits (ASICs) in which design tools and simulations are quite mature, PICs often take multiple iterations to optimize. This is especially the case when completely new devices, such as OPAs, are designed and fabricated.

For example, Analog Photonics, with support from DARPA’s Modular Optical Aperture Building Blocks program and two automotive tier 1 partners, has been developing and optimizing all-solid-state, FMCW, OPA lidar since 2016.

Continuous improvements in integration and performance have already reduced the package size by two orders of magnitude while improving performance by 10×. The next step is to complete the integration of all photonic components onto a PIC and add a digital signal processor ASIC to achieve final automotive-compliant performance and form factor. The technology is projected to be released in early 2025.

Beyond automotive

Due to the availability of first-generation lidar sensors, many applications outside of the automotive industry are finding value in 3D and 4D sensing information. These end uses, in many cases, are deploying the technology faster than the automotive industry since they have less stringent environmental specifications and qualification cycles. Examples include industrial robotics, autonomous forklifts, traffic monitoring, perimeter security, and airport monitoring.

While these applications would all benefit from small, low-cost, solid-state sensing, none of them are large enough to merit the large upfront investment required to accomplish chip-scale solutions. Nevertheless, they and many other applications stand to reap the benefits of the automotive industry’s drive to miniaturize and productionize chip-scale lidar technology.

Lidar has the potential to hugely affect the world by helping to prevent millions of automotive accidents. To fully realize this potential, however, the technology must meet all the critical automotive performance and reliability requirements while being inexpensive enough to deploy en masse. All-solid-state, FMCW lidar powered by OPAs and photonic integration is key to making that happen.

Meet the authors

Greg Smolka is vice president of business development at Analog Photonics, where he focuses on building partnerships in the auton- omous vehicle sector and other 3D sensing markets.

Michael R. Watts is CEO and founder of Analog Photonics. He was previously an associate professor of electrical engineering and computer science at the Massachusetts Institute of Technology and the CTO of AIM Photonics.

References

1. S. Rangwala. (2023). To reduce cost and ease integration, the race to miniaturize automotive lidar is on. ADAS & Autonomous Vehicle, https://www.ukimediaevents.com/publication/9cb6eeeb/36.