HUD Systems: Augmented Reality Is Coming to Your Windshield

MICHAEL WHEELER, MANAGING EDITOR, michael.wheeler@photonics.com

Only a few short years ago, the use of displays in automobiles was considered a luxury spec reserved for high-end vehicles. Today, however, rapid developments in automotive infotainment and navigation systems have made displays a centerpiece in most automotive designs.

Lane departure warnings appear when the driver goes off course. Photo courtesy of Continental AG.

Beyond the large monitors used for in-console navigation, controls and rear-seat DVD systems, the most exciting advances are in head-up display (HUD) systems, a classification extending to any transparent display that presents data without requiring the user to look away from the usual field of view.

HUD systems superimpose vehicle information such as speed, warning signals and indicator arrows on the windshield, allowing drivers to maintain their focus on the road while receiving information that would normally require a glance at an instrument panel on the dashboard.

Head-up displays like this one from WayRay SA project the virtual route onto the windshield so that it seems to lie in front of the car, enabling drivers to keep their eyes safely on the road. Photo courtesy of WayRay.

As futuristic as it sounds, HUD technology has been around for quite some time. The first HUD systems were developed for pilots in World War II, who required information be displayed in a “head up,” forward-looking position to help them hit their targets while maneuvering. Modern HUDs became commonplace in military aircraft in the 1960s and in commercial aircraft in the 1970s.

Older automotive-based HUD systems used a small cathode ray tube to project vehicle data onto a special element on the windshield. Information is superimposed on the lower half of the glass, which is inside the driver’s immediate line of sight. To allow for faster comprehension of information, data is displayed at a focal point equal to that of the roadway in front of the vehicle. Because of this, the driver doesn’t have to refocus his or her eyes to view HUD information.

Recent advances in display technology have improved the design of HUD systems, with many now utilizing liquid crystal displays (LCD) and light-emitting diode (LED) technology, which convey brighter images and are less expensive to manufacture. Newer projection technologies, including micromirror-based devices based on electromechanical systems, are coming into the market, creating brighter images and more saturated colors.

HUD market driven by system size, price and value

General Motors Co. was the first automaker to implement a HUD system, in 1988 on its Oldsmobile Cutlass Supreme model. Recently other car manufacturers have followed, primarily offering HUD systems on select sport or luxury-class models from General Motors, BMW, Mercedes, Peugeot, Mazda and others. HUD systems can be found on a broader number of mid-range and compact vehicles, including the Passat and Tiguan from Volkswagen, the Mini Cooper and Mazda 3.

BMW is among a growing list of automakers offering HUD systems. An up-close depiction of the HUD on the BMW Series 7, showing speed and navigational information. Photo courtesy of BMW.

As of 2015, total sales were estimated at 2.26 million unit sales, with 5.6 million units projected for 2021, according to the global market research firm IHS.

“The physical size of the technologies and components in HUDs — both wind-screen and combiner types — are declining, making it physically easier to engineer them into smaller vehicles,” said Mark Boyadjis, an analyst with IHS.

First-generation HUD systems were commonly monochrome and offered little information to the user, showing only RPM and speed. They were also large and couldn’t fit into vehicles with smaller cockpit sizes.

“This is changing and combiner HUDs are enabling more growth as they require much less space on the dash. Along with shrinking sizes, declining costs are enabling automotive product planners to justify offering HUD for the first time ever,” Boyadjis said.

Combiner HUD systems: the choice for mid-market cars and beyond

There are primarily two types of HUD systems in production or in development: those that take advantage of the full windshield and the combiner HUD.

Peugeot was the first automaker to offer a combiner HUD in 2009 from global automotive supplier Visteon Corp. Compared with a full windshield-HUD, the combiner HUD requires about half of the installation space. A small transparent disc called the “combiner” acts as the mirror for information. As the entire windshield doesn’t need to be incorporated into the optical path, less space is required and combiner HUDs can therefore easily fit in sports cars, compacts or in whole vehicle families.

A large-image windshield HUD concept from Visteon Corp. Photo courtesy of Visteon Corp.

A smaller footprint saves money, as designing custom windshields with exact specifications is costly for the automaker. Instead, the optical path is positioned in the combiner, which is optimized for its optical properties.

Combiner HUDs offer the same ergonomic advantages as full-windshield HUDs; drivers can keep their eyes on traffic and no longer have to look back and forth between the road and their display instruments.

Visteon Corp. is one tier supplier that has adopted an aggressive approach to the combiner HUD market, unveiling multiple custom systems for entry-level to luxury-class vehicles at this January’s Consumer Electronics Show (CES). Its combiner HUDs for emerging markets take a no-frills approach, offering a cost-effective system with a compact field of view and limited color definition. For the mid-market and luxury segments, systems feature kinematics (enabling the combiner disc to be stored inside the instrument panel when not activated), higher optical quality, increased brightness and enhanced craftsmanship.

The company plans to offer its latest combiner HUD systems on automakers in the Asian and European markets during the 2016 and 2017 model years.

Full windshield HUDs

Despite the advance of the more affordable combiner HUD option, tier suppliers continue to invest heavily in developing full-windshield HUD systems, which promise to fundamentally transform the driving experience and usher in new advanced features like augmented reality.

“My thesis is once you’ve driven a car with a HUD you can never go back,” said Paul Russo, chairman and founder of GEO Semiconductor Inc., whose eWARP processor enables real-time spatial morphing of images, calibration and digital alignment for HUD systems.

“It reminds me of my days at RCA when we were developing remote controls for TVs. People would ask, ‘Why would you want this when you can just go and stand up and change the channel?’” he said.

Russo and others tout the promise of the full windshield HUDs as a virtual blank canvas, a portion of which can be used to convey information to the driver, from pedestrian detection, intersection guidance and a wide range of warning messages. But with greater options, come greater technical hurdles.

“The problem today is not only do you need precision optics, curved optics, but you must also do a lot of manual calibration in the factory,” Russo said.

New video processing technologies, such as GEO Semiconductor’s eWARP technology, should enable greatly reduced manufacturing costs and make HUDs affordable for the mainstream auto market, he said.

Converting a windshield into a medium for depicting information requires close attention to detail, and, in order to create a distortion-free image, the windshield requires very exact images from the concave mirror in the HUD. Tolerances in manufacturing the large mirror are therefore correspondingly quite narrow.

International automotive supplier Continental AG proposes a windshield with a tolerance of less than 5 micrometers across the entire surface. In order to present an image that can easily be read, a large dynamic range of ambient brightness, extreme luminance and brightness control are required for a HUD system. Any optical distortions through the windshield are electronically corrected on the display.

Transforming the ‘reality’ of the driving experience

The next frontier is the introduction of the first “augmented reality” (AR) HUD systems. An AR HUD supplements the exterior view of traffic conditions in front of the vehicle with virtual information, or augmentation, for the driver. AR HUDs incorporate information as part of the driving experience.

“Augmented reality is — hands-down — the most exciting new advance in HUD innovation,” Boyadjis said. “The biggest opportunity for augmented reality systems is for safety, but imagine a HUD enabling a virtual arrow to be laid upon the road itself as you drive, helping you to navigate more naturally. Other options could be illumination of the road lines or pedestrians at night or in poor visibility conditions.”

Unveiled as a prototype in 2014, Continental’s AR HUD system features a virtual symbol inserted into the exterior view showing the driver exactly where to turn in front of the vehicle. The system also features adaptive cruise control, which tracks the distance between the driver’s vehicle and the vehicle ahead. A crescent-shaped icon changes color when the vehicles get too close.

Adaptive cruise control monitors the speed and distance of cars ahead, offering uninterrupted feedback to the driver in real time. Photo courtesy of Continental AG.

This and other features aim to relieve the burden on the driver, as the driver is connected with explanatory information — which, according to Continental, allows the driver to more quickly register the driving situation he or she is in, making for a safer driving experience.

Augmented reality HUD is “an important part of tomorrow’s holistic interaction” between driver and vehicle, according to Kern.

The company is now in the pre-development phase of augmented reality HUDs, with development set for next year. The earliest market introduction would be in 2019, he said.

DLP and Lasers Emerge

HUD systems typically consist of the picture generating unit (PGU), which includes a heat sink, printed circuit board, LED matrix and TFT color display and light guide.

While most HUDs feature super bright backlit LCDs, that may soon change.

Texas Instrument Inc.’s TI DLP chipsets, used for years in cinema and home theater applications, are being incorporated into some HUD systems in development. DLP, which stands for digital light processing, is a kind of projection system that contains millions of tiny mirrors capable of producing clear images across a windshield regardless of whether it’s sunny, cloudy or nighttime.

Key components of a head-up display include an aspherical mirror, light source with display generation unit and the optical component. Photo courtesy of Continental AG.

Patrick Nebout, director of advanced technology and innovation for Visteon Corp., confirmed his company is working on developing several technologies for PGU, including DLP.

“This is the same technology you see in projectors in movie theaters,” he said. “It’s done using small mirrors, and light is reflected in such a way so you can manipulate each pixel.”

Continental is following suit.

“We expect to be the first to market with DLP HUD technology for automotive use,” said Thorsten A. Kern, director of Continental’s Center of Competences Head-Up Displays based in Babenhausen, Germany. He noted that many of the company’s U.S.-based competitors are advertising future capabilities for DLP.

“DLP offers a much brighter image, more brilliant colors and [a] larger field of view than current LCD HUDs,” he said.

Another alternative to LED matrix and TFT color displays is a laser-based system — one that employs blue and green laser diodes — now available on the Jaguar Land Rover XE sedan.

Despite the promise of projecting sharp, high contrast images, it remains more expensive to manufacture, Mark Boyadjis, an analyst at IHS, said.

Visteon’s Nebout added that, historically, the laser technology hadn’t matured to be viable in automotive applications. But that is changing, he said, and the bigger challenge is overcoming speckle, a random granular pattern that emerges when a highly coherent light beam is diffusely reflected on a surface.