Challenges still exist in the development of wearable devices, but continued advances may ultimately change how people interact with the world around them.
It’s a case of a sight for sore eyes and other body parts, with display advances promising better augmented reality and wearable devices. For augmented reality, which overlays computer-generated information onto what people see, improvements include 3-D presentations of data and easier outdoor use. Wearable devices in general will benefit from flexible electronics and other advances.
To do that, though, three challenges must be overcome. System cost is one, particularly for consumer devices. Materials are a second hurdle, as many applications demand flexibility. A third issue is how to put these displays to practical use.
The laser scanning projection system enables augmented reality wearable glasses. Photo courtesy of MicroVision.
“Often overlooked is the whole human-machine interface. That’s the biggest problem,” said Bharath Rajagopalan, general manager for business development and marketing at projection display maker MicroVision Inc. of Redmond, Wash. The company’s laser beam scanning technology can be incorporated into augmented reality products, including head-up and retinal scanning displays.
“We’ve been ready for a long time,”
Rajagopalan said of the company’s decade-old augmented reality technology. “We have good, sharp text. It’s very crisp, very readable, very high contrast.”
Originally developed for the military, MicroVision’s lightweight head-worn displays deliver images that appear to the user as though on a see-through computer screen – thus a technician can view instructions on jet engine repair while performing the task.
Originally developed for the military, augmented reality technology allows information to be displayed and shared outdoors. Photo courtesy of Applied Research Associates.
What’s needed in addition to a quality display is infrastructure, according to Rajagopalan, which includes access to data, sufficient computing power and the right software.
The payoff for getting the display and wearable package right has the potential to be big. Juniper Research Ltd., a Hampshire, U.K.-based analysis firm, predicted the wearable market will grow from its present value of a few hundred million to $80 billion by 2020. The firm expects over 3 million augmented reality smart glasses to be in business use by 2018; consumers will be slower to adopt the technology.
Steffen Sorrell, senior analyst at Juniper, noted that complex wearable augmented reality applications need a large field of view with high pixel density, and that the display must respond with an update latency of less than 20 milliseconds. For augmented reality, compromises in display performance or bulk are not acceptable, according to Sorrell.
A laser scanning projection system by MicroVision. Photo courtesy of MicroVision.
“Latency is currently the greatest issue and requires desktop-class processing to come anywhere close,” he said.
That should be less of an issue in the future. Given time, increases in computing power should enable sufficiently fast display updates.
An investigational advance by professors Hong Hua of the University of Arizona and Bahram Javidi of the University of Connecticut could make augmented reality less of a pain – literally. In a 2014 Optics Express paper (doi: 10.1364/OE.22.013484), they described and demonstrated a technique that superimposes a 3-D image, rather than the standard 2-D one, to create an augmented view of the real world.
In their display, the researchers used freeform optics and integral imaging. “This is performed by capturing multiple images of an object from different perspectives, known as elemental images, and using these images to optically construct a 3-D image,” Javidi said.
With a 3-D presentation of information, eyes will not have to switch between looking at the 3-D world and a 2-D image. What’s more, the 3-D image is created without using stereo depth perception. All of these features should reduce user eyestrain.
Applied Research Associates Inc. of Albuquerque, N.M., is working to transform an augmented reality technology originally developed for the military into something suitable for industrial and consumer applications. The company’s approach can be used on the move, even outdoors, because it does not depend upon markers to determine location.
“We’re just using the sensing capabilities that we’ve built into our helmet kit that allow us to track the user’s position, their movements and their look direction so that we’re able to correctly display georeferenced information,” said Jenn Carter, senior scientist.
Possible industrial applications could include those of benefit to field technicians working in the oil and gas sector, as well as surveyors, she added. There also could be consumer uses, since the system enables collaboration. The military is interested in that capability because it allows soldiers on the move to see, in a virtual sense, where everyone is and what they’re doing. In social settings, it could allow friends to interact by sending videos of what they’re seeing, for one example.
For any consumer application, cost is important, so Applied Research Associates is investigating less expensive yet still capable sensors, Carter said. She also noted that packaging in the right form factor is critical for commercial applications. While a soldier or business user may tolerate looking like a cyborg, consumers likely won’t.
That aesthetic requirement presents its own set of challenges, such as the need to incorporate display into a fashion accessory. For some form factors – a watch, for example – solutions to this issue may be easier due to innovations in protective cover glass.
Some of those developments come from Rubicon Technology Inc. of Bensenville, Ill. The company makes monocrystalline sapphire products, which are used as LED substrates and elsewhere. Sapphire is more expensive than glass, but its greater scratch resistance, strength and durability allow products to be thinner, which can be a plus in wearables, said Hap Hewes, senior vice president of optical markets.
Monocrystalline sapphire or a coating of it could be used as cover glass for watches and other wearables that require displays be rugged, inexpensive and perhaps flexible. Photo courtesy of Rubicon Technology.
Rubicon is aware that the cost of sapphire is sometimes an issue, and the company has developed a solution. “We have a sapphire coating that you can put onto glass, either curved or flat, that can give you many of the properties of sapphire from a durability standpoint,” Hewes said.
The coating offers scratch resistance and also allows for the glass to bend to a degree.
An innovation in the research stage embeds sensors in the cover glass. Jerome Lapointe, graduate student at École Polytechnique de Montréal, was lead author of a 2014 Optics Express paper (doi: 10.1364/OE.22.015473) describing the technique.
The researchers, under the direction of professor Raman Kashyap, used a femtosecond laser to carve 3-D waveguides just below the surface of Corning Gorilla Glass, which covers smartphones. Because the waveguide’s optical properties change as the surrounding environment changes, they can be used as sensors, which researchers demonstrated by building a thermometer.
Using a laser, researchers created waveguides and sensors inside the cover glass of mobile devices, enabling photonics-based sensors that can save space and power in wearable devices. Photo courtesy of Jerome Lapointe, École Polytechnique de Montréal.
One use of a device like this is straightforward. “A mother could check if her daughter had a fever by putting the glass on her forehead,” Lapointe said.
Other uses could include authentication, chemical sensing and device size reduction. With regard to the latter, many mobile and wearable devices already contain sensors that, for instance, measure temperature. Relocating sensors to the cover glass frees up space within the device. What’s more, switching from an electronic to a photonic sensor may save power.
Consequently, the technique could in many ways increase the capabilities of mobile and wearable devices. There is still considerable work ahead before it shows up in products, but the research has attracted significant commercial interest, Lapointe said.
Also attracting interest are flexible displays. At the CES 2015 consumer electronics show, LG Display Co. Ltd. of Seoul, South Korea, demonstrated a screen with an edge radius of curvature about that of a ballpoint pen. Ensuring adhesion meant overcoming the tendency for the plastic OLED structure to pop up and thus flatten. LG Display’s model did so even in conditions as hot as 80 °C and as humid as 80 percent.
Flexible organic thin-film transistors enable printing of test structures on paper and could be used to create flexible, wearable displays. Photo courtesy of Takeo Minari, National Institute for Materials Science.
Such a curved display is the first step toward a flexible one, said Woo Keun Lee, senior consultant in the business strategy department of the LG Economic Research Institute, a think tank in Seoul. But moving past this first step demands what amounts to a materials breakthrough.
“The next step, a rollable and foldable display, can cause problems like delamination and cracks with current materials and processing. Therefore, it would be hard to find an ultimate solution if it is not an innovative material,” Lee said.
Also under way is work on the problem of flexible electronics, which must bend and move while still maintaining individual pixel control. Researchers at Japan’s National Institute for Materials Science and Okayama University developed a nanoparticle printing method that can be used at room temperature, which is important since many flexible and low-cost display materials cannot withstand high heat during processing.
Takeo Minari, a member of the research team, said that since demonstrating feasibility in 2014, investigators have been working to reduce the width of spaces and lines in printed devices below 10 µm to make the thin-film transistors (TFTs) used in displays practical.
“We have developed a high-resolution printing method for patterning of gold nanoparticle ink, which allows the width and space of patterned electrodes to be one micron, which are narrow enough for use of the printed TFTs,” Minari reported.
The research is now investigating practical applications. Plans are to commercialize flexible liquid crystal displays using a printed backplane, currently under development.
As for the uses of augmented reality and wearable displays, one possibility is as a means to interact with the Internet of Things, the growing web of interconnected machines and sensors. There also could be business and social applications.
Selecting the best or most commercially viable use of these displays is challenging and not something the technology itself can help with. As MicroVision’s Rajagopalan said, “It’s really not a technology question.”