After experiencing incremental gains over the past decade, the augmented and virtual reality (AR/VR) markets are poised for swift advancement as the technologies mature and find new uses. Market research firms predict that from 2017 to 2023, the global compound annual growth rate will be about 40% for AR and nearly 34% for VR. This means a $60 billion market for AR in just four years — up from $4.2 billion in 2017. “Virtual and augmented reality have had their place in industry for more than a decade,” said Marek Polcák, co-founder and CEO of VRgineers Inc., a Prague-based manufacturer of VR headsets. “But before 2010, devices often had limited display resolutions and slow refresh rates.” In June 2018, the company unveiled its latest version called the XTAL (Figure 1). The previous iteration of the device was a finalist for the VR Awards’ 2018 headset of the year. Figure 1. The XTAL VR headset created by VRgineers uses freeform, non-Fresnel, aspherical lenses to provide wide fields of view. Courtesy of VRgineers. Polcák credits improvements in display design, as well as faster graphics processing units, with helping to spur increased interest from the consumer market. “Retail is focused on [headset size] and affordability — it will always be entertainment driven.” The business sector, he added, has seen a continuous but more gradual growth. Several factors have confounded the widespread adoption of the technologies. Many necessary optoelectronic components are just beginning to miniaturize. These components must then come together in a format that will comfortably fit users who have differing facial structures, interpupillary distances, and tolerances for bulky headwear. In addition, AR/VR headsets have caused eyestrain, headaches, dizziness, and nausea. Freeform future Technological developments generally follow Moore’s law; processors get faster and components increase in capacity and capability, while shrinking in size. Entire categories of optoelectronics have emerged under such developments, but smart glasses and headsets for AR/VR have only made occasional and fleeting appearances. This is partly because lenses don’t follow Moore’s law. As a result, researchers and companies have had to devise new solutions to decrease the form factor of conventional head-mounted displays. One solution, according to Jannick Rolland, is to depart from the reliance on traditional lenses and embrace freeform optics instead. Rolland is a professor of optical engineering at the Institute of Optics at the University of Rochester in N.Y. and chief technical officer of LighTopTech, a noninvasive imaging company that she helped start in 2013. “We need to use more complex lens shapes to design the sunglass-like form factor with large fields of view that people want,” she said. Freeform optics are essentially nonsymmetrical lenses that can be shaped, contoured, and stacked to produce a desired functionality. Their surfaces can now be machined to nanometer precision, and they have been used in nonimaging applications such as car headlights since the early 1990s — when the precision was only at the micron level. Rolland has been exploring freeform optics for more than a decade. Because of their atypical shapes, they produce field-dependent aberrations with significantly different patterns than traditional lenses. In May 2018, Rolland’s group published a methodology to design freeform surfaces within an optical system that accounts for aberrations and emphasizes manufacturability1. “We now have a very good handle on the basic theory and method,” she said. “We have also made a lot of progress in fabrication — with many kinds of materials.” VRgineer’s Polcák said about the XTAL VR headset: “After intense sourcing and testing of all existing Fresnel lenses, we developed a unique optical system, based on one freeform, non-Fresnel, aspherical lens.” The headset’s lenses provide a horizontal field of view (FOV) — the arc of what’s observable from a stationary point — of ~180° and a vertical FOV of ~80°, depending on the user. The FOV for humans is about 210° horizontally and 150° vertically. Large FOVs can help mitigate some of the unintended uncomfortable physiological effects that VR users feel, such as dizziness. According to Polcák, with each version of their headsets, the team focused on designing lenses that lowered distortion and artifacts, increased immersion and FOV, and made the headsets more compact. To develop the lenses, VRgineers established a separate joint venture with an Israel-based consortium of optical specialists — VR Optics Inc. — dedicated only to virtual and mixed reality optics. Collaborative efforts “Large challenges in optical design can best be met through collaboration,” said Jose Pozo, director of technology and innovation for the European Photonics Industry Consortium (EPIC). In 2018, EPIC held a meeting on photonics for AR/VR in Stuttgart, Germany, which was hosted by SONY. The purpose, according to Pozo, was to bring together all stakeholders involved in the production of the headsets and encourage collaborations between suppliers, designers, and manufacturers. “This is an emerging market where novel photonic approaches can offer advantages in performance, efficiency, size, and cost reduction,” he said. EPIC also hosted a meeting earlier this year on freeform optics at WZW OPTIC AG in Balgach, Switzerland (Figure 2). Pozo said he saw “both a problem and an opportunity” in the field of freeform optics. “It is difficult to find good, skilled, and experienced optical designers. This meeting was a great opportunity for the community to form strategic partnerships.” Figure 2. EPIC earlier this year hosted a meeting on freeform optics at WZW OPTIC AG. Freeform lenses are transforming the form factor of AR/VR headsets. Courtesy of EPIC. VRgineers’ Polcák — who delivered a recent EPIC webinar on next-generation head-mounted displays — said, “If you need to consult or sort out an optical challenge, there is no better place to be than EPIC.” The promise of freeform optics and the complexity of AR/VR headsets led the University of Rochester’s Rolland to establish the Center for Freeform Optics six years ago. The center, which she spearheaded, is a collaboration between industry and academia in the U.S. and is sponsored by the National Science Foundation. “There’s a lot of innovation going on,” she said, adding that the supported research was precompetitive. This allowed companies to benefit from the institutional knowledge of academic research labs, and allowed academics to benefit from the market-oriented knowledge of industry scientists. Volume manufacturing of freeform lenses for consumers could deliver lightweight head-mounted displays for a variety of applications. Guiding the wave Early this year at the annual Consumer Electronics Show in Las Vegas, the Abingdon, U.K.-based company Wave- Optics Ltd. demonstrated the latest iteration of its diffractive waveguides, which can project images such as text messages, directions, and appointments directly into wearers’ eyes. Waveguides are essential components in AR and mixed reality systems because they direct the path of light from a virtual illumination source or microprojector to the user (Figure 3). Figure 3. WaveOptics’ diffractive waveguides project images for AR directly into wearers’ eyes. Waveguides enable users to see digital images overlaid on top of the real world — delivering crisp, undistorted text and stable imagery. Courtesy of WaveOptics. Within the past six months, Wave-Optics has undertaken a series of partnerships with companies across the world that have allowed it to secure a position in the crowded field of AR/VR component manufacturing. These include partnerships with EV Group (Austria) for nanoimprinting lithography, Corning Inc. (U.S.) for glass supply, Goertek Inc. (China) for volume manufacturing, and Compal and Wistron Corp. (both in Taiwan) for original design manufacturing. WaveOptics co-founder and director of engineering David Grey said, “We work with a broad range of partners across the ecosystem to ensure that we can deliver the high-quality, cost-effective AR solution that our technology enables.” The company went with a diffractive waveguide approach, rather than a reflective one, because of cost and manufacturing concerns. According to Grey, “Waveguides themselves are the most compact components that are capable of displaying AR images while giving a large eye box, which makes them very usable for different head shapes.” The eye box is the area where one’s eyes can scan while the virtual objects remain in focus. Larger eye boxes are often easier on wearers’ eyes. WaveOptics waveguides have been recognized for their 2D pupil expansion. Nanostructures in the component control the diffraction orders to display virtual objects with uniform brightness. Unlike the XTAL or other VR systems that fully encase a wearer’s eyes, AR systems must contend with ambient light interference that can hamper the ability to see the images. Grey is enthusiastic about the uses for the company’s waveguides. “With a compact form factor,” he said, “the applications are unlimited in commercial and consumer markets.” Future visions According to EPIC’s Pozo, “[The AR/VR] markets are being driven by the supplying of customized optics to the larger companies in high-volume compatible manufacturing processes, such as replication. EPIC has identified freeform micro-optics and diffractive optical elements as huge opportunities.” He cites potential growth for AR/VR in the home segment where the headsets would replace or complement smart speaker systems. He also sees virtual sports and tourism benefiting from the technology. “It could be really useful for venues like art museums,” Pozo said. According to VRgineers’ Polcák, his favorite use case is “telepresence, where you combine AR and VR.” He envisions a situation in which a technician could use an AR system such as a Microsoft HoloLens 2, while a specialist at a control center would wear a VR headset such as an XTAL. The AR technician would capture the situation on the ground and create a 3D map, which would be projected to the specialist working remotely in the virtual environment. “It’s a really good example of how AR and VR can work well together,” Polcák said, “supplementing rather than competing with each other.” www.linkedin.com/in/farooqtheahmed Acknowledgments The author would like to thank Marek Polcák, VRgineers; Jannick Rolland, the University of Rochester; Jose Pozo, the European Photonics Industry Consortium; and David Grey, WaveOptics. Reference 1. A. Bauer et al. (2018). Starting geometry creation and design method for freeform optics. Nat Commun, Vol. 9, Article 1756.