Smart Glasses Provide Customized Assistance Based on Worker and Task

An intelligent, assistive audiovisual system is being designed that will provide help to workers in manufacturing and logistics via a visual display with spoken instructions. The system comprises smart glasses with a specialized headset that provides information precisely at the time the information is needed. Bielefeld University and the University of Applied Sciences Bielefeld are working on the project, which is called Avikom, with four regional companies and two trade associations.

The Avikom glasses use augmented reality to display supplementary information directly in the user’s field of vision. The glasses are integrated with a pair of intelligent headphones fitted with a microphone, which allows the Avikom system to “speak” to the user, similar to a navigation system.

To enable the system to adjust to the individual user, the worker’s skills are assessed in advance using software-based diagnostics. The software is designed to make a predictive “diagnosis” of the difficulties the person will likely face when performing different work procedures. Based on this assessment, the system can then give customized tips and instructions.

The precursor model to the Avikom glasses can, for example, help workers assemble wooden birdhouses. Courtesy of CITEC/Bielefeld University.

Avikom could be used for training on the shop floor. Another potential application is remote support. For example, someone without the specialized knowledge to carry out a repair could get assistance from an expert via the smart glasses. The expert could provide audio instructions via the headphones and could display tips in the visual field. In warehouse logistics, the glasses could be used in place of printed work papers to provide a list of materials, and could then help direct the worker through the warehouse to the items that are needed to fulfill the order.

“What makes our assistive system unique is the fact that it doesn’t just give instructions,” professor Thomas Schack said. “It recognizes the person using it and understands the situation at hand by recognizing individual objects and the steps to be performed. Based on this, it then provides assistance.”