Machine vision technology has the potential to greatly improve exoskeletons and prosthetic legs, according to recent work from the University of Waterloo in Ontario, Canada. The technology combines computer vision and deep learning AI to mimic how able-bodied individuals adjust their movements based on visual information of their surroundings. Motorized exoskeletons often require manual controls, which are operated by smartphones or with joysticks. The process, as a whole, can be cognitively demanding, said Brokoslaw Laschowski, a Ph.D. candidate in systems design engineering and leader of the project. “Every time you want to perform a new locomotor activity, you have to stop, take out your smartphone, and select the desired mode," he said. To overcome those difficulties, the researchers fitted exoskeleton users with wearable cameras and are working to optimize AI software to process the video feed so that it can reliably identify stairs, doors, uneven surfaces, and other environmental features. A demonstration of an exoskeleton system equipped with machine vision technology. Courtesy of the University of Waterloo. The next phase of the project, titled ExoNet, will involve sending instructions to the exoskeleton’s motors to climb stairs, avoid obstacles, or take other appropriate actions based on analysis of the user’s current movement and the upcoming terrain. “Our control approach wouldn't necessarily require human thought,” said Laschowski, who is supervised by engineering professor John McPhee, the Canada Research Chair in Biomechatronic System Dynamics. “Similar to autonomous cars that drive themselves, we’re designing autonomous exoskeletons and prosthetic legs that walk for themselves.” The researchers are also working to improve the energy efficiency of the motors for robotic exoskeletons and prostheses by using human motion to self-charge the batteries. The research was published in IEEE Transactions on Medical Robotics and Bionics (www.doi.org/10.1109/TMRB.2021.3058323).