Computer scientists at ETH Zürich have developed a system that could transform the way animated films are made. The research team, which about a year ago showed that highly technical film scenes could be shot more easily using drones, has now demonstrated that drones also have great potential for animated film. “It’s a very time-consuming task to make figures look realistic in an animated film,” said researcher Tobias Nägeli. “For the figures to appear natural, the first step is to film an actor performing the movements. The second step is then to build the animated figure around this.” Nägeli and his colleagues at ETH Zürich and Delft University of Technology developed a system that, in its simplest configuration, consists of two commercially available drones and a laptop. The drones follow the actor’s every move and automatically adjust their position so that the target can always be shot from two angles. This reduces the amount camerawork required, since the cameras only have to be in the spots where they are actually needed. The system anticipates the actor’s movements in real time and then calculates where the drones need to fly to keep the actor in the frame. Actors’ movements can be easily recorded using commercial drones, which greatly reduces the technical effort required for animated film. Courtesy of ETH Zürich/Tobias Nägeli. To minimize the volume of data generated, IR diode markers are fixed to the actor’s joints. The drones, which are equipped with a true light filter, record only the light from the markers, simplifying data processing. The system sees only a few points, from which it then determines the body’s position and directional movement. It can capture sudden and fast movements reliably, the researchers said. The researchers believe that the system could be extended with additional drones to capture movement in even greater detail. They also think that the current approach using light markers could be replaced by automatic image analysis, which would further reduce the technical complexity of film production. The team tested the system’s ability to track human movement over longer distances — a feature that could make it useful for sports motion analysis. “Until now, it has been impossible to perform a comprehensive motion analysis on runners, for example, because it is much too complicated,” Nägeli said. “With our system, it’s very easy now to examine how a runner’s kinetics change over a period of time.” While the system does not yet meet the rigorous requirements of the film industry, the researchers say that it offers a promising approach. Their next challenge will be to continue to develop the system for practical applications. Together with two colleagues, Nägeli plans to tackle this task at a new startup company, Tinamu Labs. The research was presented at Siggraph Asia 2018 Tokyo, Dec. 4-7, 2018 (https:/doi.org/10.1145/3272127.3275022). To read about the team’s work on use of drones to film action scenes, see “Spectacular images thanks to an efficient algorithm.” The team conducted various tests to show how the system can be used to track human movement over longer distances. Courtesy of ETH Zürich/Tobias Nägeli.