You wouldn’t normally expect to see a robot controlling a marionette in today’s technological world, especially because there’s little to no demand for these robots, and it would be very complex to develop such a system. However, that didn’t prevent ETH Zurich engineers from building a robot puppeteer that can be useful in warehouses or stores. Their results can be found in their technical paper.


While the puppet is moving in real life, the robot simulates the puppets motions and attempts to get the puppet to move as intended.  (Image Credit: ETH Zunich)

“Marionettes are underactuated, high-dimensional, highly non-linear coupled pendulum systems. They are driven by gravity, the tension forces generated by a small number of cables, and the internal forces arising from mechanical articulation constraints. As such, the map between the actions of a puppeteer and the motions performed by the marionette is notoriously unintuitive, and mastering this unique art form takes unfaltering dedication and a great deal of practice. Our goal is to enable autonomous robots to animate marionettes with a level of skill that approaches that of human puppeteers.” ETH Zurich engineers stated.



The movement of the robot actually looks pretty decent, considering how the ABB YuMi robot is missing a few degrees of freedom in its hands.


The ETH Zurich robot isn’t fully capable of carrying out the same movements as a complex marionette just yet. All that’s required of the robot is to know the exact design of the puppet at its target motion, so it can precisely imitate its movements. While the puppet is moving in real life, the robot constantly simulates each of its next motions every second while optimizing in an effort to get the puppet to move as intended.


"Our long term goal is to enable robots to manipulate various types of complex physical systems – clothing, soft parcels in warehouses or stores, flexible sheets and cables in hospitals or on construction sites, plush toys or bedding in our homes, etc – as skillfully as humans do," they wrote in their technical paper. "We believe the technical framework we have set up for robotic puppeteering will also prove useful in beginning to address this very important grand-challenge."


Have a story tip? Message me at: cabe(at)element14(dot)com