This article is part of a series produced in collaboration with the Diplomatic Courier for the ETH Zurich exhibition at the Annual Meeting of the World Economic Forum 2020.
This technology enables you to reimagine reality
by Robert Sumner, 21 January 2020
“...We are approaching a turning point in technology that will allow the characters that I imagined as a child to step outside of my head and into the real world, ready to join me on my next adventure,” says Robert Sumner, Director of the Game Technology Center, ETH Zurich.
The "PuppetPhone" focuses on novel interactions in augmented reality by transforming digital characters into animated puppets that users can manipulate naturally using the physical movement of a smart phone. Researchers in the Game Technology Center embed a digital character into the real world using augmented reality on a smart phone. With the new puppet-interaction metaphor, when the user taps and holds on the screen, the digital character virtually attaches to the phone with an invisible rigid bar. The character directly mimics any physical movement of the phone. However, rather than simply moving like a rag doll, we interpret the user's gestures and enhance the character movement with semantically relevant animation.
Image: ETH Zurich & Disney Research
Thus, the character walks or runs across the table as the phone is moved. By tilting the phone forward, the character crouches, and by moving it upward, the character jumps. We even created a snowy scene where the user can encourage the puppet character to roll large balls of snow and assemble them into a snowman -- all with simple, natural gestures. Of course, the snowman also comes to life to join the user on his or her next adventure.
