Digital Einstein

Where art meets high-tech

Einstein’s digital twin is every bit as communicative as he was. The 3D-animated character answers a wide variety of questions and will even throw the occasional counterquestion or witty remark at the person he is chatting with. A conscious decision was made for the digital figure’s 3D appearance to portray a young Einstein, in keeping with his time in Zurich. The “Digital Einstein” with his ability to see, hear and interact with the person in front of him is driven by a complex algorithmic system developed by external page ETH spin-off Animatico. Key technologies deployed here include dynamic rendering of the individual’s expressions and body language, natural language processing and intention recognition.

Both the screen and the comfortable armchair that constitute the visible parts of the platform are crammed with technology (see Box 1). A camera is used to scan people’s movements and reactions, and whenever someone addresses Einstein, microphones filter their voice out of the ambient noise. Special language-processing software then converts their words into text, analyses them, and generates the most appropriate answer – and all in real time.

The responses are drawn from a predefined dialogue structure, covering topics such as Einstein’s famous theories, his time as a student, and his friendships. Swiss comedian Karpi provided Einstein’s voice and devised the dialogue content. To ensure that the voice, facial expressions and gestures are perfectly synchronised, Animatico used a special recording technique. An algorithm guarantees a degree of randomness in the answers, which means the conversation is unpredictable and can take new turns.
 

Pioneering research on digital avatars

A completely free conversation, however, is still a distant prospect. Which is why researchers at Markus Gross’ Computer Graphics Laboratory are keen to further develop the first digital Einstein prototype created in the summer of 2021. Over the next few years, projects will be launched, for example as Master’s and doctoral theses, to address unresolved research questions.

Advances in artificial intelligence and computer graphics have spurred the development of avatars and chatbots worldwide. Einstein and other digital characters offer an opportunity to make interaction between humans and machines more intuitive and more accessible to people who are less tech-savvy or who have disabilities. The potential applications of such 3D technology are many. Digital humans might be deployed at ticket counters and hotel check-ins; they could also provide product advice and act as virtual coaches in healthcare settings.