Learning in mixed realities

In this project, the Hololens user literally immerses into the world of molecules. The Molegram project provides a mixed-reality framework to facilitate and broaden our understanding of molecular structure. The first application is to create the surface of a holographic molecular representation. Understanding of where a molecule ends and where its environment begins is as fundamental as the definition of foreign and self in biology. Molegram provides a possibility to not only load a molecule as a three-dimensional structure and project it as a hologram in the room but to actively interact with this holographic molecule. We have realized this concept in tight collaboration with the company afca [external page www.afca.ch].

During one semester, Students of the GIS and Geoinformatics Lab explore the possibilities of Microsoft HoloLens – one of the most advanced mixed reality products.

With basic knowledge in software engineering and development, they identified and implemented small projects and mixed reality applications for Geoinformation-related real world tasks that incorporate gamification considerations. Spatially related physical objects are transformed into three-dimensional, interactive, and holographic objects. The students have implemented the following three projects:

• A spatially connected system such as supply lines in utility management
• A geographic application such as modeling in landscape planning
• An spatial-cognitive entertainment application like the board game «The Crazy Maze»

On their biodiversity excursions students of environmental science map the distribution of a set of species belonging to an organism group. They prepare with an online tutorial consisting of an identification key, video material and a final test. In the field, however, the initial classification of certain groups of organisms is still difficult because size, appearance and detailed characteristics can differ from the visual material in the tutorials. Thus experts normally have to provide initial assistance.

In the case of lichens with their very stable location on trees, however, the Hololens can take over this function. Our Holucator app recognizes individual trees by their bark structure and highlights lichen colonies whose position has been entered previously in teaching-mode . Users can then interactively select species names in a multiple choice question, tag colonies of the same species or call up detailed information about the lichen species in focus, such as sketches of of charateristic structures. The Hololens-Architecture even allows to use conventional lenses at the same time. A total of 12 trees with 8 different lichen species are available at the Bürkliplatz site, so that several students can practice simultaneously and then begin with the actual species mapping.

The Holucator App was first used in March 2018 by a total of 40 students. The fascination with the new medium was clearly at the forefront of the students' feedback and the actual knowledge gained was not recorded. Nevertheless, we were able to establish that the time the students spent individually observing the lichens was clearly longer compared with the expert-instruction-scenario.

The Holucator (Lichen-Edition) App is the first outdoor application of the HoloLens at the ETH. It can be adapted for other similar applications, provided that the objects to be examined have a permanent location and a sufficient amount of fixed objects nearby to allow the HoloLens to orient itself in space.