Learning in mixed realities

holoLens in use
Photo: afca.ch; View through the Lens: A molecule of the app "Molegram“ in action. Developed by ETH CADD, ETH LET and afca ag.

In mixed realities, users are immersed in a hybrid world that involves both virtual and physical elements. Virtual elements are placed in the view field of the user. These elements are anchored in the real world, making it possible to treat them as ?real? objects. In late 2016 Microsoft HoloLens, the first usable all-in-one device for holographic 3D representation, became available to the public and as a result, mixed reality learning became possible. We at LET wanted to figure out in which educational context this new technology makes sense and facilitates better learning and understanding for students.

For this purpose, we started a project called ?Learning in mixed realities?. LET supported lecturers with technical equipment, support, and networking opportunities to gain insights. 12 HoloLenses were available for use and LET provided connections to developer resources and development partners as well as specialists at Microsoft.

The project ?Learning in mixed realities? was established with the substantial support of the Adrian Weiss Foundation

Realized HoloLens-projects at ETH Zurich:

Enlarged view: hololens_afca
Photo: afca.ch

In this project, the Hololens user literally immerses into the world of molecules. The Molegram project provides a mixed-reality framework to facilitate and broaden our understanding of molecular structure. The first application is to create the surface of a holographic molecular representation. Understanding of where a molecule ends and where its environment begins is as fundamental as the definition of foreign and self in biology. Molegram provides a possibility to not only load a molecule as a three-dimensional structure and project it as a hologram in the room but to actively interact with this holographic molecule. We have realized this concept in tight collaboration with the company afca [external page www.afca.ch].

Enlarged view: molegram_explorer
Image: afca.ch

The Molegram app allows the observer to load and examine differently sized pharmaceutically relevant protein structures. Molegram then adds a new dimension to these molecules, namely a surface. Importantly, the app does not automatically calculate the molecular surface with one click, but the user can interactively create the so-called solvent-accessible-surface by virtually becoming a water molecule and moving as such over the molecular structure. This process provides a literal understanding of the ?ups and downs? of the surface. Within Molegram, the user freely moves around and inside the molecular hologram to examine it from all angles. No longer do we turn and twist the molecule to our whim on a computer screen but we enter its virtual realm as it becomes a holographic entity in our real world. This innovative concept leads to a new way of perceiving molecules and enables new possibilities for chemical structure analysis and design by human-machine interaction.

The Molegram app is freely available for teaching purposes at ETH Zurich.

ETH News article: https://www.ethz.ch/en/news-and-events/eth-news/news/2018/02/hololens.html

Project website: http://www.cadd.ethz.ch/education/hololense.html

Course: 535-0023-00L  Computer-Assisted Drug Design (Practical Course), autumn Semester 2017

Chair: Computer-Assisted Drug Design (cadd.ethz.ch), institute of Pharmaceutical Sciences

Contact: Dr. Jan Hiss, , Prof. Dr. Gisbert Schneider

Enlarged view: geoinfk1

During one semester, Students of the GIS and Geoinformatics Lab explore the possibilities of Microsoft HoloLens – one of the most advanced mixed reality products.

With basic knowledge in software engineering and development, they identified and implemented small projects and mixed reality applications for Geoinformation-related real world tasks that incorporate gamification considerations. Spatially related physical objects are transformed into three-dimensional, interactive, and holographic objects. The students have implemented the following three projects:

  • A spatially connected system such as supply lines in utility management
  • A geographic application such as modeling in landscape planning
  • An spatial-cognitive entertainment application like the board game ?The Crazy Maze?
Enlarged view: geoinfk2

The students were supported and assisted in their project by tutors and had access to relevant literature and technology.

The outcome of these projects can be experienced as immersive applications where virtual geographical features are interacted with and placed on and around real world objects.

Enlarged view: geoinfk1

In particular, the outcome provides us with insights on the execution of technical labs with students that do not have extensive prior knowledge of this specific technology. And based on the results, this teaching approach can be transferred to other faculties such as biology, chemistry, or architecture.

external page Project video

Course: GIS and Geoinformatics Lab (103-0778-00L), autumn semester 2017

Affiliation: Chair of Geoinformation Engineering

Lecturers and Assistants: David Rudi, Fabian G?bel, Christian Sailer, Dominik Bucher 

Contact: Christian Sailer,  

Enlarged view: SplashScreen
Photo: Urs Br?ndle, D-USYS, ETH Zurich

 

On their biodiversity excursions students of environmental science map the distribution of a set of species belonging to an organism group. They prepare with an online tutorial consisting of an identification key, video material and a final test. In the field, however, the initial classification of certain groups of organisms is still difficult because size, appearance and detailed characteristics can differ from the visual material in the tutorials. Thus experts normally have to provide initial assistance.

In the case of lichens with their very stable location on trees, however, the Hololens can take over this function. Our Holucator app recognizes individual trees by their bark structure and highlights lichen colonies whose position has been entered previously in teaching-mode . Users can then interactively select species names in a multiple choice question, tag colonies of the same species or call up detailed information about the lichen species in focus, such as sketches of of charateristic structures. The Hololens-Architecture even allows to use conventional lenses at the same time. A total of 12 trees with 8 different lichen species are available at the Bürkliplatz site, so that several students can practice simultaneously and then begin with the actual species mapping.

The Holucator App was first used in March 2018 by a total of 40 students. The fascination with the new medium was clearly at the forefront of the students' feedback and the actual knowledge gained was not recorded. Nevertheless, we were able to establish that the time the students spent individually observing the lichens was clearly longer compared with the expert-instruction-scenario.

The Holucator (Lichen-Edition) App is the first outdoor application of the HoloLens at the ETH. It can be adapted for other similar applications, provided that the objects to be examined have a permanent location and a sufficient amount of fixed objects nearby to allow the HoloLens to orient itself in space.

Enlarged view: SplashScreen
Image: afca.ch

The app in use (uncut):
 

Contact

Thomas Korner
Lecturer at the Department of Mechanical and Process Engineering

Abt. für Lehre und Lernen
Haldenbachstr. 44
8092 Zürich
Switzerland

JavaScript has been disabled in your browser