Education 2.0: How Virtual Reality Can Transform the Way We Learn

Enabling virtual reality experiences — then and now

Travelling to different places, learning to operate various tools, studying complex systems, such as brain areas and functions, envisioning new types of art pieces, or simply playing immersive video games – all of this is possible in virtual reality (VR). VR is no longer science fiction since VR headsets have become less expensive and widely available. VR is a computer-generated environment created through images and sounds that allows users to be fully immersed in their surroundings. Although this technology seems to be futuristic, its origins are not as recent as they might appear. In 1962 American filmmaker Morton Heilig offered people the experience of multi-sensory theatre. He built a mechanical device called the Sensorama, which included a stereoscopic color display, stereo-sound system, odor emitters, and even a motion chair. The users entered a cabinet where they had to sit on the imaginary motorcycle while simultaneously looking at the streets of Brooklyn on the display, hearing street noise and experiencing fan-generated wind and smell of the city, such as pizza snack bars and petrol fumes. This is quite an impressive set-up if you consider that this was built 60 years ago. The Sensorama can thus be considered the precursor to VR technology.

The term itself ‘virtual reality’ was coined by Jaron Lanier in the mid-1980s. Lanier founded VPL Research which developed the gear, such as goggles and gloves, needed for an immersive experience. Another attempt to create a VR headset happened in 1991 by Sega VR, however the headset was never released, which is unlike Forte VFX1 (1995) and Glasstron (1996) that became the first consumer-level VR headsets. Both were helmets with displays that allowed users to experience a new visual perspective of playing video games and seeing the battlefield from inside the cockpit of the craft. In 2016, Oculus Rift was released and became a game-changer. It started to be used not only in gaming but also in media, industry, education, television, sports, and even some casinos where online players, equipped with a VR headset, could feel as if they were inside the lobby of a casino. Due to the increasing popularity of VR headsets, companies have also started to enable VR experiences for smartphones. VR content can be viewed from the screen of a mobile device through the lenses acting as a stereoscope. One of the popular kits for VR on smartphones is Google Cardboard. Samsung Electronics and LG Electronics also designed their own dedicated headsets as well as other Asian hardware manufacturers, such as Xion, Kolke, and Tencent. Modern VR headset displays are based on gyroscopes and motion sensors for tracking the head, body, and hand positions, small HD screens, and lightweight and fast computer processors.

Limitless opportunities to apply VR

While VR application areas vary from video games to education, some interesting and less-known areas are, for example, prescription-use VR systems. In 2021 FDA approved RelievRX, applied VR, designed to help adults to deal with chronic back pain. This treatment program consists of 56 VR sessions that last from 2 to 16 minutes for 8 weeks. Another interesting VR application is in courtrooms. Specifically, VR is used to visualize crime scenes and re-enact the sequence of events for providing people with better understanding of the details of a crime. This in turn helps to make better decisions about someone’s innocence or guilt. Finally, VR is even used to visualize complex and abstract data. For example, now you can not only read about the changes in the stock market but also experience them in a 3D environment.

VR tries to mimic the real world, but it can create new experiences altogether. Only a few select have the opportunity to travel to space or reach the summit of the highest mountains which are the experiences said to involve transformative feelings, such as awe. People who have travelled to space in VR report goosebumps, which suggests that VR can indeed facilitate new experiences. Some researchers compare VR with psychedelics and claim that they both might work in an analogous way by altering sensory experience and evoking awe. Both VR and psychedelics are used in therapies to treat a variety of mood disorders by altering mental perspectives and disrupting rigid thinking patterns. VR could also be a tool that prepares people for clinical trials with psychedelics by showing sensory distortions that are experienced in psychedelic states.

For trully full immersion VR still needs improvements

VR headsets are based on human perception and they try to mimic our field of view even though it is not such a trivial task. First of all, humans can see around 220 degrees of surrounding view, whereas a VR headset is still limited to 180 degrees. Another visual element that limits VR experiences is the frame rate since neuroscientists believe that the human eye can handle up to 1,000 frames per second. This is an enormous amount in comparison to the 120 frames per second that is currently available on VR headsets. The developers of VR also found out that anything less than 60 frames per second causes disorientation and nausea.

A fully immersive experience requires audio inputs in addition to visuals since this is what we experience in the real world. The better the audio quality, the more immersed users will feel. The audio cues also convey information about which direction one should turn to move in different environments, especially when sound is coming from behind the person or above them. However, producing realistic sounds is a challenging task because VR must use spatialized audio input to enable users to experience sounds in a 3D space. One of the problems for this is sound occlusion. In simple terms, sounds should be heard differently depending on the environment. Sound waves can travel through or around any barrier, and as such, the wave intensity perceived by users as the volume will be different depending on whether something blocks the sound wave or not. The volume will also depend on the object material that blocks the sound wave because the wave will bounce off of it differently.

A trully immersive experience necessarily involves touch sensation and movements that together are called haptic modality. There are multiple attempts to create a haptic experience for VR that would allow users to touch and feel virtual objects. However, so far no technology can even approximately match the capabilities of the human sense of touch due to the extraordinary spatio-temporal tactile sensitivity of the human skin.

The workings of augmented reality

VR is often confused with another interactive experience called augmented reality (AR) where the main difference between the two is that AR combines the real world with computer-generated content. AR can include various modalities, such as vision and audio, but also haptic or olfactory (smell) senses in addition to the real environment. Some well-known examples of AR consist of a shopping experience where users can ‘add’ any artificially-generated piece of furniture to their living room, as well as applying filters offered by companies like Snapchat or Instagram. If you ever played one of the most popular mobile games PokemonGo, then you have already experienced AR in action. AR is often used in the entertainment industry, including various mobile games, but also in concerts and performances; for example, recently Gorillaz, a virtual band, invited fans to their live-immersive performance with the help of AR technology.

The basic idea of AR technology is to superimpose graphics, audio, or other sensory information over a real-world environment. The basic components are a camera, a small projector, a smartphone, and a mirror — this is what the first AR project called SixthSense used for creating one of the first AR systems. Basically, the device captures your surroundings with the camera and other sensors, and then the software processes this data to decide what the best place to position an AR objects is, for example, using object detection and tracking. Then using computer graphics the object is generated and displayed for the user. Besides sensors, devices, and algorithms, AR often uses Simultaneous Localization and Mapping technology that allows digital content to be accurately placed in the real world by combining sensory and computer vision inputs as well as the movements of the user to produce a map of the environment.

VR’s potential for social experiences

VR progress has been quite rapid and also more and more VR applications tend to be linked to social situations that include multiple users. Social VR is a simulated world that serves as a 3D VR system and social app together where participants can socialize, build worlds and play games. One of the best-known examples is Second Life introduced in 2003 as the first virtual world. In order for people to interact, computer-generated virtual avatars are used to represent human users. Although many realistic avatars have been created and used in the gaming industry, VR applications still lack the realism of the employed avatars. With the growing interest in the metaverse, there is also a higher demand for users to be realistically represented in the digital world. One of the solutions is to use a Generative Adversarial Network (GAN) to model human faces that consists of generative algorithms with deep learning methods, such as convolutional neural networks. Although GANs received a lot of criticism for deep fake videos portraying politicians and celebrities, such technology permits creating photorealistic representations of people, scenes, and objects at a minimal cost and can enable progress of various industries, including but not limited to VR. GANs have also been used in combination with 3D scanning to create virtual agents (the difference between virtual agents and avatars being that the former are autonomous, whereas the latter are controlled by users). Many technology companies, such as Adobe, Nvidia, and Unity have introduced new products using GANs to significantly reduce the time and cost of creating photorealistic characters where users can generate their own digital versions through mobile phones or webcams.

VR technology has come a long way since the creation of the Sensorama. The progress of VR has resulted in it being applied to a range of domains, from the treatment of chronic back pain to visualizations of crime scenes in courtrooms. Despite the many advances in VR technology, it remains an emerging field with limitless potential for further development and application.

Authors: Julija Vaitonyte & Judita Rudokaite