Virtual reality (VR) is a simulated experience that employs 3D head-mounted displays and pose tracking to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment (particularly video games), education (such as medical, safety, or military training), research and business (such as virtual meetings).
Currently, standard virtual reality systems use either virtual reality headsets or multi-projected environments to generate some realistic images, sounds, and other sensations that simulate a user's physical presence in a virtual environment. A person using virtual reality equipment is able to look around the artificial world, move around in it, and interact with virtual features or items. The effect is commonly created by VR headsets consisting of a head-mounted display with a small screen in front of the eyes but can also be created through specially designed rooms with multiple large screens. Virtual reality typically incorporates auditory and video feedback but may also allow other types of sensory and force feedback through haptic technology.
VR is one of the key technologies in the reality-virtuality continuum. As such, it is different from other digital visualization solutions, such as augmented virtuality and augmented reality.
Contents
Etymology
"Virtual" has had the meaning of "being something in essence or effect, though not actually or in fact" since the mid-1400s. The term "virtual" has been used in the computer sense of "not physically existing but made to appear by software" since 1959.
In 1938, French avant-garde playwright Antonin Artaud described the illusory nature of characters and objects in the theatre as "la réalité virtuelle" in a collection of essays, Le Théâtre et son double. The English translation of this book, published in 1958 as The Theater and its Double, is the earliest published use of the term "virtual reality". The term "artificial reality", coined by Myron Krueger, has been in use since the 1970s. The term English term "virtual reality" was used in a 1982 novel by Damien Broderick, The Judas Mandala. The first use of the term in the modern sense given in the Oxford English Dictionary dates to 1979, in an announcement from IBM.
Widespread adoption of the term "virtual reality" in the popular media is attributed to Jaron Lanier, who in the late 1980s designed some of the first business-grade virtual reality hardware under his firm VPL Research, and the 1992 film Lawnmower Man, which features use of virtual reality systems.
Forms and methods
One method of realizing virtual reality is through simulation-based virtual reality. For example, driving simulators give the driver the impression of actually driving a vehicle by predicting vehicular motion based on the driver's input and providing corresponding visual, motion, and audio cues.
With avatar image-based virtual reality, people can join the virtual environment in the form of real video as well as an avatar. One can participate in the 3D distributed virtual environment in the form of either a conventional avatar or a real video. Users can select their own type of participation based on the system capability.
In projector-based virtual reality, modeling of the real environment plays a vital role in various virtual reality applications, including robot navigation, construction modeling, and airplane simulation. Image-based virtual reality systems have been gaining popularity in computer graphics and computer vision communities. In generating realistic models, it is essential to accurately register acquired 3D data; usually, a camera is used for modeling small objects at a short distance.
Desktop-based virtual reality involves displaying a 3D virtual world on a regular desktop display without use of any specialized VR positional tracking equipment. Many modern first-person video games can be used as an example, using various triggers, responsive characters, and other such interactive devices to make the user feel as though they are in a virtual world. A common criticism of this form of immersion is that there is no sense of peripheral vision, limiting the user's ability to know what is happening around them.
A head-mounted display (HMD) more fully immerses the user in a virtual world. A virtual reality headset typically includes two small high resolution OLED or LCD monitors which provide separate images for each eye for stereoscopic graphics rendering a 3D virtual world, a binaural audio system, positional and rotational real-time head tracking for six degrees of movement. Options include motion controls with haptic feedback for physically interacting within the virtual world in an intuitive way with little to no abstraction and an omnidirectional treadmill for more freedom of physical movement allowing the user to perform locomotive motion in any direction.
History
The development of perspective in Renaissance European art and the stereoscope invented by Sir Charles Wheatstone were both precursors to virtual reality. The first references to the more modern-day concept of virtual reality came from science fiction.
20th century
Morton Heilig wrote in the 1950s of an "Experience Theatre" that could encompass all the senses in an effective manner, thus drawing the viewer into the onscreen activity. He built a prototype of his vision dubbed the Sensorama in 1962, along with five short films to be displayed in it while engaging multiple senses (sight, sound, smell, and touch). Predating digital computing, the Sensorama was a mechanical device. Heilig also developed what he referred to as the "Telesphere Mask" (patented in 1960). The patent application described the device as "a telescopic television apparatus for individual use... The spectator is given a complete sensation of reality, i.e., moving three-dimensional images that may be in color, with 100% peripheral vision, binaural sound, scents, and air breezes."
In 1968, Harvard Professor Ivan Sutherland, with the help of his students, including Bob Sproull, created what was widely considered to be the first head-mounted display system for use in immersive simulation applications, called The Sword of Damocles. It was primitive both in terms of user interface and visual realism, and the HMD to be worn by the user was so heavy that it had to be suspended from the ceiling, which gave the device a formidable appearance and inspired its name. Technically, the device was an augmented reality device due to optical passthrough. The graphics comprising the virtual environment were simple wire-frame model rooms.
The virtual reality industry mainly provided VR devices for medical, flight simulation, automobile industry design, and military training purposes from 1970 to 1990.
David Em became the first artist to produce navigable virtual worlds at NASA's Jet Propulsion Laboratory (JPL) from 1977 to 1984. The Aspen Movie Map, a crude virtual tour in which users could wander the streets of Aspen in one of the three modes (summer, winter, and polygons), was created at MIT in 1978.
In 1979, Eric Howlett developed the Large Expanse, Extra Perspective (LEEP) optical system. The combined system created a stereoscopic image with a field-of-view wide enough to create a convincing sense of space. The users of the system have been impressed by the sensation of depth (field of view) in the scene and the corresponding realism. The original LEEP system was redesigned for NASA's Ames Research Center in 1985 for their first virtual reality installation, the VIEW (Virtual Interactive Environment Workstation) by Scott Fisher. The LEEP system provided the basis for most of the early virtual reality headsets.
21st century
The 2000s decade was a period of relative public and investment indifference to commercially available VR technologies.
In 2001, SAS Cube (SAS3) became the first PC-based cubic room, developed by Z-A Production (Maurice Benayoun, David Nahon), Barco, and Clarté. It was installed in Laval, France. The SAS library gave birth to Virtools VRPack. In 2007, Google introduced Street View, a service that shows panoramic views of an increasing number of worldwide positions such as roads, indoor buildings and rural areas. It also features a stereoscopic 3D mode, introduced in 2010.
In 2010, Palmer Luckey designed the first prototype of the Oculus Rift. This prototype, built on a shell of another virtual reality headset, was only capable of rotational tracking. However, it boasted a 90-degree field of vision that was previously unseen in the consumer market at the time. Luckey eliminated distortion issues arising from the type of lens used to create the wide field of vision using software that pre-distorted the rendered image in real-time. This initial design would later serve as a basis from which the later designs came. In 2012, the Rift was presented for the first time at the E3 video game trade show by John Carmack. In 2014, Facebook (later Meta) purchased Oculus VR for what at the time was stated as $2 billion but later revealed that the more accurate figure was $3 billion. This purchase occurred after the first development kits ordered through Oculus' 2012 Kickstarter had shipped in 2013 but before the shipping of their second development kits in 2014. ZeniMax, Carmack's former employer, sued Oculus and Facebook for taking company secrets to Facebook; the verdict was in favor of ZeniMax, settled out of court later.
In 2013, Valve discovered and freely shared the breakthrough of low-persistence displays which make lag-free and smear-free display of VR content possible. This was adopted by Oculus and was used in all their future headsets. In early 2014, Valve showed off their SteamSight prototype, the precursor to both consumer headsets released in 2016. It shared major features with the consumer headsets including separate 1K displays per eye, low persistence, positional tracking over a large area, and Fresnel lenses. HTC and Valve announced the virtual reality headset HTC Vive and controllers in 2015. The set included tracking technology called Lighthouse, which utilized wall-mounted "base stations" for positional tracking using infrared light.
Technology
Hardware
Modern virtual reality headset displays are based on technology developed for smartphones including: gyroscopes and motion sensors for tracking head, body, and hand positions; small HD screens for stereoscopic displays; and small, lightweight and fast computer processors. These components led to relative affordability for independent VR developers, and led to the 2012 Oculus Rift Kickstarter offering the first independently developed VR headset.
Independent production of VR images and video has increased alongside the development of affordable omnidirectional cameras, also known as 360-degree cameras or VR cameras, that have the ability to record 360 interactive photography, although at relatively low resolutions or in highly compressed formats for online streaming of 360 video. In contrast, photogrammetry is increasingly used to combine several high-resolution photographs for the creation of detailed 3D objects and environments in VR applications.
To create a feeling of immersion, special output devices are needed to display virtual worlds. Well-known formats include head-mounted displays or the CAVE. In order to convey a spatial impression, two images are generated and displayed from different perspectives (stereo projection). There are different technologies available to bring the respective image to the right eye. A distinction is made between active (e.g., shutter glasses) and passive technologies (e.g. polarizing filters or Infitec).
In order to improve the feeling of immersion, wearable multi-string cables offer haptics to complex geometries in virtual reality. These strings offer fine control of each finger joint to simulate the haptics involved in touching these virtual geometries.
Special input devices are required for interaction with the virtual world. Some of the most common input devices are motion controllers and optical tracking sensors. In some cases, wired gloves are used. Controllers typically use optical tracking systems (primarily infrared cameras) for location and navigation so that the user can move freely without wiring. Some input devices provide the user with force feedback to the hands or other parts of the body so that the user can orientate themselves in the three-dimensional world through haptics and sensor technology as a further sensory sensation and carry out realistic simulations. This allows the viewer to have a sense of direction in the artificial landscape. Additional haptic feedback can be obtained from omnidirectional treadmills (with which walking in virtual space is controlled by real walking movements) and vibration gloves and suits.
Software
The Virtual Reality Modelling Language (VRML), first introduced in 1994, was intended for the development of "virtual worlds" without dependency on headsets. The Web3D consortium was subsequently founded in 1997 for the development of industry standards for web-based 3D graphics. The consortium subsequently developed X3D from the VRML framework as an archival, open-source standard for web-based distribution of VR content. WebVR is an experimental JavaScript application programming interface (API) that provides support for various virtual reality devices, such as the HTC Vive, Oculus Rift, Google Cardboard or OSVR, in a web browser.
Visual immersion experience
Display resolution
Minimal Angle of Resolution (MAR) refers to the minimum distance between two display pixels. At a distance, a viewer can clearly distinguish the independent pixels. Often measured in arc-seconds, MAR between two pixels has to do with the viewing distance. For the general public, resolution is about 30–65 arc-seconds, which is referred to as the spatial resolution when combined with distance. Given the viewing distance of 1m and 2m respectively, regular viewers won't be able to perceive two pixels as separate if they are less than 0.29mm apart at 1m and less than 0.58mm apart at 2m.
Image latency and display refresh frequency
Most small-size displays have a refresh rate of 60 Hz, which adds about 15ms of additional latency. The number is reduced to less than 7ms if the refresh rate is increased to 120 Hz or even 240 Hz and more. Participants generally feel that the experience is more immersive with higher refresh rates as a result. However, higher refresh rates require a more powerful graphics processing unit.
Relationship between display and field of view
In assessing the achieved immersion by a VR device, one needs to consider their field of view (FOV) in addition to image quality. Eyes have a horizontal FOV from about 107 or 110 degrees to the temporal side to about 60 or 70 degrees toward the nose and a vertical FOV from about 95 degrees downward to 85 degrees upward, and eye movements are estimated as roughly 30 deg to either side horizontally and 20 vertically. Binocular vision is limited to the 120 or 140 degrees where the right and the left visual fields overlap. With eye movements, we have an FOV of roughly 300 degrees x 175 degrees with two eyes, i.e., approximately one third of the full 360-deg sphere.
Applications
Virtual reality is most commonly used in entertainment applications such as video games, 3D cinema, amusement park rides including dark rides and social virtual worlds. Consumer virtual reality headsets were first released by video game companies in the early-mid 1990s. Beginning in the 2010s, next-generation commercial tethered headsets were released by Oculus (Rift), HTC (Vive) and Sony (PlayStation VR), setting off a new wave of application development. 3D cinema has been used for sporting events, pornography, fine art, music videos and short films. Since 2015, roller coasters and theme parks have incorporated virtual reality to match visual effects with haptic feedback. VR not only fits the trend of the digital industry but also enhances the film's visual effect. The film gives the audience more ways to interact through VR technology.
In social sciences and psychology, virtual reality offers a cost-effective tool to study and replicate interactions in a controlled environment. It can be used as a form of therapeutic intervention. For instance, there is the case of the virtual reality exposure therapy (VRET), a form of exposure therapy for treating anxiety disorders such as post traumatic stress disorder (PTSD) and phobias.
A VR therapy has been designed to help people with psychosis and agoraphobia manage their avoidance of outside environments. In the therapy, the user wears a headset and a virtual character provides psychological advice and guides them as they explore simulated environments (such as a cafe or a busy street). NICE is assessing the therapy to see if it should be recommended on the NHS.
During the COVID-19 pandemic, social VR has also been used as a mental-health tool in a form of self-administered, non-traditional cognitive behavioural therapy.
Virtual reality programs are being used in the rehabilitation processes with elderly individuals that have been diagnosed with Alzheimer's disease. This gives these elderly patients the opportunity to simulate real experiences that they would not otherwise be able to experience due to their current state. 17 recent studies with randomized controlled trials have shown that virtual reality applications are effective in treating cognitive deficits with neurological diagnoses. Loss of mobility in elderly patients can lead to a sense of loneliness and depression. Virtual reality is able to assist in making aging in place a lifeline to an outside world that they cannot easily navigate. Virtual reality allows exposure therapy to take place in a safe environment.
Medical uses of VR
Virtual reality (VR) technology has emerged as a significant tool in medical training and education. Specifically, there has been a major leap in innovation in surgical simulation and surgical real-time enhancement. Studies done at North Carolina medical institutions have demonstrated improvement in technical performance and skills among medical students and active surgeons using VR training as compared to traditional training, especially in procedures such as total hip arthroplasty. Alongside this, other VR simulation programs such as LapSim, improve basic coordination, instrument handling, and procedure-based skills. These simulations aim to have high ratings for feedback and haptic touch, which provides a more realistic surgical feel.
Studies show significant improvement in task completion time and scores after 4-week training sessions of LapSim. This simulation environment also allows surgeons to practice without risk to real patients, promoting patient safety.
Based on data from research conducted from the University Hospitals Schleswig-Holstein and collaborators from other institutions, medical students and surgeons with years of experience, show marked performance boosts after practicing with LapSim VR technology.
Another recent study at North Carolina University of Chapel Hill has shown that developing VR and Augmented Reality (AR) systems have allowed surgeons to keep their eyes on a patient while accessing CT scans. This VR system allows for laparoscopic imaging integration, real-time skin layer visualization, and enhanced surgical precision capabilities.
These are both examples of how studies have shown surgeons can take advantage of additional virtual reality simulation practices, which can create incredible experiences, provide customized scenarios, and provide independent learning with haptic feedback. These VR systems need to be realistic enough for education tools alongside being able to measure performance of a surgeon.
Some potential future challenges of this technology would be enhancing complex scenarios alongside the realism aspects. These technologies would need to incorporate stress-inducing factors along with other realistic simulation ideas. Furthermore, there would be a need to have better AR integration to help the surgeon have better eyes-on precision guidance. Lastly, there would be a strong need to keep things cost-effective with an abundance of availability.
Concerts
In June 2020, Jean Michel Jarre performed in VRChat. In July, Brendan Bradley released the free FutureStages web-based virtual reality venue for live events and concerts throughout the 2020 shutdown, Justin Bieber performed on November 18, 2021 in WaveXR. On December 2, 2021, non-player characters performed at the Mugar Omni Theater with audiences interacting with a live performer in both virtual reality and projected on the IMAX dome screen. Meta's Foo Fighters Super Bowl VR concert performed on Venues. Post Malone performed in Venues starting July 15, 2022. Megan Thee Stallion performed on AMAZE at AMC Theaters throughout 2022.
On October 24, 2021, Billie Eilish performed on Oculus Venues. Pop group Imagine Dragons performed on June 15, 2022.
Concerns and challenges
Health and safety
There are many health and safety considerations of virtual reality. A number of unwanted symptoms have been caused by prolonged use of virtual reality, and these may have slowed the proliferation of the technology. Most virtual reality systems come with consumer warnings, including seizures; developmental issues in children; trip-and-fall and collision warnings; discomfort; repetitive stress injury; and interference with medical devices. Some users may experience twitches, seizures, or blackouts while using VR headsets, even if they do not have a history of epilepsy and have never had blackouts or seizures before. One in 4,000 people, or .025%, may experience these symptoms. Motion sickness, eyestrain, headaches, and discomfort are the most prevalent short-term adverse effects. In addition, because of the virtual reality headsets' heavy weight, discomfort may be more likely among children. Therefore, children are advised against using VR headsets. Other problems may occur in physical interactions with one's environment. While wearing VR headsets, people quickly lose awareness of their real-world surroundings and may injure themselves by tripping over or colliding with real-world objects.
VR headsets may regularly cause eye fatigue, as does all screened technology, because people tend to blink less when watching screens, causing their eyes to become more dried out. There have been some concerns about VR headsets contributing to myopia, but although VR headsets sit close to the eyes, they may not necessarily contribute to nearsightedness if the focal length of the image being displayed is sufficiently far away.
Virtual reality sickness (also known as cybersickness) occurs when a person's exposure to a virtual environment causes symptoms that are similar to motion sickness symptoms. Women are significantly more affected than men by headset-induced symptoms, at rates of around 77% and 33% respectively. The most common symptoms are general discomfort, headache, stomach awareness, nausea, vomiting, pallor, sweating, fatigue, drowsiness, disorientation, and apathy. For example, Nintendo's Virtual Boy received much criticism for its negative physical effects, including "dizziness, nausea, and headaches". These motion sickness symptoms are caused by a disconnect between what is being seen and what the rest of the body perceives. When the vestibular system, the body's internal balancing system, does not experience the motion that it expects from visual input through the eyes, the user may experience VR sickness. This can also happen if the VR system does not have a high enough frame rate, or if there is a lag between the body's movement and the onscreen visual reaction to it. Because approximately 25–40% of people experience some kind of VR sickness when using VR machines, companies are actively looking for ways to reduce VR sickness.
Children and teenagers in virtual reality
Children are becoming increasingly aware of VR, with the number in the USA having never heard of it dropping by half from Autumn 2016 (40%) to Spring 2017 (19%).
A 2022 research report by Piper Sandler revealed that only 26% of U.S. teens own a VR device, 5% use it daily, while 48% of teen headset owners "seldom" use it. Of the teens who don't own a VR headset, 9% plan to buy one. 50% of surveyed teens are unsure about the metaverse or don't have any interest, and don't have any plans to purchase a VR headset.
Studies show that young children, compared to adults, may respond cognitively and behaviorally to immersive VR in ways that differ from adults. VR places users directly into the media content, potentially making the experience very vivid and real for children. For example, children of 6–18 years of age reported higher levels of presence and "realness" of a virtual environment compared with adults 19–65 years of age.
Studies on VR consumer behavior or its effect on children and a code of ethical conduct involving underage users are especially needed, given the availability of VR porn and violent content. Related research on violence in video games suggests that exposure to media violence may affect attitudes, behavior, and even self-concept. Self-concept is a key indicator of core attitudes and coping abilities, particularly in adolescents. Early studies conducted on observing versus participating in violent VR games suggest that physiological arousal and aggressive thoughts, but not hostile feelings, are higher for participants than for observers of the virtual reality game.
Experiencing VR by children may further involve simultaneously holding the idea of the virtual world in mind while experiencing the physical world. Excessive usage of immersive technology that has very salient sensory features may compromise children's ability to maintain the rules of the physical world, particularly when wearing a VR headset that blocks out the location of objects in the physical world. Immersive VR can provide users with multisensory experiences that replicate reality or create scenarios that are impossible or dangerous in the physical world. Observations of 10 children experiencing VR for the first time suggested that 8-12-years-old kids were more confident to explore VR content when it was in a familiar situation, e.g. the children enjoyed playing in the kitchen context of Job Simulator, and enjoyed breaking rules by engaging in activities they are not allowed to do in reality, such as setting things on fire.
Privacy
Digital privacy concerns have been associated with VR platforms; the persistent tracking required by all VR systems makes the technology particularly useful for, and vulnerable to, mass surveillance, including information gathering of personal actions, movements and responses. Data from eye tracking sensors, which are projected to become a standard feature in virtual reality headsets, may indirectly reveal information about a user's ethnicity, personality traits, fears, emotions, interests, skills, and physical and mental health conditions.
The nature of VR technology means that it can gather a wide range of data about its users. This can include obvious information such as usernames and account information, but also extends to more personal data like physical movements, interaction habits, and responses to virtual environments. In addition, advanced VR systems can capture biometric data like voice patterns, eye movements, and physiological responses to VR experiences. Virtual reality technology has grown substantially since its inception, moving from a niche technology to a mainstream consumer product. As the user base has grown, so too has the amount of personal data collected by these systems. This data can be used to improve VR systems, to provide personalized experiences, or to collect demographic information for marketing purposes. However, it also raises significant privacy concerns, especially when this data is stored, shared, or sold without the user's explicit consent.
Existing data protection and privacy laws like the General Data Protection Regulation (GDPR) in the EU, and the California Consumer Privacy Act (CCPA) in the United States, can be applied to VR. These regulations require companies to disclose how they collect and use data, and give users a degree of control over their personal information. Despite these regulations, enforcing privacy laws in VR can be challenging due to the global nature of the technology and the vast amounts of data collected.
Due to its history of privacy issues, the involvement of Meta Platforms (formerly Facebook, Inc.) in the VR market has led to privacy concerns specific to its platforms. In August 2020, Facebook announced that Oculus products would become subject to the terms of use and privacy policy of the Facebook social network, and that a Facebook account would be required to use future Oculus headset models, and all existing models (via deprecation of the separate Oculus account system) beginning January 2023. The announcement was criticized for the mandatory integration of Oculus headsets with Facebook data collection and policies (including the Facebook real-name policy), and preventing use of the hardware if the user's account is suspended. The following month, Facebook halted the sale of Oculus products in Germany due to concerns from regulators that the new policy was a violation of GDPR. In 2022, the company would later establish a separate "Meta account" system.