VR & XR Development

XR (extended reality), is a general term for any enhanced technology combining the real and digital worlds to create new kinds of interactivity and perception. Essentially, this term is an umbrella under which the following types of applications:

• Virtual Reality (VR): is an immersive self-contained environment that makes the user feel inside a virtual or digitally reproduced real 3D world.
• Mixed Reality (MR): user's real-world environment combined with the application's virtual environment with an ability to interact with each other.
• Augmented Reality (AR): application's virtual content is projected onto physical glass or a display over a view of the real world.
  Notice

  Development of AR applications for mobile platforms is not currently supported.

These technologies provide new ways to train, educate, entertain, and collaborate.

UNIGINE offers built-in functionality for development of Virtual Reality and Mixed Reality applications. It enables you to implement logic for various VR devices through a single API interface without using additional plugins.

See Also#

The built-in VR system provides integration of OpenVR and Varjo.

OpenVR Integration#

Varjo Integration and Supported Features#

Image Courtesy of Varjo.

Varjo's industrial-grade headsets with eye-tracking and mixed reality capabilities allow you to create VR and MR applications. With Mixed Reality, you can combine the real-world view from front-facing cameras mounted on the headset with the VR image.

Moreover, there is hand-tracking in VR available for Varjo VR and XR headsets through the Ultraleap integration plugin. It provides finger fidelity, two-hand performance, higher tracking robustness, and a lot of other improvements.

The following table lists Varjo native features that are supported by UNIGINE VR system:

Varjo Features	VR System
Supported graphics APIs	DX11, DX12, Vulkan
Human-eye resolution
Eye tracking
Eye tracking parameters
Eye camera datastream
Mixed reality (MR)
MR environment reflections
MR post-process shaders
MR datastream
Chroma keying
Varjo Markers
Depth buffer submission
Velocity buffer submission
Multi-app support
Support for multiple layers
Occlusion mask
Instanced rendering
Foveated rendering
SLI-compatible

UNIGINE enables development of VR applications for diffent VR platforms.

Oculus VR#

SteamVR#

The following VR device models are supported:

• Oculus Rift / Rift S / Quest / Quest 2 (with Oculus Link cable / Oculus Link wireless)
• HTC Vive / Vive Pr / Focus / Cosmos
• Varjo VR-1 / VR-2 / VR-3 / XR-3 (with extended mixed reality support)
• Windows Mixed Reality (WMR)-compatible
• OpenVR-compatible

The following graphics APIs are supported out of the box:

• DirectX 11
• DirectX 12
• Vulkan

By default, VR is not initialized. To run the engine with VR, you need to specify the -vr_app command-line option on the application start-up.

• For OpenVR:
  Shell commands

  -vr_app openvr

• For Varjo:
  Shell commands

  -vr_app varjo

UNIGINE VR input system provides access to a wide range of VR devices: you can manage input from VR controllers, head-mounted displays (HMDs), base stations, and trackers.

For more details on types of VR devices and their examples, check the article on VR Input System.

UNIGINE provides a VR Template that is used to facilitate the creation of a custom application for VR. It contains a set of 3D models of popular VR controllers and the implementation of basic mechanics such as grabbing and throwing objects, pressing buttons, opening/closing drawers, and a lot more.

The template is created using the Component System, so you can easily extend its functionality.

Refer to the Getting Started with VR section to learn how to use the template for C++/C# VR project development.

Unlike other 3D applications, developing a VR application is more complex: you should set up a VR environment, add interactive elements, process user input from various VR devices, and so on.

The general workflow of the VR application development in UNIGINE is the following:

1. Set up your UNIGINE project for VR development:

   1. Set up a VR device (check the instructions provided by the vendor, if necessary).
   2. Configure the project to use the appropriate VR mode. You can perform it via SDK Browser for C++/C# projects or in UnigineEditor for C# projects, depending on your preferred method of running your application.
   3. Initialize VR.

2. Create the VR environment:

   • Create a 3D environment for your VR project by using 3rd-party 3D modeling and level design tools and UnigineEditor.
   • Extend the VR environment with animations, sounds, videos, and other interactive elements.

3. Make the VR environment more interactive:

   • Implement components to extend the functionality of objects in the environment and process user input.
   • Use UNIGINE's VR Template (available for C++ and C# APIs) that offers a basic set of VR components and allows users to move around and interact with objects in the environment.
4. Debug and profile your VR application. Use the VR platform-specific and UNIGINE built-in profiling tools to improve performance.
5. Build and deploy your VR application.