VR Development

VR System in UNIGINE#

The built-in VR system provides integration of OpenVR and Varjo.

Varjo's industrial-grade headsets with eye-tracking and mixed reality capabilities allow you to create VR and XR applications (eXtended Reality). With Mixed Reality, you can combine the real-world view from front-facing cameras mounted on the headset with the VR image.

Moreover, there is hand-tracking in VR available for Varjo VR and XR headsets through the Ultraleap integration plugin. It provides finger fidelity, two-hand performance, higher tracking robustness, and a lot of other improvements.

OpenVR is a comprehensive VR SDK (API and runtime) that allows access to VR hardware from various vendors without requiring that applications have specific knowledge of the hardware they are targeting.

So, thanks to OpenVR integration, UNIGINE supports VR development for a wide range of VR devices, including the Oculus Rift, HTC Vive, and other OpenVR-compatible devices.

Platforms#

UNIGINE enables development of VR applications for diffent VR platforms.

Oculus VR#

With UNIGINE VR system, you can implement projects for Oculus HMDs through the integration of OpenVR functionality.

For the correct work, you need to install Oculus Rift PC Runtime.

For performance profiling, you can use the Oculus PerfHud tool. It displays performance summary, including frame rate of the application and the unused hardware performance available, latency timing, application render timing, information about the HMD version, and other. To use Oculus PerfHud with UNIGINE, follow the instructions in Oculus documentation.

VR Template provides the models of the Oculus Touch and Oculus Quest controllers that are compatible with Oculus HMDs only.

SteamVR#

OpenVR integration allows you to develop projects for the SteamVR platform. It is not tied to a particular hardware: such projects can be used with any VR devices that support SteamVR.

When developing for SteamVR, first you should download and install Steam and then launch SteamVR.

Notice

If you install one of the HTC Vive HMDs, the SteamVR will be set automatically on the HMD's installation.

For performance profiling, you can use the SteamVR Frame Timing tool.

VR Template provides the models of the following SteamVR-compatible controllers: HTC Vive, HTC Vive Pro, and HTC Vive Cosmos.

Mixed Reality Development#

Thanks to Varjo integration, you can implement Mixed Reality applications to be run with Varjo HMDs.

To develop an application, you need to install Varjo Base and SteamVR.

Mixed Reality management is performed via the VRMixedReality class of UNIGINE API.

Supported Graphics APIs#

The following graphics APIs are supported out of the box:

DirectX 11
DirectX 12
Vulkan

Notice

For OpenGL API support, you will have to use the deprecated VR plugins.

Initialization#

By default, VR is not initialized. To run the engine with VR, you need to specify the -vr_app command-line option on the application start-up.

For OpenVR:
Shell commands
```
-vr_app openvr
```
For Varjo:
Shell commands
```
-vr_app varjo
```

Notice

You cannot launch the VR plugins (OpenVR, Varjo, Oculus) when the built-in VR is initialized, as it overrides the plugin's functionality. If you try to add a VR plugin after VR initialization, you will get an error message.

Console Commands and Variables#

After initialization, a set of VR-related console commands will become available.

vr_info
Description: Command. Prints the information on the current VR API and the connected VR devices — HMD, controllers, and others. For the controllers, it also displays the battery level (if available).
vr_reset_zero_pose
Description: Command. Sets the zero pose to the current tracker position.
vr_render_enabled	^{Config file: *.boot}
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if rendering into the head-mounted display is enabled.	Arguments: 0 - disabled (by default) 1 - enabled
vr_peripheral_rendering_mode_enabled	^{Config file: *.boot}
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if the peripheral rendering mode is enabled. In this mode, the HMD has two context (peripheral) and two focus displays. You can disable two additional viewports to improve peformance. Notice This feature is available for the Varjo devices only.	Arguments: 0 - disabled (by default) 1 - enabled
vr_service_init_timeout	^{Config file: *.boot}
Description: Variable. Prints the current VR initialization timeout in number of attempts. The engine attempts to run VR Service and has to wait for its initialization. Command. Sets the VR initialization timeout in number of attempts. The engine attempts to run VR Service and has to wait for its initialization.	Arguments: [0; 200] - available range 60 - by default
vr_mirror_mode	^{Config file: *.boot}
Description: Variable. Prints the current mirror mode that defines how the mirrored image (i.e. VR image) will be displayed in the target window. Command. Sets the mirror mode that defines how the mirrored image (i.e. VR image) will be displayed in the target window.	Arguments: 0 - black screen (no image is displayed). 1 - image rendered for the left eye. 2 - image rendered for the right eye. 3 - stereo image (both the left and right eyes). (by default)
vr_window_mode	^{Config file: *.boot}
Description: Variable. Prints the current window mode that defines which window will display the mirrored image (i.e. VR image). Command. Sets the window mode that defines which window will display the mirrored image (i.e. VR image).	Arguments: 0 - mirroring is disabled. 1 - main window displays the mirrored image. (by default)
vr_tracking_space	^{Config file: *.boot}
Description: Variable. Prints the current zero pose of the tracking origin. Command. Sets the zero pose of the tracking origin.	Arguments: 0 - seated. 1 - standing. (by default) 2 - raw (uncalibrated).
vr_head_tracking_position_enabled	^{Config file: *.boot}
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if the head position is locked.	Arguments: 0 - disabled 1 - enabled (by default)
vr_head_tracking_rotation_enabled	^{Config file: *.boot}
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if the head rotation is locked.	Arguments: 0 - disabled 1 - enabled (by default)
vr_motion_prediction	^{Config file: *.boot}
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if motion prediction in the Varjo headsets is enabled. When enabled, the engine submits the velocity value from the GBuffer to the Varjo Composer.	Arguments: 0 - disabled (by default) 1 - enabled
vr_motion_prediction_velocity_precision	^{Config file: *.boot}
Description: Variable. Prints the current factor of velocity scale before packing a floating point value into a 2x8 bit unsigned integer (uint). Notice This feature is available for the Varjo devices only. Command. Sets the factor of velocity scale before packing a floating point value into a 2x8 bit unsigned integer (uint). Notice This feature is available for the Varjo devices only.	Arguments: [eps; inf] - available range 32.0f - by default
vr_motion_prediction_velocity_time_delta	^{Config file: *.boot}
Description: Variable. Prints the current factor for optimizing between fast and slow-moving objects. A smaller number works better for fast-moving objects, and vice versa. Notice This feature is available for the Varjo devices only. Command. Sets the factor for optimizing between fast and slow-moving objects. A smaller number works better for fast-moving objects, and vice versa. Notice This feature is available for the Varjo devices only.	Arguments: [eps; inf] - available range 1.0f / 60.0f - by default
vr_foveated_rendering_enabled	^{Config file: *.boot}
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if foveated rendering is enabled. Foveated rendering makes use of the eye tracking functionality in the Varjo headsets to improve performance by reducing the image quality in peripheral areas where the user is not looking. Foveation allows applications to render fewer pixels and achieve a better VR experience. Notice This feature is available for the Varjo devices only.	Arguments: 0 - disabled 1 - enabled (by default)

vr_mixed_reality_chroma_key_enabled
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if chroma keying is enabled. VST capturing from HMD cameras must be enabled.	Arguments: 0 - disabled (by default) 1 - enabled
vr_mixed_reality_depth_test_enabled
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if depth buffer submission is enabled. VST capturing from HMD cameras must be enabled.	Arguments: 0 - disabled (by default) 1 - enabled
vr_mixed_reality_alpha_blend_enabled
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if alpha blending is enabled. This option is used for blending VR and AR images using the alpha channel. VST capturing from HMD cameras must be enabled and the screen precision must be 1.	Arguments: 0 - disabled (by default) 1 - enabled
vr_mixed_reality_video_enabled
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if the video signal from the real-world view from the front-facing HMD-mounted cameras is enabled. The real-world view is used for combining virtual and real-world elements to create an immersive experience in mixed reality.	Arguments: 0 - disabled (by default) 1 - enabled
vr_mixed_reality_depth_test_range_enabled
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if the depth test range usage is enabled. Use the depth test range (Depth Test Near Z, Depth Test Far Z) to control the range for which the depth test is evaluated.	Arguments: 0 - disabled (by default) 1 - enabled
vr_mixed_reality_depth_test_range
Description: Variable. Prints the current depth test range as a two-component vector (the near and far planes). The depth test range usage must be enabled. Command. Sets the depth test range as a two-component vector (the near and far planes). The depth test range usage must be enabled.	Arguments: vec2(0.0f, 1.0f) - default value
vr_mixed_reality_camera_exposure_time
Description: exposure time value that is valid for the connected device.
vr_mixed_reality_camera_exposure_time_mode
Description: Variable. Prints the current exposure adjustment mode for the camera. Command. Sets the exposure adjustment mode for the camera.	Arguments: 0 - exposure adjustment is disabled 1 - automatic exposure adjustment (by default) 2 - manual exposure adjustment
vr_mixed_reality_camera_white_balance
Description: white balance correction value that is valid for the connected device.
vr_mixed_reality_camera_white_balance_mode
Description: Variable. Prints the current white balance adjustment mode for the camera. Command. Sets the white balance adjustment mode for the camera.	Arguments: 0 - white balance adjustment is disabled 1 - automatic white balance adjustment (by default) 2 - manual white balance adjustment
vr_mixed_reality_camera_iso
Description: ISO value for the camera.
vr_mixed_reality_camera_iso_mode
Description: Variable. Prints the current ISO adjustment mode for the camera. Command. Sets the ISO adjustment mode for the camera.	Arguments: 0 - ISO adjustment is disabled 1 - automatic ISO adjustment (by default) 2 - manual ISO adjustment
vr_mixed_reality_camera_flicker_compensation
Description: flicker compensation value for the camera. This is useful when using the HMD indoors with mostly artificial light bulbs, which flicker at the frequency of 50Hz or 60Hz and can cause visual flicker artifacts on the video see through image. The correct setting depends on the underlying power grid's frequency. For example, in most parts of Africa/Asia/Australia/Europe the frequency is 50 Hz and in most parts of North and South America 60 Hz.
vr_mixed_reality_camera_sharpness
Description: Variable. Prints the current sharpness filter power value for the camera. Command. Sets the sharpness filter power value for the camera.	Arguments: [0; 10] - available range 0 - by default
vr_mixed_reality_view_offset
Description: Variable. Prints the current eyes view offset (where eye camera should be positioned when using Mixed Reality): 0 for physical eye position 1 for VST camera position Command. Sets the eyes view offset (where eye camera should be positioned when using Mixed Reality): 0 for physical eye position 1 for VST camera position	Arguments: [0.0; 1.0] - available range 0.0 - by default
vr_mixed_reality_marker_tracking_enabled
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if marker tracking is enabled.	Arguments: 0 - disabled (by default) 1 - enabled
vr_mixed_reality_marker_visualizer_enabled	^{Config file: *.boot}
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating the marker visualizer is enabled.	Arguments: 0 - disabled (by default) 1 - enabled
vr_mixed_reality_blend_masking_mode
Description: Variable. Prints the current mode of the Blend Control Mask that can be used to extend or restrict the chroma key mask or to control the depth testing against the estimated video depth. Command. Sets the mode of the Blend Control Mask that can be used to extend or restrict the chroma key mask or to control the depth testing against the estimated video depth.	Arguments: 0 - Disabled (masking mode is disabled). (by default) 1 - Restrict Video to Mask (show the video pass-through image (VST) in the mask; can be used with chroma key) 2 - Restrict VR to Mask (show VR in the mask; can be used with chroma key) 3 - Restrict VR to Chromakey reduced by Mask (show VR in the mask and chroma elsewhere; requires chroma key)
vr_mixed_reality_blend_masking_debug_enabled
Description: Variable. Prints a value indicating if the option controlled by the command below is enabled. Command. Toggles the value indicating if blend masking debug visualization is enabled. The blend masking mode must be enabled.	Arguments: 0 - disabled (by default) 1 - enabled
vr_mixed_reality_cubemap_mode
Description: Variable. Prints the current mode defining the way the AR texture is set for the environment. Command. Sets the mode defining the way the AR texture is set for the environment.	Arguments: 0 - cubemap streaming from AR cameras is disabled. 1 - environment texture substitutes the sky. 2 - the first environment preset defines the way the AR texture is set for the environment. (by default) 3 - the second environment preset defines the way the AR texture is set for the environment. 4 - the third environment preset defines the way the AR texture is set for the environment.
vr_mixed_reality_cubemap_ggx_quality
Description: Variable. Prints the current quality of the generated GGX mips for the AR cubemap. Command. Sets the quality of the generated GGX mips for the AR cubemap.	Arguments: 0 - low 1 - medium (by default) 2 - high 3 - ultra
vr_mixed_reality_override_color_correction_mode
Description: Variable. Prints the current color correction mode for the stream from the AR cameras. Command. Sets the color correction mode for the stream from the AR cameras.	Arguments: 0 - correction is disabled. (by default) 1 - exposure correction for the stream from the AR cameras. 2 - exposure and white balance correction for the stream.

VR Input#

UNIGINE VR input system provides access to a wide range of VR devices: you can manage input from VR controllers, head-mounted displays (HMDs), base stations, and trackers.

For more details on types of VR devices and their examples, check the article on VR Input System.

VR Template#

UNIGINE provides a VR template that is used to facilitate the creation of a custom application for VR. It contains a set of 3D models of popular VR controllers and the implementation of basic mechanics such as grabbing and throwing objects, pressing buttons, opening/closing drawers, and a lot more.

The template is created using the Component System, so you can easily extend its functionality.

Refer to the Getting Started with VR section to learn how to use the template for C++/C# VR project development.

Basic Workflow on Using VR#

Unlike other 3D applications, developing a VR application is more complex: you should set up a VR environment, add interactive elements, process user input from various VR devices, and so on.

The general workflow of the VR application development in UNIGINE is the following:

Set up your UNIGINE project for VR development:
1. Set up a VR device (check the instructions provided by the vendor, if necessary).
2. Configure the project to use the appropriate VR mode. You can perform it via SDK Browser for C++/C# projects or in UnigineEditor for C# projects, depending on your preferred method of running your application.
3. Initialize VR.
Create the VR environment:
- Create a 3D environment for your VR project by using 3rd-party 3D modeling and level design tools and UnigineEditor.
- Extend the VR environment with animations, sounds, videos, and other interactive elements.
Make the VR environment more interactive:
- Implement components to extend the functionality of objects in the environment and process user input.
- Use UNIGINE's VR Template (available for C++ and C# APIs) that offers a basic set of VR components and allows users to move around and interact with objects in the environment.
Debug and profile your VR application. Use the VR platform-specific and UNIGINE built-in profiling tools to improve performance.
Build and deploy your VR application.

Last update: 2023-12-19

Help improve this article

Was this article helpful?

Report a problem related to this article

(or select a word/phrase and press Ctrl+Enter)

VR Development

In This Article:

See Also#

VR System in UNIGINE#

Platforms#

Oculus VR#

SteamVR#

Mixed Reality Development#

Supported Graphics APIs#

Initialization#

Console Commands and Variables#

VR Input#

VR Template#

Basic Workflow on Using VR#