vr Attach to HMD

[max_iz]

I'm attempting to simulate an AR monocle, in a VR environment.
I understand that GUI cannot be rendered in VR normally, so I am using a world space ObjectGUI.

The problem I am having is this: I can't seem to be able to get the ObjectGUI to stick to the user's field of view.
I also need to render the GUI to only one eye, but I haven't looked into that yet.

I've tried:
• Setting the ObjectGUI as a child of the camera.
• Moving the ObjectGUI to camera position on update, using a C# script:

ui.WorldPosition = cam.WorldPosition + offset;

• Moving the ObjectGUI to camera position, alternative method:

camera.GetDirectionFromScreen(out var p0, out var p1, 0.5f, 0.5f, 16f / 9);
var dir = (p1 - p0).Normalized;
ui.WorldPosition = p0 + dir * distance;

• There is mention of an "Attach to HMD" property, here: https://developer.unigine.com/en/docs/2.13/start/vr/?rlang=cpp&words=hmd#attach_to_hmd
However, there is no such file in my project, and I couldn't find a way to get it.

All my attempts ended up moving the ObjectGUI in front of the camera object before accounting for the HMD position (at the player's feet).

I've read in the documentation that "camera is not a node".
Does this mean I can't get the VR camera's transform information?
Surely there must be a way.

Any help would be appreciated.

Edited November 29, 2020 by max_iz
Removed a line with black background I copied from VSCode by accident.

[alexander]

Hi max_iz,

I can't seem to be able to get the ObjectGUI to stick to the user's field of view.
All my attempts ended up moving the ObjectGUI in front of the camera object before accounting for the HMD position (at the player's feet).
Player's position (Player.WorldPosition) is the position of the HMD's center of tracking in world coordinates.
Camera's position (Player.Camera.IModelview) is the position (transformation) of the HMD in world coordinates. Use this if you want to attach a GUI to the HMD (but you need to update positions every frame in the WorldLogic.PostUpdate() method).

I also need to render the GUI to only one eye
Unfortunately, you can't do this yet. I'll create a feature ticket for the next SDK releases.

Best regards,
Alexander

[max_iz]

Thank you for replaying, Alexander.

Unfortunately, I'm still getting the same result.
The object is locked to the HMD's center of tracking, the parent node of the camera, and not the camera itself.

This is my code, that I placed in an Update method, in a component:

var rot = camera.IModelview.GetRotate();
var pos = camera.IModelview.Translate;
mountNode.SetWorldRotation(rot);
mountNode.WorldPosition = pos;

Am I doing something wrong here?

Edit: I'm using the HTC Vive Cosmos.

Edited November 30, 2020 by max_iz

[max_iz]

[SOLVED]

I've verified that I'm getting the correct camera object, going as far as creating a new one, and assigning it to the player.

I'm printing the position and rotation of the camera, from the IModelView matrix as suggested, and I get the values of the camera tracking center, and not the HMD.

This really seems like a bug.
Or at the very least, misleading documentation.

After some time, I found something interesting:
AppVive.IsDeviceConnected(1) returns false, despite the headset working fine.
According to the AppVive.Device enum, 1 should be the HMD.
Another possible bug, or incorrect documentation.

After some trial and error, I found that the HMD is enumerated to 0, not 1.

Finally, I was able to get the HMD rotation and position offset, using the following code:

var matrix = AppVive.GetDeviceTransform(0);
var rot = matrix.GetRotate();
var pos = matrix.Translate;

Unfortunately, roll and yaw appear to be switched.
So I have to break it down to Euler angles, and assign in a different order, but it works now.

What a mess.

Please let me know if you have another way to achieve this.

Edited December 7, 2020 by max_iz

[silent]

Quote

Or at the very least, misleading documentation.

Could you please give us some additional info on this? We will improve related articles in the documentation if possible.

Thanks!

[max_iz]

camera.Position doesn't work in VR, returning the position of the HMD center of tracking.
Despite claiming to get the position from the view matrix.

This is either a bug, or a mistake in the documentation, both in the code XML comments, as well as the website.

If this class does not work in VR, it should say so, but this is not mentioned anywhere.

 

I should mention, this is in C#.

Edited December 8, 2020 by max_iz
Forgot to mention, it's in C#

[Ugly1-1]

Watching this thread. I was attempting to solve the same problem the same way.

It would be nice if the "AdditionalCameraOffset" and "AdditionalCameraRotation" applied by VR devices (an assumption on my part on how they work with player controllers) could be turned off for ObjectGui. For HMDs, or VR games in general there's a case to have Widgets always front and centre.

In addition, the restoration of the HMD recenter API call would be greatly appreciated in a future update as it's not present in 2.13 (at time of writing).