VR head position issue after migration 2.18 -> 2.19

[Lales.Charles]

Hi,

We just got a weird issue in VR regarding head positionning.
Our code is quite related to VR sample one.

If you consider VRPlayerVR::update() (we use ::post_update() instead), here are small difference between SDK version..
2.18

Mat4 hmd_transform = Mat4(head_device->getTransform()); //2.18
Mat4 hmd_transform_world = player_transform * hmd_transform; //2.18

2.19

Mat4 hmd_transform_world = head_device->getWorldTransform(); //2.19

Later, when changing current player to different reference cameras, we use this code:

void VRPlayerVR::onCameraSelectionChanged(const char* viewName)
{
	auto navPlayer = PlayerController::get()->getPlayer(viewName);
	player->setParent(navPlayer);
	player->setPosition(Vec3_zero);

	if (navPlayer) {
			navPlayer->setTransform(
				navPlayer->getTransform() *			// point VR camera ("foot/seat") to the target
				translate(-getHead()->getPosition())	// put "eyes" to the "foot/seat" position
			);
	}
}

So, the code VRPlayerVR::update() of 2.18 is working fine, whereas 2.19 induce some wrong head positioning.

Would you see why?

Regards,
Charles

[silent]

Hi Charles,

Could you please elaborate on what do you mean by "wrong head positioning"? Maybe you have a screenshot or coordinates data before (2.18) / after (2.19)?

Thanks!

[Lales.Charles]

Hi,

Yes indeed, my post is lacking few information..
Also, I found back where comes the difference from: 2.16 -> 2.18 migration, not 2.18 -> 2.19 as descibed previously.

2.16 VRPlayerOpenVR::update()

[..]	
Mat4 player_transform = player->getWorldTransform();
Mat4 hmd_transform = Mat4(openvr.getDevicePose(HMD_DEVICE_0));
Mat4 hmd_transform_world = player_transform * hmd_transform;
head_offset =
	(openvr.isDeviceConnected(HMD_DEVICE_0) && HMD_DEVICE_0 != -1) ?
	player_transform.getTranslate() - hmd_transform_world.getTranslate() :
	Vec3_zero;
head_offset.z = 0;
head->setWorldTransform(hmd_transform_world);
[..]

2.18 VRPlayerVR::update() (no more VRPlayerOpenVR)

[..]
Mat4 player_transform = player->getWorldTransform();
Mat4 hmd_transform_world = head_device->getWorldTransform();
head_offset =
	(head_device && head_device->isAvailable()) ?
	player_transform.getTranslate() - hmd_transform_world.getTranslate() :
	Vec3_zero;
head_offset.z = 0.0f;
head->setWorldTransform(hmd_transform_world);
[..]

So in fact I missed to properly sync our code base with 2.18 VR sample.. and kept the 2 lines until 2.19 when we got the issue.

The issue in question: "wrong head positioning".. what does it mean?
We do use some camera reference positions from 2 contexts:
- world cameras: taxi way, control tower, etc.
- entity cameras: pilot, copilot, cabin, etc.
Entil now we constraint the user, preventing him from teleporting himself where he would like.
We only let an option for an operator to move with keyboard / mouse, positionning freely the headset of the VR user.
But quite soon, we'd like to reactive this feature (self teleportation for the user), with here also 2 contexts: world (e.g. walking in show-room) or entity (cabin).
So, "wrong head positioning" means that the headset is not located at the expected "camera reference position" currently selected. In this case, when we select pilot, we get positionned slighty behind in the cabin. Same issue with all reference positions from the entity.. as from the world. Simply more visible inside the entity.

Also here is the main difference from VR sample: we do have a PlayerController (CIGI, Locked, UnLocked) and a CameraController (differente cameras used as nodes, simply cameras are more convenient to set positions in editor). 

If the VR plugin is loaded, each time an operator pick another Camera (from world or current entity), the VR headset position needs to be sync.

VRPlayerVR::init()

[..]
PlayerController::get()->addOnSelectionChangedCallback(MakeCallback(this, &VRPlayerVR::onCameraSelectionChanged));
[..]

VRPlayerVR::onCameraSelectionChanged(const char* viewName)

[..]
auto navPlayer = PlayerController::get()->getPlayer(viewName);
player->setParent(navPlayer);
player->setPosition(Vec3_zero);

if (navPlayer) {
	navPlayer->setTransform(
		navPlayer->getTransform() *		// point VR camera ("foot/seat") to the target
		translate(-getHead()->getPosition())	// put "eyes" to the "foot/seat" position
	);
}
[..]

So, the question would be: why the different code 2.16 / 2.19 to update hmd_transform_world would impact the getHead()->getPosition() in the way we use it?

Mat4 hmd_transform = Mat4(head_device->getTransform()); //2.16
Mat4 hmd_transform_world = player_transform * hmd_transform; //2.16

Mat4 hmd_transform_world = head_device->getWorldTransform(); //2.19

Also of course, feel free to suggest another approach to full fill the needs in term of ref positions and user self teleportation in the 2 given contexts!

Regards,
Charles

[Lales.Charles]

Hi,

I'd like to add another difference between our code base and VR sample..

void VRPlayerVR::init_player()
{
	player = checked_ptr_cast<PlayerDummy>(node);
	//Game::setPlayer(player); //this is managed by our Interactive plugin, the VR plugin reacting to change events from it
	head = NodeDummy::create();
	head->setParent(player); //thanks to Alexander from Unigine, fix given time ago, this is different from VR sample..
	[..]
}

Regards,
Charles