VR head position issue after migration 2.16 -> 2.19

[Lales.Charles]

Hi,

We just got a weird issue in VR regarding head positionning.
Our code is quite related to VR sample one.

If you consider VRPlayerVR::update() (we use ::post_update() instead), here are small difference between SDK version..
2.18

Mat4 hmd_transform = Mat4(head_device->getTransform()); //2.18
Mat4 hmd_transform_world = player_transform * hmd_transform; //2.18

2.19

Mat4 hmd_transform_world = head_device->getWorldTransform(); //2.19

Later, when changing current player to different reference cameras, we use this code:

void VRPlayerVR::onCameraSelectionChanged(const char* viewName)
{
	auto navPlayer = PlayerController::get()->getPlayer(viewName);
	player->setParent(navPlayer);
	player->setPosition(Vec3_zero);

	if (navPlayer) {
			navPlayer->setTransform(
				navPlayer->getTransform() *			// point VR camera ("foot/seat") to the target
				translate(-getHead()->getPosition())	// put "eyes" to the "foot/seat" position
			);
	}
}

So, the code VRPlayerVR::update() of 2.18 is working fine, whereas 2.19 induce some wrong head positioning.

Would you see why?

Regards,
Charles

[silent]

Hi Charles,

Could you please elaborate on what do you mean by "wrong head positioning"? Maybe you have a screenshot or coordinates data before (2.18) / after (2.19)?

Thanks!

[Lales.Charles]

Hi,

Yes indeed, my post is lacking few information..
Also, I found back where comes the difference from: 2.16 -> 2.18 migration, not 2.18 -> 2.19 as descibed previously.

2.16 VRPlayerOpenVR::update()

[..]	
Mat4 player_transform = player->getWorldTransform();
Mat4 hmd_transform = Mat4(openvr.getDevicePose(HMD_DEVICE_0));
Mat4 hmd_transform_world = player_transform * hmd_transform;
head_offset =
	(openvr.isDeviceConnected(HMD_DEVICE_0) && HMD_DEVICE_0 != -1) ?
	player_transform.getTranslate() - hmd_transform_world.getTranslate() :
	Vec3_zero;
head_offset.z = 0;
head->setWorldTransform(hmd_transform_world);
[..]

2.18 VRPlayerVR::update() (no more VRPlayerOpenVR)

[..]
Mat4 player_transform = player->getWorldTransform();
Mat4 hmd_transform_world = head_device->getWorldTransform();
head_offset =
	(head_device && head_device->isAvailable()) ?
	player_transform.getTranslate() - hmd_transform_world.getTranslate() :
	Vec3_zero;
head_offset.z = 0.0f;
head->setWorldTransform(hmd_transform_world);
[..]

So in fact I missed to properly sync our code base with 2.18 VR sample.. and kept the 2 lines until 2.19 when we got the issue.

The issue in question: "wrong head positioning".. what does it mean?
We do use some camera reference positions from 2 contexts:
- world cameras: taxi way, control tower, etc.
- entity cameras: pilot, copilot, cabin, etc.
Entil now we constraint the user, preventing him from teleporting himself where he would like.
We only let an option for an operator to move with keyboard / mouse, positionning freely the headset of the VR user.
But quite soon, we'd like to reactive this feature (self teleportation for the user), with here also 2 contexts: world (e.g. walking in show-room) or entity (cabin).
So, "wrong head positioning" means that the headset is not located at the expected "camera reference position" currently selected. In this case, when we select pilot, we get positionned slighty behind in the cabin. Same issue with all reference positions from the entity.. as from the world. Simply more visible inside the entity.

Also here is the main difference from VR sample: we do have a PlayerController (CIGI, Locked, UnLocked) and a CameraController (differente cameras used as nodes, simply cameras are more convenient to set positions in editor). 

If the VR plugin is loaded, each time an operator pick another Camera (from world or current entity), the VR headset position needs to be sync.

VRPlayerVR::init()

[..]
PlayerController::get()->addOnSelectionChangedCallback(MakeCallback(this, &VRPlayerVR::onCameraSelectionChanged));
[..]

VRPlayerVR::onCameraSelectionChanged(const char* viewName)

[..]
auto navPlayer = PlayerController::get()->getPlayer(viewName);
player->setParent(navPlayer);
player->setPosition(Vec3_zero);

if (navPlayer) {
	navPlayer->setTransform(
		navPlayer->getTransform() *		// point VR camera ("foot/seat") to the target
		translate(-getHead()->getPosition())	// put "eyes" to the "foot/seat" position
	);
}
[..]

So, the question would be: why the different code 2.16 / 2.19 to update hmd_transform_world would impact the getHead()->getPosition() in the way we use it?

Mat4 hmd_transform = Mat4(head_device->getTransform()); //2.16
Mat4 hmd_transform_world = player_transform * hmd_transform; //2.16

Mat4 hmd_transform_world = head_device->getWorldTransform(); //2.19

Also of course, feel free to suggest another approach to full fill the needs in term of ref positions and user self teleportation in the 2 given contexts!

Regards,
Charles

[Lales.Charles]

Hi,

I'd like to add another difference between our code base and VR sample..

void VRPlayerVR::init_player()
{
	player = checked_ptr_cast<PlayerDummy>(node);
	//Game::setPlayer(player); //this is managed by our Interactive plugin, the VR plugin reacting to change events from it
	head = NodeDummy::create();
	head->setParent(player); //thanks to Alexander from Unigine, fix given time ago, this is different from VR sample..
	[..]
}

Regards,
Charles

[mifril]

Hi,

If I understand correctly, you have an architecture like this: a PlayerController that stores all the "viewpoints" in your scene, when you want to change the position of a VRPlayer, you look up the "viewpoint", then make the VRPlayer a child of that viewpoint, then set up a transform on the VRPlayer to set the camera to the viewpoint.

What is viewpoint? Is it Player or NodeDummy?

I think it would be better if you transform player instead of navPlayer in onСameraSelectionChanged, because as I understand it, navPlayer is a fixed pre-set point that is better not to move.

[..]
auto navPlayer = PlayerController::get()->getPlayer(viewName);
player->setParent(navPlayer);
player->setTransform(Mat4_identity);

if (navPlayer) {
	[layer->setPosition(-getHead()->getPosition());	// put "eyes" to the "foot/seat" position
}
[..]

It would be better if you send us some log, as Silent asked earlier.

Just add logging to places where you change head node transform, and player transform and run the app, then you can send it to us as screenshot or text, it will help to understand issue better.

 

[Lales.Charles]

Hi,

I'll try to setup a default project to share with you both Players and Cameras managers.

But here are already some screenshots.

In the editor, you can see how we simply declare reference positions by entity: pilot, copilot & co.

In VR, here is the correct position of headset (using 2 lines from 2.16)..

.. whereas here is the incorrect position this time using 1 line from 2.18.
The user is to too high, inside the model of the helicopter, for any of reference positions attached to the entity.

Are the lines stricly equivalent?

//until 2.16
Mat4 hmd_transform = Mat4(head_device->getTransform()); 
Mat4 hmd_transform_world = player_transform * hmd_transform;
//since 2.18
Mat4 hmd_transform_world = head_device->getWorldTransform(); 

Because variables are different:

2.16
hmd_transform_world = {m00=0.99996239086897798 m10=-0.0078743674364005461 m20=-0.00080522395213819444 ...}
head_offset = {x=0.00061792749192335350 y=0.79030197339214636 z=0.0000000000000000 ...}

2.18
hmd_transform_world = {m00=0.99996393857422328 m10=-0.00032108850903008775 m20=0.0080671081556780329 ...}
head_offset = {x=-0.00015311432013331849 y=1.9895404325787069e-05 z=0.0000000000000000 ...}

Regards,
Charles

[silent]

Hi Charles,

We did couple of tests in 2.16 and 2.19 and HMD coordinates always matches between these two versions.

In attachment you can find a small test component that we used. In the empty project we also attached head to player (as you do):

Maybe you can check this component with your nodes hierarchy in empty project and you easily find out where the difference is? Also I've send you PM with 2 project (2.16.x and 2.19.x) so you can also test it as we tested.

Thanks!

Test.cpp Test.h

[Lales.Charles]

Hi,

Thanks for investigation and both test projects. Tested, they both work fine with our VR setup.

In our code, we don't do the setPlayer in the VR plugin..

void VRPlayerVR::init_player()
{
	player = checked_ptr_cast<PlayerDummy>(node);
	//Game::setPlayer(player);
	head = NodeDummy::create();
	head->setParent(player); //thanks to Alexander from Unigine
	[..]
}

.. in fact the only setPlayer is done by the Interactive plugin

void applyNavPlayer(const char* viewName, const char* viewGroup, const char* viewPoint)
{
	// Ingame, the camera hierarchy will be:
	// 
	// * For CIGI:
	//   + nav player = cigi camera
	// 
	// * For LOCKED and UNLOCKED:
	//   + entity or world node
	//     + camera viewpoint inside node
	//       + nav player

	// Selected view point
	PlayerPtr viewPointCamera = CameraController::get()->getCamera(viewGroup, viewPoint);
	// Selected navigation camera
	PlayerPtr navPlayer = PlayerController::get()->getPlayer(viewName);

	bool isCigi = (0 == strcmp(viewName, "CIGI"));

	if (!isCigi) { // Never reparent the cigi camera!
		navPlayer->setParent(viewPointCamera);
		navPlayer->setPosition({});
		navPlayer->setRotation({});
	}

	// activate camera
	Game::setPlayer(navPlayer);
}

whereas players come from 2 plugins

	//CIGI plugin
	auto cigi_player = Game::getPlayer();
	PlayerController::get()->registrate("CIGI", cigi_player);

	//Interactive plugin
	// The LOCKED player can't be moved by the user
	PlayerDummyPtr player = PlayerDummy::create();
	player->setZNear(0.01f);
	player->setZFar(100000.0f);
	PlayerController::get()->registrate("LOCKED", player);
	// The UNLOCKED player can be moved freely by the user, while still being tethered
	PlayerSpectatorPtr player = PlayerSpectator::create();
	player->setCollision(0);
	player->setZNear(0.01f);
	player->setZFar(100000.0f);
	player->setMinVelocity(2.0f);
	player->setMaxVelocity(200.0f);
	PlayerController::get()->registrate("UNLOCKED", player);

And as reminder, here is the code using these players in VR plugin

void VRPlayerVR::onCameraSelectionChanged(const char* viewName)
{
 	[..]
	auto navPlayer = PlayerController::get()->getPlayer(viewName);
	player->setParent(navPlayer);
	player->setPosition(Vec3_zero);

	//ignore CIGI
	if (navPlayer) {
		if (String::compare(navPlayer->getName(), "CIGI") != 0)
		{
			navPlayer->setTransform(
				navPlayer->getTransform() *			// point VR camera ("foot/seat") to the target
				translate(-getHead()->getPosition())	// put "eyes" to the "foot/seat" position
			);
		}
	}
}

Do you think this could be related to the issue?

Regards,
Charles