Head tracking

[cor]

Hi,

 

I'm very new to Unigine. I have a project for which I need to implement head tracking.

I want to check if this can be done with Unigine?

 

So suppose I have some head tracking system. How can I make this system communicate with Unigine?

What are the preferred methods to make other applications talk to Unigine?

And how can I specify the needed asymitrical frustums for each eye?

 

I appriciate if someone could point me into the right direction.

[ulf.schroeter]

How can I make this system communicate with Unigine?

What are the preferred methods to make other applications talk to Unigine?

 

In general external harware can be integrated via custom c++ driver plugin for hardware access and using this functionality from within UNIGINE script

 

1. write a c++ plugin for hardware setup/shutdown in <pluginname>.init()/.shutdown() and headtracker value polling in <pluginname>.update()
   
2. expose c++ value query function to UNIGINE script e.g. <pluginname>.getHeadtrackerOrientation()
   
3. query current head tracker orientation via <pluginname>.getHeadtrackerOrientation() from within world script update() function each frame
   
4. update camera orientation accordingly each frame
   

 

There is a useful plugin sample in source\samples\Api\Plugin demonstrating c++ part and UNIGINE script access

 

And how can I specify the needed asymitrical frustums for each eye?

 

Not sure about you hardware configuration. Do you want to use some kind of helmet mounted 2-channel display head-tracker or just a head-tracker with 1-channel stereo-rendering ?

 

Regardless of harware configuration, be advised that there are still UNIGINE issues with asymmetric view frustums. If you just meant 2 view frustums with parallax-offset for stereo-rendering but still with symmetric horizontal/vertical field of view this should be no problem.

[cor]

In general external harware can be integrated via custom c++ driver plugin for hardware access and using this functionality from within UNIGINE script

 

1. write a c++ plugin for hardware setup/shutdown in <pluginname>.init()/.shutdown() and headtracker value polling in <pluginname>.update()
   
2. expose c++ value query function to UNIGINE script e.g. <pluginname>.getHeadtrackerOrientation()
   
3. query current head tracker orientation via <pluginname>.getHeadtrackerOrientation() from within world script update() function each frame
   
4. update camera orientation accordingly each frame
   

 

There is a useful plugin sample in source\samples\Api\Plugin demonstrating c++ part and UNIGINE script access

 

Thanks you very much Url!

I'm very new to Unigine so it will take a while before I will be able to get this working. But it's good to know it is possible.

 

Not sure about you hardware configuration. Do you want to use some kind of helmet mounted 2-channel display head-tracker or just a head-tracker with 1-channel stereo-rendering ?

 

Regardless of harware configuration, be advised that there are still UNIGINE issues with asymmetric view frustums. If you just meant 2 view frustums with parallax-offset for stereo-rendering but still with symmetric horizontal/vertical field of view this should be no problem.

 

It's bad to hear that assymetrical frustums do not work. It would be a show stopper for my project.

 

I want to use head tracking with one or more stereoscopic displays. I will track the position of both eyes and need a frustum for each eye.

It should be possible to move your head outside the screen boundary. Thus a frustum with both left and right boundings negative of possitive should be possible.

Is this possible?

[ulf.schroeter]

I want to use head tracking with one or more stereoscopic displays. I will track the position of both eyes and need a frustum for each eye.

It should be possible to move your head outside the screen boundary. Thus a frustum with both left and right boundings negative of possitive should be possible.

 

Hm, I am no tracking/display device expert and I don't think that I fully understand your specific hardware/render requirements. Do you have some kind of schematic system layout or some links to product details of your planned input and display devices ?

[cor]

Hm, I am no tracking/display device expert and I don't think that I fully understand your specific hardware/render requirements. Do you have some kind of schematic system layout or some links to product details of your planned input and display devices ?

 

Sorry it is all in development, so there is no product information yet.

But it is not very difficult. I think I did not explain it very well.

 

Lets start with one display. I will use head tracking hardware that will track the position of both eyes.

So I will need to calculate two frustums each generated frame. One for each eye.

When the persons eye is exactly in the middle of the screen, the frustum for that eye is off course symetrical.

When moving the head to one side the frustum will get assymetrical. It should be possible that the person is capable of moving the

head so far that he actually would not look at the screen when looking straight ahead (that is what I called outside the screen boundary).

So I suppose the left and right boundaries for that frustum become both negative of both positive. Right?

 

Is this possible with Unigine?

 

The multiple display setup just has more screens. So I will have to use 6 frustums for the 3 displays setup. I hope that is possible too.

[ulf.schroeter]

So I suppose the left and right boundaries for that frustum become both negative of both positive. Right?

 

ok, understood. Actually its possible with UNIGINE to define asymmetric camera view frustum, but as described there is still a problem with reflections and probably shadowing. UNIGINE will have to fix it.

[cor]

Let's hope they fix it soon. It will probably be a go or no-go for me.

[ulf.schroeter]

Let's hope they fix it soon. It will probably be a go or no-go for me.

 

Maybe required bug fixing could get higher priority if you contact Denis and Alexander from UNIGINE directly.

[binstream]

We are interested in working in simulation field, so if you'll describe your requirements, you'll get these features done to the next update of Unigine SDK (usually in a month) because as far as I can see there is nothing complicated in them and it will not break things for other customers.

[ulf.schroeter]

We are interested in working in simulation field, so if you'll describe your requirements, you'll get these features done to the next update of Unigine SDK (usually in a month) because as far as I can see there is nothing complicated in them and it will not break things for other customers.

 

hi Denis, very good to hear :) It would be great if we could get asymmetric view frustums to work water tight quickly ! I will prepare additional test case for shadowing/scattering issues.

[cor]

We are interested in working in simulation field, so if you'll describe your requirements, you'll get these features done to the next update of Unigine SDK (usually in a month) because as far as I can see there is nothing complicated in them and it will not break things for other customers.

 

Hi Denis,

 

That sounds great!

 

The project for which I'm searching a game engine is a flight simulator project.

We developped a new type of motion platform.

It will have a one person cabine with minimal 3 stereoscopic displays.

We want to use head tracking to make the experience as real as possible.

 

During my search for the game engine I use a list of features of which the most important ones are:

 

1: High quality graphics

2: Suitable for flight simulator game

3: Multiple display support with individual cameras

4: 3D support

5: Head tracking support with (per eye) asymmetrical frustum

 

about 1: I guess this is more than ok with Unigine.

 

about 2: As far as I understand this is ok with Unigine (Large terains, good LOD system, etc). Although I

did not see a flight sim application yet that used Unigine.

 

about 3: Not sure if mutiple displays with for each a separate camera (or actually 2 cameras) is supported.

It could be that we will choose a separate computer plus graphics card per display. Not sure yet what this

would mean for the game engine.

 

about 4: I'm very glad 3D is natively supported by Unigine. Although I do not know how efficient this is.

Are all rendering passes justd done twice? And what about line alternating polarized stereoscopic displays?

Not all lines would need to be rendered for the display.

 

about 5: Asymmetrical frustum is supported but according to Ulf not perfect yet. And for our system we

would like a complete separate asymmetrical frustum per eye. So it would be great if that will be added!

[cor]

1: High quality graphics

 

I forgot to mention that a high frame rate and low latency are important too.

High frame rate because we expect fast movements.

Low latency because of the head tracking.

 

I'm a little concerned about the high frame rate because Unigine does a lot of rendering passes.

Can some of them be switched off when needed? I know they are needed for good picture quality so

maybe we will use multiple computers and multiple graphic cards (if possible).

 

About the latency. Does Unigine do all the rendering within one frame?

Is the latency limitted to the frame time?

[binstream]

I'm a little concerned about the high frame rate because Unigine does a lot of rendering passes.

Can some of them be switched off when needed? I know they are needed for good picture quality so

maybe we will use multiple computers and multiple graphic cards (if possible).

 

Yes, any of the rendering pass can be switched on/off in runtime: see render_skip_* console variables in "Getting Started / Console" article of the documentation.

 

About the latency. Does Unigine do all the rendering within one frame?

Is the latency limitted to the frame time?

 

Some effects (like cubemap reflections) can be rendered through several frames, but this behavior can be tweaked via UnigineEditor as well.

[cor]

Some effects (like cubemap reflections) can be rendered through several frames, but this behavior can be tweaked via UnigineEditor as well.

 

But I assume this means the reflection is delayed a few frames. Or are some frames stored?

 

So what would the total latency be?

[binstream]

But I assume this means the reflection is delayed a few frames. Or are some frames stored?

 

So what would the total latency be?

 

You can set it to a single frame.

[ulf.schroeter]

I'm a little concerned about the high frame rate because Unigine does a lot of rendering passes.

Can some of them be switched off when needed? I know they are needed for good picture quality so

maybe we will use multiple computers and multiple graphic cards (if possible).

It's also possible to select global shader quality level [low,medium,high] for easy performance tuning. For your flight simulation use-case most probably selecting medium shader quality will be more than enough (most of high shader quality effects will only be hardly visible when flying at higher speeds/altitude), while giving you a huge performance boost.

[danni.coy]

 

 

about 3: Not sure if mutiple displays with for each a separate camera (or actually 2 cameras) is supported.

It could be that we will choose a separate computer plus graphics card per display. Not sure yet what this

would mean for the game engine.

 

This can be done rather hackishly by using A WidgetSpriteRender object. We were able to do 3 cameras that were stitched to be projected on a 180 degree panaramic screen. On sure about using multiple contexts though

[cor]

This can be done rather hackishly by using A WidgetSpriteRender object. We were able to do 3 cameras that were stitched to be projected on a 180 degree panaramic screen. On sure about using multiple contexts though

 

Thanks Danni.

 

Can this be outputted to three different displays?

Does your panoramic screen use 3 displays or projectors?

[ulf.schroeter]

I will prepare additional test case for reflection/scattering issues.

 

done, see asymmetric view frustum bug

[danni.coy]

Thanks Danni.

 

Can this be outputted to three different displays?

Does your panoramic screen use 3 displays or projectors?

 

It involved a special display driver that split a single display buffer into three distorted the image and sent it to separate projector (3 in total).