Mixed Reality


photo

Recommended Posts

 

The important scene is between 1:00 and 1:12.

Is there a means to develop MR content?

Do I have to use varjo hmd?

Link to post

Dear @dongju.jeong,

Quote

Is there a means to develop MR content?

I am using Unigine for Mixed Reality. So content creation is exactly similar as we make it, the only rule is to make it with the scale of real world when you really want to match the virtual against real world.

Quote

Do I have to use varjo hmd?

For Mixed reality you can really use any camera. From where you will take the real world feed and match it with virtual.

There are two difficult parts in Mixed reality

1. Tracking the correct 3D pose of camera in real world (The same pose you feed to your virtual camera)

2. The other difficult part is to calibrate the lens of your camera. It gives you correct Field of View to your virtual camera as your real camera. Even real camera zoom and focus gears are considered for this thing. Also Depth of field can be simulated after lens calibration.

Varjo is one such device which gives you Immersive feeling with inbuilt camera feed.

For my implementation I had used 

1. Real Broadcast ENG camera.

2. Stype Red Spy, Anti latency for tracking camera pose in real world

3. Canon 2/3" ENG Lens which was calibrated for Field of View, Nodal Offset, Depth Of Filed effect and lens distortions K1 and K2 for all Zoom and Focus gear ranges. This will be easier with Varjo as it will be having prime lens with fixed FOV. 

Regards,

Rohit.

  • Thanks 1
Link to post

Thank you for answer.

image.thumb.png.a5578c1dff77a15eaf04a6e40c7947b9.png

Images of the real world taken with real cameras should be partially projected onto HMD displays.

In other words, players wearing HMD can see both the real cockpit and the virtual world at the same time.


 

Varjo and vive pro are equipped with cameras.

Can we perform this function with only hmd and unigine?

Link to post

Dear @dongju.jeong,

I have experience with old models of Vive. The driver initiates front camera only in certain modes. It is bit sloppy for such applications. Most of the times you will get blur images. Any how I see the steam dashboard still not allowing the full usage of the front camera. So it could be a blocker. 

Second point is HTC Vive tracking has noise. Which is considerable for the application of Mixed reality. The devices I used for tracking like Red Spy unit which is very costly, gives sub mm pixel accuracy for tracking and it composites your real world with virtual world at that level.  Though adding two more substations and less occlusions you can have the good results with HTC Vive but still the tracking will be limited to small area. One solution could be adding Anti latency device to the environment and use HMD for only display. It will scale up the tracking area and tracking will be much more precise. Still antilatency also has 1mm3 noise when things are static but for mixed reality applications it works.

Looking at above discussion. you can still have to understand Varjo. I don't know much more about Varjo than their promotional videos. But one needs to get hands on and then pass the product. So for above issues you need to get clarifications from Varjo. Write to them. I just checked the specs on their site. Looks like they are also with Base stations (Lighthouse) Steam VR drivers, It limits the area unless there is a driver support for cascading the base stations. Good to know that they have Lidars and depth sensing in some models (Nice).

Again Mixed reality must consider following parameters

1. Area of tracking

2. External of Indoor

3. Lens data of camera you are using. You must know the lens intrinsic parameters of HMD or Varjo. Anyhow you can compute it.

4. Physical form of the device and deployment challenges.

5. Cost. (I think mixed reality costs are still very high and they will remain high)

But answer to your question is Yes, you can do and mix that virtual with real. No problem. 

Regards,

Rohit

  • Thanks 1
Link to post