Multi-Channel Effect Synchronization

[ulf.schroeter]

Hi UNIGINE crew, as the engine has made tremendous progress for being number-one-selection for virtual training simulations requiring double-precision support and multi-channel rendering only a few really critical issues remain for prodution use in this area.

An "ugly" one is (?) robust avoidance of de-synchronization of special effects e.g. particle systems but also effect physics e.g. swinging rope/ wind-waving flag cloth simulation in multi-pc multi-channel rendering.

Especially with the latter aspect of effect physics we encountered all kinds of heavy de-synchronization between channels. This was further amplified by the fact that the physics simulation stoped when exceeding simulation distances and restarting with slight time differences between nodes.

Also in case of different frame times on different channel nodes (e.g. one channel near overload due to heavy scene content in this channel frustum while other nodes nearly idle) de-synchronization kicked in.

So the question: is there any existing approach/engine modification possible to get "deterministic" effects/effects physics other than standard full state replication, which is hardly possible in practice for effects due to the large amount of state data e.g. all particel positions/velocitys) ?

BTW the other issue is roadmap-announced support world for node file streaming, hope to see this feature quite soon :)

[binstream]

We are a assembling new multi-channel system in the studio for tests together with working on a large test project (hundreds of kilometers in size), so multi-channel SFX sync will be improved within about 2 months.

[ulf.schroeter]

Denis, this is very good news and for sure the best approach for pushing the engine's capabilities to even higher limits.

[yan.wuzhi]

Good News! B)

[ulf.schroeter]

Well, it's an older post and I think this challenging aspect hasn't been addressed so far

[ulf.schroeter]

Hi UNIGINE crew, any plans to address these issues with some clever modifications ? Another simple example might be 3D clouds de-synchronization with multi-channel rendering:

 

As the cloude update is not based on some globally synchronized game time or frame number, but just dependant on local per-render-node frame rate this should cause a de-synchronization of the cloud state over longer time e.g. caused by some single node frame rate drop to 30Hz due to overload while other render nodes stay at 60Hz (though it might be harder to spot due to the very diffuse nature of the clouds).

int Clouds::update(float ifps) {
	
	// check clouds
	if(isCreated() == 0) {
		Log::error("Clouds::update(): clouds is not created\n");
		return 0;
	}
	
	// simulate clouds
	switch(state) {
		case 0: if(ifps > 0.0f) randomization(); state = 1; break;
		case 1: subtraction(); state = 2; break;
		case 2: if(ifps > 0.0f) simulation(); state = 3; break;
		case 3: addition(); state = 4; break;
		case 4: extraction(); state = 5; break;
		case 5: illumination(); state = 6; break;
		case 6: layer++; state = 0; break;
	}
	
	return (state == 0);
}

Another side-effect of current frame-based approach is that the cloud simulation does not stop on game pause or adheres to game time scaling. Therefore 3D clouds might be a good starting point for problem/solution evaluation and engine "hardening".

[ulf.schroeter]

At least for particle effects or 3D clouds I think the general solution approach is to use per-instance random number generator with synchronized seed value and some particle spawn/update algorithm (as far as I can rember particle simulation already is based on some internal fixed frame rate approach ?) based on master-managed synchronized game time or game frame number.

 

Current usage of float game time value might cause itself temporal issues for long-running simulations (3h+) due to continous increasing float precision losses for larger float time values. BTW this float time precision issue will most probably also cause issues for long-running object/skinned character animations. I can remember that also UNIGINE water simulation already had some jittering issues due to time value precision loss in the past.

 

In our simulations we therefore switched to integer game frame number using some virtual fixed game frame rate e.g. 30/60Hz. The logical game frame rate was centrally managed on the simulation master and fully independant of the physical render frame rate. Exact game time in double precision can always be recreated on render nodes by simple multiplication of game frame number by fixed game frame time step.

 

For pseudo-deterministic particle simulation we had modified the ObjectParticles internal spawn/simulation logic to be based on the delta between current game frame number and game frame number on particle emission start (per-instance variable set/reset during ObjectParticles::setEnable() in addition to usage of per-instance random generators)

 

Therefore even in case of different physical frame rates the particle simulations on different render-nodes stayed more or less synchronized: If a node dropped some physical frame due to overload in one frame it just did some more spawn/update iterations on the following frame(s) (there are still some possible de-synchronization glitches when changing things like spawn rate during particle simulation, but in practice the differences were not visible between render channels).

 

This also allowed "freezing" all particle simulations on all render-nodes by simply stopping incrementing of the master game frame number e.g. during training after-action review (which is another VERY important requirement for training and simulation applications).

 

With the current implementation particle and cloud simulation simply keeps running during game pauses which will falsify the simulation playback afterwards compared to the previous actual simulation situation leading e.g. to differences in object visibilities due to dynamic smoke/clouds occlusion -> BIG NO-GO in simulation and training.

[binstream]

As we continue to work on multi-channel projects, there will be more improvements on synchronization this year. Thank you for the suggestions!

[ulf.schroeter]

 

Bohemia Interactive SIM VBS IG

The latest innovation in image generation. We demonstrated VBS IG running at 60 hz and perfectly synchronized across all displays, including particle effects and vehicles. VBS IG includes a VBS3 host communicating via CIGI v3.3 protocols and allows users to connect desktop instances of VBS3 with full mission simulators.

 

ok, once again the competitors are faster in the end...

 

http://us1.campaign-archive1.com/?u=a26fbe53f68fee4dde9f902b7&id=baac36344f&e=0eec3033f5