Syncker Plugin

Simply connect multiple computers into one high-resolution seamless display using LAN. And it does not matter if these computers are running on Windows and Linux at the same time, as Syncker works even across different platforms. Moreover, created virtual environments can have any monitor configuration, since all viewports are fully configurable and multi-monitor grids on slave computers are supported.

Syncker

Computers in LAN connected via Syncker

Structure and Principles#

Syncker allows you to render a world across different computers synchronized over LAN. These computers can be of two types:

Master	Master application is an application running on the main computer apart from rendering it also performs calculations of physics, animations and controls slaves. Notice The main computer should have the most advanced hardware configuration, as it performs most of the calculations. Nodes, materials and cameras are created/deleted and moved according to the application logic. All other applications are synchronized with a master.
Slaves	Slave applications are all other applications running on computers connected over the network. There can be an unlimited number of slaves connected to one master (as long as network bandwidth allows it). The purpose of such applications is to render all nodes that are seen in their viewports. All rendering is done directly on a slave GPU. Physics simulation is performed by the master. Notice Physics simulation in slave applications should be disabled to improve performance. Objects of the following types are synchronized automatically: ObjectWaterGlobal, ObjectCloudLayer, WorldLight if they present in the *.world file on all computers. Animations are updated on the master computer, while slaves only get calculated bone transformations. Particle systems are updated on the slave but according to parameters sent from the master. Physics simulation in slave applications should be disabled not to override position and orientation of nodes sent over the network from a master. Configuration of slaves is set up to fit the configuration of monitors.

Master

Master application is an application running on the main computer apart from rendering it also performs calculations of physics, animations and controls slaves.

NoticeThe main computer should have the most advanced hardware configuration, as it performs most of the calculations.

Nodes, materials and cameras are created/deleted and moved according to the application logic.

All other applications are synchronized with a master.

Slaves

Slave applications are all other applications running on computers connected over the network. There can be an unlimited number of slaves connected to one master (as long as network bandwidth allows it). The purpose of such applications is to render all nodes that are seen in their viewports.

All rendering is done directly on a slave GPU.
Physics simulation is performed by the master.
Notice
Physics simulation in slave applications should be disabled to improve performance.
Objects of the following types are synchronized automatically: ObjectWaterGlobal, ObjectCloudLayer, WorldLight if they present in the *.world file on all computers.
Animations are updated on the master computer, while slaves only get calculated bone transformations.
Particle systems are updated on the slave but according to parameters sent from the master.
Physics simulation in slave applications should be disabled not to override position and orientation of nodes sent over the network from a master.

Configuration of slaves is set up to fit the configuration of monitors.

Launching Order#

The order of launching master and slave applications does not matter: you can launch several slaves, then the master, and then the rest of the slaves - synchronization will start automatically. This is achieved by having a buffer of object creation/deletion calls. Each time a new slave connects to the network, the master tells it, which objects are to be created or deleted.

Interpolation and Extrapolation#

Unigine Syncker uses Interpolated Snapshots (IS) to tackle with the problem of lost packets. It works by taking two old, but known positions and interpolating the object between them. It is accomplished by having a buffer of received positions and rotations, along with the time they represent. We usually take our current local time minus some predefined amount - interpolation period (40 ms by default), then go into our buffer, find the two indices which are just before and just after this time and interpolate.

If we don't have a received position and rotation for the time we're looking for, the extrapolation (guessing) is used. It also has a limited time - extrapolation period (200 ms by default). If the extrapolation period is over, but there are still no packets received and all objects will freeze.

In most cases this method provides a very accurate representation of the world to each of the slaves, as in general only already known positions of the remote objects are rendered and in rare cases the system will try to extrapolate (guess) where an object is. This, however, comes at a cost, as we always render 40 ms (interpolation period) behind current time, so that new packets have time to arrive with data.

Optimized Synchronization#

The scene contains static and dynamic objects, and "dynamic" doesn't mean that object's states are changed constantly. Taking this into account Syncker uses the following strategy:

for static objects synchronization packets are sent on their creation and deletion.
for dynamic objects the packets are sent only when their state has changed.

Moreover, Syncker offers you a flexible customization of the whole synchronization process, based on custom user TCP/UDP messages. Instead of sending the whole bunch of transformation data for objects that can only be rotated or sending huge amounts of transformation data for all parts of complex objects, when their positions can be determined using just a handful of simple parameters, you can send a single quaternion or a set of parameters.

All packets are compressed before sending (LZ4 or ZLIB). You can choose the desired algorithm and compression quality.

Thus, the excessive load on the network is minimized, while keeping performance high.

Two-Way Communication#

Slaves are not silent anymore. If a slave fails to receive a message via UDP protocol, it immediately reports to the master (via TCP) and requests to resend the message again. In this case the master sends the current state of all synchronized objects via UDP to perform update.

NoticeAll critical messages are sent via TCP protocol with guaranteed delivery, others are sent via UDP.

Moreover, a slave can control the master and other slaves (e.g. run the profiler on all computers or shutdown all applications) by sending the syncker_console command.

Multiple Cameras and Multi-Monitor Slaves#

Syncker allows to synchronize views from multiple cameras. There are two types of cameras:

Main master camera - a single camera that corresponds to the main viewer's position. The screen configuration determines viewports relative to this camera. Example: a camera in the plane's cockpit, corresponding the the pilot's point of view.
Auxiliary camera - an additional camera (static or dynamic), that can be set anywhere on the scene. You can have as many cameras of this type as necessary. Example: a ground-based surveillance camera or a thermal imaging camera mounted on the plane's wing.

By default, the main master camera is used. You can create auxiliary cameras and specify their viewports to be displayed by selected slaves. Such cameras will be synchronized automatically.

Syncker offers you an extreme flexibility of viewport configuration. You can use up to 6 monitors on a single slave, each having it own viewport assigned. For this purpose you will have to use the AppWall plugin.

Screen Configurations#

You can set up the desired screen / projection configuration for the Syncker in the screen/projection setup mode. To activate this mode use the syncker_setup console command from the master or a slave:

Source code

syncker_setup 1

Screen configuration for the Syncker has to meet the requirements.

Two types of configuration can be used:

A multi-monitor setup formed by displays, where each slave renders its viewport on the corresponding monitor (or several monitors).
Notice
Projection and modelview matrices are automatically calculated on the basis of the viewer's position.
The following picture shows the screen configuration for the multi-monitor setup, where each slave renders its viewport to a single monitor.

Screen Configuration (the left picture) for Multi-Monitor Setup (the right picture)
A multi-projector setup formed also by displays, where each slave renders its viewport via the corresponding projector as if a surface which the viewport was projected onto was a display.

A multi-projector setup. The screens are considered as displays.

Notice
Projection and modelview matrices are automatically calculated on the basis of the viewer's position.
A special case of the multi-projection setup is a CAVE (Cave Automatic Virtual Environment):

Projection Configuration for CAVE

Syncker Pipeline#

To connect computers and synchronize them, the following pipeline is used.

After being launched the master starts sending broadcast messages that it is available using a UDP protocol.
All slaves in LAN after being launched listen to the selected UDP port. Upon receiving this message, they connect to a master via a TCP protocol.
Notice
Actually 2 TCP ports are used:
- TCP port - for messages.
- TCP ping port - for latency measurement (to ensure correct interpolation).
After the connection was established, both UDP and TCP protocols are used to exchange data between the master and slaves.
- Over UDP protocol non-critical messages are sent. This includes messages on render parameters (for example, ambient color or HDR exposure), position of a player and all nodes and materials that are synchronized. It also includes frame-related information (if the game is enabled, the current frame number, frame duration, a time scale and even the game seed for random number generators).
- Over TCP protocol critical messages are sent. These are messages on creation and deletion of nodes and user messages.

Using Syncker#

The basic workflow is as follows:

Implement Syncker logic for your master and slave application (you can use the same application on both master and slave sides).
Set up screen / projection configuration.
Prepare your environment.

It is recommended to use a 100 Mb LAN. Otherwise, you may experience network lags (see Troubleshooting section).

All applications you use must have access to all Unigine files and project data. So, you should copy your project to all computers. If some nodes are missing in the world file on a local computer, they will not be rendered.
Run the master application.

To run a master application you should substitute the System Runtime Script with the special one: core/systems/syncker/unigine.cpp and provide necessary startup command-line options, e.g.:
Shell commands
```
<your_app_name> -system_script core/systems/syncker/unigine.cpp -extern_plugin "Syncker" -sync_master 1 -sync_broadcast_address xxx.xxx.xxx.xxx -sync_view_config <config_name>.sync -sync_view <display_name>
```
Notice
The order of launching master and slave applications does not matter.
Run a slave application.
To run a slave application you should also substitute the System Runtime Script with the special one: core/systems/syncker/unigine.cpp and provide necessary startup command-line options, e.g.:
Shell commands
```
<your_app_name> -system_script core/systems/syncker/unigine.cpp -extern_plugin "Syncker" -sync_master 0 -sync_slave_name <slave_name> -sync_view_config <config_name>.sync -sync_view <display_name>
```
If you want a slave with a multi-monitor configuration to render several viewports, you should use the AppWall plugin and assign viewports to monitors via the sync_view_n options, e.g.:
Shell commands
```
<your_app_name> -system_script core/systems/syncker/unigine.cpp -extern_plugin "AppWall,Syncker" -sync_master 0 -sync_slave_name <slave_name> -sync_view_config <config_name>.sync -sync_view_0 <display_0> -sync_view_1 <display_1>
```
Notice
The order of plugins in the list matters: AppWall must be specified before Syncker.

Debug Window#

Syncker's debug window enables you to monitor the hierarchy of objects at run time. The whole hierarchy means all objects (not only the ones registered in the Editor) including objects in cache, internal structure of node references, etc. When a node is selected, all necessary debug information is displayed. This information is updated each frame.

Search option allows you to find any node by its name or ID (on the Master or on a Slave), so a faulty node can be found quickly.

To show the debug window, run the syncker_debug console command from the master or a slave as follows:

Source code

syncker_debug 1

The debug window can also be opened via the System Menu:

Open the System Menu by pressing Esc.
Go to the Plugins tab and enable the Show debug window option in the Syncker group:

The debug window will open:

Debug Window

Troubleshooting#

If the network latency is too large despite 1Gb bandwidth or higher, it can be caused by a 100 Mb or 10 Mb device connected to a network. Data exchange rate will drop down to the maximum rate supported by such device, slowing down Syncker connection speed.

Some 100 Mb or 10 Mb devices can have a working network interface when they are turned off.
It is also possible that when turned off, 1 Gb devices have a network interface working at 100 Mb rate, which will slow down connection in LAN.

Useful Tool#

If you have a source SDK, you can use a simple and useful tool to monitor the network messages exchange speed. It is server.usc found in source/tools/Interpreter/scripts/network/.

Last update: 27.12.2018

Помогите сделать статью лучше

Была ли эта статья полезной?

Сообщить о проблеме в этой статье

(или выберите слово/фразу и нажмите Ctrl+Enter