This page has been translated automatically.
Programming
Fundamentals
Setting Up Development Environment
Usage Examples
UnigineScript
C++
C#
UUSL (Unified UNIGINE Shader Language)
File Formats
Rebuilding the Engine and Tools
GUI
Double Precision Coordinates
API
Containers
Common Functionality
Controls-Related Classes
Engine-Related Classes
Filesystem Functionality
GUI-Related Classes
Math Functionality
Node-Related Classes
Networking Functionality
Pathfinding-Related Classes
Physics-Related Classes
Plugins-Related Classes
CIGI Client Plugin
Rendering-Related Classes
Warning! This version of documentation is OUTDATED, as it describes an older SDK version! Please switch to the documentation for the latest SDK version.
Warning! This version of documentation describes an old SDK version which is no longer supported! Please upgrade to the latest SDK version.

Distributed Computing

Terrain generation is a complex task and may require a substantial amount of time depending on the area size and data resolution. The Landscape Tool enables you to use the advantages of distributed computing in the process of terrain generation.

Distributed computing is available for the following tasks:

  • raster cache generation (elevation, imagery, masks)
  • procedural cache generation
  • procedural cache blending
  • procedural tileset updating (heights,masks)
  • tileset generation for all LODs
  • detail masks generation
  • landcover objects generation

Notice
  • If distributed computing is to be used for procedural refinement, you should have the Houdini Engine version 16.0.600 or higher installed on all computers.
  • All workstations performing computations for a certain terrain generation job must be running on the same operating system (Windows or Linux).

All calculations to be performed in the framework of the terrain generation can be distributed between different computers of your LAN. These computers can be of two types:

Master

Master computer is the one, that manages the whole generation process, forms the task pool and distributes tasks between the Workers. The Master tracks all connected Workers and assigns tasks to them in accordance with their performance. The Master itself performs calculations.

Notice
You can use several Masters in your network to generate several terrains.
Worker

Worker is a computer connected over the network to the Master and performing certain tasks assigned by the Master.

Notice
The number of Workers connected to a single Master is not limited (as long as network bandwidth allows it).

Launching Order

The order of launching Master and Workers matters. You should Activate Workers before configuring the Master. As a Worker is launched is starts sending broadcast messages that it is available. So, when configuring the Master you can choose Workers from the list of available ones.

Setting Up the Master

To set up a Master, open the Distributed Computing settings window by choosing Distributed Computing -> Settings in the Landscape Tool window menu. The following window will appear:

Select the Master mode.

In the Required common paths field you will find the list of common shared folders, that will be used by all computers in the process of terrain generation.

Notice
Make sure all these folders are accessible (for reading and writing) from the network.

In the Workers list you will see all Workers in you network with their ports and statuses.

Select desired Workers in the list one by one and Connect for each of them. If the connection was successful the Worker's status will be changed to Connected. Workers, that are occupied calculating for another Master will have the Busy status.

To disconnect a connected Worker, select it in the list and click Disconnect.

As soon as you are done with settings and connections, click OK.

Setting Up a Worker

To set up a Worker, open the Distributed Computing settings window by choosing Distributed Computing -> Settings in the Landscape Tool window menu.

In the window that appears select the Worker mode. The following settings will be displayed:

Here you can specify the port to be used by the Worker in the corresponding field. You can set different ports for different Workers.

After setting a port, click Activate Worker. If the Worker is activated successfully, you will see the following Status window:

Now the Worker is switched to online mode and ready to perform calculations for the Master. This window will remain opened during the whole generation process.

Notice
If you click Disconnect the worker will go offline and all results will be lost. In this case the Master shall redistribute all the tasks, that were assigned to this Worker, between the others.

Using Distributed Computing

The basic workflow is as follows:

  1. Prepare your environment.

    Connect all computers to the network. It is recommended to use network with at least 1 Gb bandwidth. Otherwise, you may experience network lags (see Troubleshooting section). Broadcasting should not be blocked in your network, otherwise distributed computation will be unavailable.

    Notice
    It is recommended to use an SSD drive to store the generation cache and output files. Otherwise, the generation performance will be significantly lower, due to multiple read/write and head positioning operations, in case if an HDD is used.
  2. On each computer run the UnigineEditor and open the Landscape Tool via the corresponding menu option Tools -> Landscape.
    Notice
    The versions of binary executable files must be the same on all computers.
  3. On each Worker computer set up a Worker and activate it.
    Notice
    It is not necessary to open your world or the landscape asset on a Worker computer.
  4. On the Master computer perform the following steps:
    • Open your world in the UnigineEditor.
    • Open the *.landscape asset, containing all necessary settings via File -> Open Asset, or specify all necessary generation settings for your terrain, if you do not have a *.landscape asset file yet.
    • Specify Master's settings.
    • Now you can start generation process by clicking Generate in the Landscape Tool window and specifying required generation steps.
      Notice
      Any unsaved changes made to generation parameters must be saved to the *.landscape asset before you start the generation process. You will be prompted to save your changes if any.
    • In the process of generation you will see the list of connected workers in the progress window. You can disconnect Workers in the process of generation when necessary by clicking remove.

      Notice
      • In case of a repeated generation (e.g., due to failed previous attempt) using a procedural data source, it is required to restart the engine on all Workers.
      • In case if a Worker fails to perform certain tasks due to a system crash or a network failure, the Master shall redistribute all the tasks, that were assigned to this Worker, between the others.

Troubleshooting

If the network latency is too large despite 1Gb bandwidth or higher, it can be caused by a 100 Mb or 10 Mb device connected to a network. Data exchange rate will drop down to the maximum rate supported by such device, slowing down generation speed.

  • Some 100 Mb or 10 Mb devices can have a working network interface when they are turned off.
  • It is also possible that when turned off, 1 Gb devices have a network interface working at 100 Mb rate, which will slow down connection in LAN.

Useful Tool

If you have a source SDK, you can use a simple and useful tool to monitor the network messages exchange speed. It is server.usc found in source/tools/Interpreter/scripts/network/.

Last update: 2018-02-13
Build: ()