UnigineEditor
Interface Overview
Assets Workflow
Settings and Preferences
Adjusting Node Parameters
Setting Up Materials
Setting Up Properties
Using Editor Tools for Specific Tasks
FAQ
Programming
Fundamentals
Setting Up Development Environment
Usage Examples
UnigineScript
C++
C#
UUSL (Unified UNIGINE Shader Language)
File Formats
Rebuilding the Engine and Tools
GUI
Double Precision Coordinates
API
Containers
Common Functionality
Controls-Related Classes
Engine-Related Classes
Filesystem Functionality
GUI-Related Classes
Math Functionality
Node-Related Classes
Objects-Related Classes
Networking Functionality
Pathfinding-Related Classes
Physics-Related Classes
Plugins-Related Classes
CIGI Client Plugin
Rendering-Related Classes

Distributed Computing

Terrain generation is a complex task and may require a substantial amount of time depending on the area size and data resolution. The Landscape Tool enables you to use the advantages of distributed computing in the process of terrain generation.

Notice
Distributed computing makes sense when used for speeding up terrain generation that takes more than an hour. Applying distributed computing to a small terrain area with low-detail data (10 meters or more per pixel) may be unprofitable at all.

Distributed computing is available for the following tasks:

  • Raster cache generation (elevation, imagery, masks)
  • Procedural cache generation
  • Procedural cache blending
  • Procedural tileset updating (heights, masks)
  • Tileset generation for all LODs
  • Detail masks generation
  • Landcover objects generation

Notice
  • If distributed computing is to be used for procedural refinement, you should have the Houdini Engine version 16.0.600 or higher installed on all computers.
  • All workstations performing computations for a certain terrain generation job must be running on the same operating system (Windows or Linux).

All calculations to be performed in the framework of the terrain generation can be distributed between different computers on your LAN. These computers can be of two types:

Master

Master computer is the computer that manages the whole generation process, forms the task pool, and distributes tasks between the Workers. The Master tracks all connected Workers and assigns tasks to them in accordance with their performance. The Master itself performs calculations.

Notice
You can use several Masters in your network to generate several terrains.
Worker

Worker is a computer connected over the network to the Master and performing certain tasks assigned by the Master.

Notice
The number of Workers connected to one Master is not limited (as long as the network bandwidth allows that).

Launching Order#

The order of launching Master and Workers matters. You should activate Workers before configuring the Master. As soon as a Worker is launched, it starts sending broadcast messages that it is available. So, when configuring the Master, you can choose Workers from the list of available ones.

Setting Up the Master#

To set up a Master, open the Distributed Computing settings window by choosing Distributed Computing -> Settings in the Landscape Tool window menu. The following window appears:

Select the Master mode.

The Required common paths field contains the list of common shared folders that are used by all computers in the process of terrain generation.

Notice
In case of distributed computing, all output paths must be set to network shares (not local paths). Make sure all these folders are accessible (for reading and writing) from the network.

In the Workers list you will see all Workers in you network with their ports and statuses.

Select desired Workers in the list one by one and click Connect for each of them. If the connection is successful, the Worker's status is changed to Connected. Workers that are occupied calculating for another Master have the Busy status.

To disconnect a connected Worker, select it in the list and click Disconnect.

As soon as you are done with settings and connections, click OK.

Setting Up a Worker#

To set up a Worker, open the Distributed Computing settings window by choosing Distributed Computing -> Settings in the Landscape Tool window menu.

In the window that appears, select the Worker mode. The following settings will be displayed:

Here you can specify the port to be used by the Worker in the corresponding field. You can set different ports for different Workers.

After setting a port, click Activate Worker. If the Worker is activated successfully, the following Status window is displayed:

Now the Worker is switched to online mode and ready to perform calculations for the Master. This window remains opened during the whole generation process.

Notice
If you click Disconnect, the Worker goes offline and all results are lost. In this case the Master redistributes all tasks assigned to this Worker between the other Workers.

Using Distributed Computing#

The basic workflow is as follows:

  1. Prepare your environment.

    Connect all computers to the network. It is recommended to use a network with at least 1 Gb bandwidth. Otherwise, you may experience network lags (see Troubleshooting section). Broadcasting should not be blocked on your network, otherwise distributed computation would be unavailable.

    Notice
    It is recommended to use an SSD drive to store the generation cache and output files. If an HDD is used, the generation performance is significantly lower due to multiple read/write and head positioning operations.
  2. On each computer, run UnigineEditor and open the Landscape Tool via the corresponding menu option Tools -> Landscape.
    Notice
    The versions of binary executable files must be the same on all computers.
  3. On each Worker computer, set up a Worker and activate it.
    Notice
    It is not necessary to open your world or the landscape asset on a Worker computer.
  4. On the Master computer, perform the following steps:
    • Open your world in UnigineEditor.
    • Open the *.landscape asset containing all necessary settings via File -> Open Asset or specify all necessary generation settings for your terrain, if you don't have a *.landscape asset file yet.
    • Specify the Master's settings.
    • Now you can start generation process by clicking Generate in the Landscape Tool window and selecting the required generation steps.
      Notice
      Any changes made to the generation parameters must be saved to the *.landscape asset before you start the generation process. You will be prompted to save your changes, if any.
    • During the generation process, the list of connected workers is displayed in the progress window. You can disconnect Workers during generation, if necessary, by clicking remove.

      Notice
      • In case of a repeated generation (e.g., due to a failed previous attempt) using a procedural data source, it is required to restart the engine on all Workers.
      • If a Worker fails to perform certain tasks due to a system crash or network failure, the Master shall redistribute all tasks assigned to this Worker between the other Workers.

Troubleshooting#

If the network latency is too large despite 1Gb bandwidth or higher, this can be caused by a 100 Mb or 10 Mb device connected to the network. Data exchange rate drops down to the maximum rate supported by such device, slowing down the generation speed.

  • Some 100 Mb or 10 Mb devices can have a working network interface when they are turned off.
  • Some 1 Gb devices, when turned off, have a network interface working at 100 Mb rate, which slows down connection on LAN.

Useful Tool#

If you have a source SDK, you can use a simple and useful tool server.usc stored in source/tools/Interpreter/scripts/network/ to monitor the network messages exchange speed.

Last update: 2018-12-17