shown in other instead.
Terrain generation is a complex task and may require substantial time depending on the area size and data resolution. Sandworm allows using distributed computing to generate terrain.
The concept is to use several computers united into a network, one of which is Master and assigns tasks, and the other are Workers that perform portions of work (processing of source data) assigned to them. UnigineEditor with the Sandworm tool and a terrain project should be open on Master, while Workers run the console application. Source files (geodata) and common cache shall be stored in a shared folder. If a Worker still has available resources, additional processes may be run on it and participate in terrain generation immediately, i.e., without restarting the generation process. Master creates the asset based on the result.
Participants of the process:
Before launching any of the network participants, disable Microprofile for it — in \data\configs\default.user, set:
Otherwise, every instance would run Microprofile, causing memory waste.
You can launch Master and Workers in any sequence.
You can run additional Workers after the terrain generation process has started, and Master will distribute jobs for them.
One machine can have several Workers running. A Worker can have several processes (forks) that are activated via the menu on Master.
Using Distributed Generation#
Prepare your environment.
Connect all computers to the network. It is recommended to use a network with at least 1 Gb bandwidth. Otherwise, you may experience network lags (see the Troubleshooting section). Check that your network doesn't block broadcasting, otherwise distributed computation will be unavailable.It is recommended to use an SSD drive to store the generation cache and output files. If an HDD is used, the generation performance is significantly lower due to multiple read/write and head positioning operations.
On each Worker computer, run the Worker console application.The versions of binary executable files must be the same on all computers.
On the Master computer:
- Run UnigineEditor and open the Sandworm tool (Tools -> Sandworm).
- Open the *.sworm asset you are going to generate and configure the generation settings.
- Enable the Distributed option and specify the Master's settings. Save the project configuration.
Click the Generate button. The window with the list of available Workers will open:If you regenerate the project, this window will appear after the warning on losing any manual modifications.
As soon as this window is open, Workers start parsing the shared source data and generating cache. In the end of this process, the console message Worker: Ready To Import is displayed.
Clicking the right mouse button on a Worker makes the following options available:Worker Options
Create Fork Creates one more process on the same Worker.We estimate a reasonable number of forks as 4 to 6. Higher values reduce the Worker's performance. Shutdown Removes the fork or Worker from the list. Clear Cache Clears the Worker's local cache.
Set the number of forks you need and click Generate.
During the generation process, you can connect more Workers, if necessary: run the console application on a Worker (even if this Worker is already running one or several). This console application will automatically connect to Master, and Master will assign a process to this Worker.
- When the terrain generation is completed, disable all Worker processes.
Setting Up the Master#
To set up Master, enable Distributed Generation in Generation Settings: toggle on the Enabled option.
|Enabled||If toggled on, distributed computing is enabled for the terrain generation, and the machine running the Editor with Sandworm is Master.|
|Broadcast Port||Port for listening to Sandworm by Workers.|
|Server Port||Port used for information exchange.|
|Workload on Master||Percentage of terrain generation performed by Master.|
Setting Up a Worker#
To set up a Worker:
Run the SandwormNode_x64.exe console application stored in the bin/ folder of your project. The message that Worker is ready should appear in console:
- Set the path for storing the local cache using the --sw_local_cache_path console command.
- Use --sw_broadcast_port, if you want to change the default broadcast port.
- Worker detects Master on the network, connects to it, downloads the scene, prepares all layers (checks their accessibility), and is ready for terrain generation.
You can run multiple console applications on one machine and/or fork one Worker via Master to create more processes.
- Operating System versions on Master and Workers should be identical.
- The SDK version of the Editor run on Master should be the same as the version of the console app (including precision).
- Development/Release version of SDK does not matter.
When the terrain generation is completed and you don't need Workers anymore, close the console application on Workers or shut down processes via Master.
If the network latency is too large despite 1Gb bandwidth or higher, this can be caused by a 100 Mb or 10 Mb device connected to the network. Data exchange rate drops down to the maximum rate supported by such device, slowing down the generation speed.
- Some 100 Mb or 10 Mb devices can have a working network interface when they are turned off.
- Some 1 Gb devices, when turned off, have a network interface working at 100 Mb rate, which slows down connection on LAN.
- Check that your projects on Master and Workers use the same precision (double or float), the same engine versions, and the same GDAL versions, otherwise the Workers would display the Different Builds error.
- Use SSD to store shared source data and cache.
- Add the application to the antivirus exclusions, as it may reduce performance.
- Check that network ports used by Master and Workers are allowed by the firewall.