UnigineEditor
Interface Overview
Assets Workflow
Settings and Preferences
Adjusting Node Parameters
Setting Up Materials
Setting Up Properties
Landscape Tool
Using Editor Tools for Specific Tasks
FAQ
Programming
Fundamentals
Setting Up Development Environment
Usage Examples
UnigineScript
C++
C#
UUSL (Unified UNIGINE Shader Language)
File Formats
Rebuilding the Engine and Tools
GUI
Double Precision Coordinates
API
Containers
Common Functionality
Controls-Related Classes
Engine-Related Classes
Filesystem Functionality
GUI-Related Classes
Math Functionality
Node-Related Classes
Networking Functionality
Pathfinding-Related Classes
Physics-Related Classes
Plugins-Related Classes
CIGI Client Plugin
Rendering-Related Classes

File System

The UNIGINE engine has its own file system module used to access files and folders. It has a few peculiarities you should be aware of, when loading resources or organizing the structure of your UNIGINE-based project.

GUIDs

In the UNIGINE file system, each file has a GUID (Globally Unique Identifier), which defines a virtual path to this file (not a file on a disk). Using GUIDs provides more flexible file management: you can abstract from file names (which can be the same in different folders of the project). For example, you can change a path to the file while keeping the same GUID.

The Engine generates GUIDs for all files of the virtual file system.

Files of the UNIGINE file system can be accessed by using both names or GUIDs: you can obtain a GUID for the specific file, change the file GUID, get the file name by its GUID, add/remove the file with the specific name or GUID to/from a blob or a cached file, and so on.

Notice
To manage GUIDs via API use the UGUID class.

If the UnigineEditor is loaded, it will save the generated GUIDs to the data/guids.db file automatically. Otherwise, you can implement logic of updating guids.db via the code. The guids.db file stores a pair: a GUID of a file and a path to this file relative to the data folder. It also includes GUIDs of all files stored in additional directories.

Deleting the guids.db file will cause all references by GUIDs to be invalid. For example, if a node refers to a *.mesh by a GUID stored in the guids.db file, the reference become invalid, as GUIDs of all runtime files will be re-generated.

File System Update

Dynamic Scanning vs Pre-Cached

Dynamic scanning allows the Engine to form a virtual file system for all files within the data folder. Dynamic scanning is performed on the Engine start-up. It enables tracking file changes in real-time and using partial and relative paths to address files.

If the dynamic scanning is not necessary or takes too many resources, you can use the pre-cached file hierarchy that is specified in the .ulist file. When the Engine finds this file in a folder during the start-up scanning, it stops and uses the list of files specified there to form the virtual file system.

The .ulist file is generated by using the special script <UnigineSDK>/utils/datalist.py that should be run from the data folder.

Notice
You need to re-generate .ulist files in case you modify contents of folders.
If changes to the file system are made not by means of UnigineScript, you may need to call the filesystem_reload console command.

Automatic Resource Reloading

When the UnigineEditor is loaded, the Engine tracks changes made in files at run time: it checks the time of the last modification of such files and updates them in the memory. If the UnigineEditor isn't loaded, the changed files will be reloaded after reloading the world.

Notice
However, it is possible only if the files are added to the virtual file system (dynamic scanning has been performed on the start-up or the file has been specified in the *.ulist file).

Known vs Unknown Files

If you add new files at run time, the Engine won't know anything about such files (as the virtual file system has been formed on the start-up). Re-scanning the file system is resource-consuming, so, in this case, you can add new files to the virtual file system via API by using addKnownFile().

Data Directory

All files that are used by the Engine at run time are stored in the data folder specified via the -data_path start-up option. By default, it is the data folder created automatically on project creation via UNIGINE SDK Browser.

Notice
The path to the data directory can be specified relatively to the binary executable or as an absolute path.
When the file system resolves paths, it tries to concatenate data paths with the specified path and perform a lookup. In case of absolute paths, the file system will use them as-is without any checks.
Notice
When the path to the data directory is relative, the engine switches the current directory to the directory with the binary executable.
For example, if the project folder has the following structure:
  • unigine_project
    • assets
    • bin
    • data
The data path will be unigine_project/data after the application start-up:
Shell commands
bin\main_x64d.exe -data_path "../"

If the -data_path isn't specified (neither in the command-line nor the configuration file), the default ../ path will be used. If multiple -data_path are specified, only the first one will be treat as the path to the data folder. Moreover:

  • The engine will check only the first -data_path when searching for the configuration file.
  • The relative path specified in the -gui_path will be taken relatively to the first -data_path.
  • If the -project_name isn't specified, the *.cache files will be saved to the first specified -data_path.
  • All runtime files generated by the Editor for files stored in the assets folder are stored in the folder specified in the first -data_path.
  • When referring to the file created after initialization of the engine file system, the file system will check only the first specified -data_path directory.
  • When multiple data paths are specified, the latest path has the top loading priority when resolving relative paths.
Notice
A path specified in the -data_path option won't be written into the configuration file.
Writing to files stored in all -data_path folders is allowed.

Current Directory

When the first specified -data_path is absolute, the current working directory may differ from the directory with the binary executable. However, when the path to the data directory is relative, the engine switches the current directory to the one with the binary executable.

When accessing a file outside the data directory (which is also not specified in an additional -data_path) via API, the path to such file should be specified relative to the current directory. For example:

Source code (C++)
// cbox.mesh is stored outside the data directory, so the path is specified relative to the current directory
ObjectMeshStatic cbox = new ObjectMeshStatic("../../data/cbox.mesh");

File Packages

Types

UNIGINE supports the following types of file archives to save space or pack the production version of resources:

  • UNG (a UNIGINE-native format for archives created with Archiver tool)
    Notice
    The maximum size for a file inside a UNG archive is limited to 2 GB.
  • ZIP
  • Custom C++ packages created via UNIGINE API
Besides saving space, archives also speed up resource loading, as files in an archive are read linearly.

UNG and ZIP archives are loaded automatically if they are found within the data folder. Files are added to the virtual file system just like in case with non-archived files.

Content Access

Archives are completely transparent to the engine. There is no need to explicitly unpack the archives, as their content is automatically handled as not packed. Archived files are addressed as if they are non-archived. For example, if you have data/project/archive.ung and want to address directory/file.txt within it, simply specify the following path: project/directory/file.txt or even file.txt (only if the file name is unique).

Inside the archive, files can be organized in any way. However, in the root of the archive only files with unique names should be placed. Otherwise, the file search will return incorrect results.

Notice
Using GUIDs allows you to avoid the need to control uniqueness of names when working with archived files.

Here is an example of an incorrect file tree for an archive:

  • my_archive.ung
    • my_folder
      • file_2.txt
    • file_1.txt
    • file_2.txt
In this case, there is no problem with file_1.txt, since its name is unique. file_2.txt, on the other hand, will cause problems. Even if you specify a directory in the path name (the full one: my_folder/file_2.txt or a partial one: older/file_2.txt), it does not guarantee that a non-root file will be returned.

The correct archive structure can be specified as follows:

  • my_archive.ung
    • my_folder
      • file_2.txt
    • another_folder
      • file_2.txt
    • file_1.txt
In this case, you can specify directories in the file search and the result will be perfectly correct. For example, full ones: my_folder/file_2.txt, another_folder/file_2.txt or partial ones: another_folder/file_2.txt.

If there is a name collision between an archived file and a non-archived one, the first matching file is returned. The search is performed in the following order:

  1. Unarchived files
  2. Files in UNG archives
  3. Files in ZIP archives

From UNIGINE API, archives are handled using the FilesSystem functions as well.

Extending File System

To extend the virtual file system, 3 ways are possible:

Notice
GUIDs are generated by the Engine for all external files.
When the UnigineEditor is loaded, GUIDs for the external files are automatically written to the guids.db file. Also you can implement logic of updating guids.db via API.

Multiple Data Paths

For the last variant, use the additional -data_path command-line options as follows:

Shell commands
-data_path "../" -data_path "D:/resources/my_project/test_0/" -data_path "../../resources/my_project/test_1/"

Paths can be both relative to the binary executable or absolute.

Notice
Remember that only the first -data_path option specified on the application start-up will be treated as the path to the data folder.

The list of all data paths specified on the application start-up can be obtained via API by using the following methods:

The getDataPath() function without arguments will return the first specified-data_path directory.

ULINK

*.ulink files can be used to add a folder (together with all its sub-folders and archives, if any) from outside the data directory to the virtual file system.

In such files, a path to the directory is stored. A link file can be stored anywhere inside data. On the start-up or when filesystem_reload is called to update the list of resources, the engine scans through files, finds a link file and adds a linked directory as if its root is a data folder.

Paths inside *.ulink files can be relative or absolute.

  • Relative to the data directory:
    Source code
    ../../my_project/resources/
  • An absolute path can be specified.
    For example, on Windows:
    Source code
    D:\resources\
    On Linux:
    Source code
    /username/resources/

External Packages

It is possible to add archives to the virtual file system even if they are stored outside the data folder. External packages are added only on the start-up. Use one of the following methods for that:

  • Add archives on the start-up via -extern_package CLI option (together with the rest of required ones). Relative and absolute paths can be used.
    Shell commands
    bin\main_x86d.exe -extern_package "../../my_project/archive.ung"
    Several packages:
    Shell commands
    bin\main_x64d.exe -extern_package "../2/core.ung" -extern_package "D:/Unigine/core.ung"
  • Using UNIGINE API, via loadPackage() function.
Notice
The path can be specified relatively to the binary executable or as an absolute path.

As well as the data paths, the paths to external packages won't be saved into the configuration file. However, they can be added to it manually.

Paths

The engine accepts both the relative and absolute paths. Moreover, the files stored in the data folder can be accessed by using partial path.

Notice
The UNIGINE file system features path normalization: it automatically converts Windows paths to UNIX paths.

Partial

Partial paths simplify handling of files. If you want to load a file and you are sure that there is only one resource with the given name in the data folder, only a file name can be provided without a path.

Notice
If a name is not unique, the first found file with such name is loaded.
It is also possible to provide a sub-path that uniquely specifies a file. For example, to load data/project/image_1.tga, you can use image_1.tga (if a name is unique) or project/image_1.tga.

Also you can refer files by GUIDs or use the strict flag to uniquely specify a file.

Strict

Sometimes, it is required to know whether the file with the specified name exists in the exact directory, not just somewhere inside the data. Setting the strict flag allows you to tell the file system to check the exact file location (i.e., the partial path won't be resolved if the strict flag is set).

This flag is used in most of the FileSystem methods.

Relative vs Absolute

When relative paths are used, you can relocate your Unigine-based application or copy it onto another machine, and all resources will be properly loaded. There is no loading speed penalty as well: it is as fast as loading files by an absolute path due to the virtual file system.

Notice
It is possible to use absolute paths to load resources outside data folder, but such project will not be be portable.

As file names are added to the virtual file system, usually the same name and path should be used to load and remove file when accessing from your source code by using FileSystem functions:

  • For default resources, functions return full paths relative to the data folder.
  • If you load a file and specify a relative path, use a relative path to delete the resource.
  • If you load a file using an absolute path, use an absolute path to delete the resource.

Notice
Using GUIDs allows you to avoid the need to control paths when working with files.

You can check whether the path is absolute or relative via the isAbsolute() function.

Also the file system allows you to get a path to a file relative to the data folder by using the getRelativeFileName() function.

Loading Priorities

When the Engine resolves a virtual path, it performs it as follows:

  1. It tries to resolve the path as if it is absolute.
  2. It tries to resolve the path as if it is relative to the -data_path. If multiple data paths are specified, the latest one has the top priority.
  3. It tries to resolve the path as if it is relative to the binary executable.

A virtual path can represent up to four entities at the same time: it can be a file on the disk, a file stored in a package, a file added to a cache and to a blob.

Notice
A GUID is generated for a virtual path: neither a file on the disk, a package file, a cached or a blobbed file can have a GUID.
For example, the textures/white.dds path has only one GUID, however, it can represent the following at the same time:
  • Within the project folder, you can have both core/textures/white.dds and core.ung/textures/white.dds.
  • In addition, in the code, you can have both:
    Source code (C++)
    // the file loaded into a cache
    FileSystem::get()->addCacheFile("core/textures/white.dds");
    // the file loaded into a blob
    FileSystem::get()->addBlobFile("core/textures/white.dds");
Notice
You can check whether the virtual path or the GUID represent a certain entity via the corresponding methods of API.

During read / write operations, the Engine will load the first found entity for such virtual path. The entities will be checked in the following order:

  • For read operations:
    1. The file loaded into a blob.
    2. The file added into a cache.
    3. The read-only file stored on the disk.
    4. The file stored in a package.
  • For write operations:
    1. The file loaded into a blob.
    2. The file stored on the disk.
    Notice
    The cached and packed files aren't checked as write operations aren't allowed for them.

Accessing Assets and Runtime Files

Working with assets via the Editor is clear and simple, but in order to access your project's files properly you should have a clear understanding of the concepts of asset and runtime files.

Generated runtime files have constant GUIDs and are named as follows:
<GUID>.<extension> (e.g., ab23be4cd7832478edaaab23be4cd7832478edaa.dds).

These files are stored in subfolders of the data/.runtimes folder. The structure or this folder is optimized for the file system. A runtime file generated for a non-native asset with a certain GUID will be placed in a folder, that has a name equal to the first two bytes of this GUID.

Notice
A sub-folder of the data/.runtimes folder may contain runtime files generated for different assets (if these assets have matching first two bytes in their GUIDs).

E.g., your non-native asset data/my_textures/1.tga (GUID = "aeb53b44cdbbbbbbbbaaabccc1c1c1c1c1c1c1c1") will have runtime file generated for it in a folder: ./runtimes/ae/ab23be4cd7832478edaaab23be4cd7832478edaa.dds

Therefore each runtime file has an alias - a human-readable form of a path used to refer to this file.

Full aliases are constructed as follows: <source_asset_path>/<runtime_alias>

E.g.:

  • 1.tga/1.dds
  • 1.fbx/material/1.mat

In order to simplify access to runtime files, we also use a concept of the primary runtime - a runtime file uniquely assoсiated with the asset. It acts like an implied reference to a runtime file: when we say "model.fbx", we actually mean "model.node". So, that we could write:

Source code (C++)
NodeReferencePtr node = NodeReference::create("model.fbx");
Notice
Primary runtime is actually used instead of an non-native asset, when you refer to it by path.

There are two ways you can access your assets and runtime files:

The file system includes a subsystem for managing assets and runtime files. This subsystem is implemented as a separate class named FileSystemAssets.

You can use assets_info and assets_list console commands to view infomation on non-native assets and runtimes generated for them.

Accessing by Path

The way of accessing a certain asset by path is determined by its type:

  • Native assets are accessed simply by their name:
    Source code (C++)
    ImagePtr image = Image::create("image.dds");
  • All non-native assets have a primary runtime file. So, when you refer to the asset by its name, this primary runtime file will be actually used. For example, if you specify:

    Source code (C++)
    ImagePtr image = Image::create("image.png");
    The image.dds generated primary runtime file will actually be used.

    You can also directly access any asset source file (not a runtime file). For example, if you need to specify a .png texture, you should write the following:

    Source code (C++)
    ImagePtr image = Image::create("asset://image.png");
    In this case, the runtime .dds file will be ignored, .png source file will be used.
  • Each container asset also has a primary runtime, in case of an FBX asset it is a generated .node file. So, you can use the following reference:

    Source code (C++)
    NodeReferencePtr node = NodeReference::create("teapot.fbx");
    The teapot.node generated runtime file will be used in this case.

    You can access each runtime file of a container asset. For example, an FBX file has .node and .mesh runtime files generated for it. You can access the generated .mesh file in the following way:

    Source code (C++)
    MeshPtr mesh = Mesh::create("teapot.fbx/teapot.mesh");

Accessing by GUID

You can also access any runtime or asset in your project using GUIDs. In case if a GUID of a file is specified, the exact path corresponging to this GUID will be used:

Source code (C++)
UGUID asset_guid; // GUID of the asset named "1.tga"
const char *asset_path = "1.tga";

ImagePtr image = Image::create(asset_guid); // -> 1.tga
ImagePtr image = Image::create(asset_path); // -> 1.dds

Modifiers

File modifiers serve to automatically choose what resources to load when a UNIGINE project is run on different platforms or with different localizations. Instead of keeping multiple versions of the same project and copying shared data between them, you can add a custom postfix to file or folder names, and load only required resources on demand.

Modifiers are added to file or folder names as a postfix (only one can be specified). Any custom postfix can be used. For example, it could be:

  • File name modifier: file.small.node ortexture.eng.dds
  • Folder name modifier: textures.lowres
    Notice
    If a folder has a modifier, files inside of it should not have modifiers. Otherwise, their modifiers will be ignored.

Register necessary modifiers in code via addModifier(). When the project running, resources with the registered modifiers will be automatically loaded. Files without modifiers have the lowest priority (can be used for default resources).

Usage Example

For example, three localization languages are supported in the project: English (by default), German and French. Depending on the language, different splash textures need to be loaded on the start-up.

To organize your resources, name them using the following file modifiers:

  • data
    • splashes
      • splash.png (this would be a default version of the texture. In our case, a texture with an English title)
      • splash.de.png (a German title)
      • splash.fr.png (a French title)

After that, in the code you need to specify what modifier to use via engine.filesystem.addModifier(). This function is called in the system script (unigine.cpp) since a modifier need to be registered before the world and its resources start to be loaded. For example, to load a German splash screen and the low resolution textures interface:

Source code (UnigineScript)
// unigine.cpp

int init() {
	...
	
	// Register modifier
	engine.filesystem.addModifier("de");
	
	// Set a splash texture
	engine.splash.setWorld("textures/splash.png");   // splash.de.png will be automatically used
	...
	
	return 1;
}

Also you can use -extern_define CLI option to pass the language (for example, if a user chooses a language in the launcher).

Shell commands
bin\main_x64d.exe -extern_define "LANG_DE"

And here is how passed defines can be handled in the code.

Source code (UnigineScript)
// unigine.cpp
string lang = "";

int init() {
	...
	
	// Parse EXTERN_DEFINE
	#ifdef LANG_DE
		lang = "de";
	#elif LANG_FR
		lang = "fr";
	#endif
	
	if(lang != "") {
		engine.filesystem.addModifier(lang);
	}
	
	// Set a splash texture: splash.de.png or splash.fr.png will be used if the language is passed
	engine.splash.setWorld("textures/splash.png");	// otherwise, splash.png
	...

	return 1;
}

Asynchronous Loading

The UNIGINE Engine allows you to control asynchronous loading of files by means of the AsyncQueue class. All file-related methods of this class will load a file and add it to the file system as a cached one.

Last update: 2018-06-04