UnigineEditor
Interface Overview
Assets Workflow
Settings and Preferences
Adjusting Node Parameters
Setting Up Materials
Setting Up Properties
Landscape Tool
Using Editor Tools for Specific Tasks
FAQ
Programming
Fundamentals
Setting Up Development Environment
Usage Examples
UnigineScript
C++
C#
UUSL (Unified UNIGINE Shader Language)
File Formats
Rebuilding the Engine and Tools
GUI
Double Precision Coordinates
API
Containers
Common Functionality
Controls-Related Classes
Engine-Related Classes
Filesystem Functionality
GUI-Related Classes
Math Functionality
Node-Related Classes
Networking Functionality
Pathfinding-Related Classes
Physics-Related Classes
Plugins-Related Classes
CIGI Client Plugin
Rendering-Related Classes

File System

The UNIGINE engine has its own file system module used to access files and folders. It has a few peculiarities you should be aware of, when loading resources or organizing the structure of your UNIGINE-based project.

GUIDs#

In the UNIGINE file system, each file has a GUID (Globally Unique Identifier), which defines a virtual path to this file (not a file on a disk). Using GUIDs provides more flexible file management: you can abstract from file names (which can be the same in different folders of the project). For example, you can change a path to the file while keeping the same GUID.

The Engine generates GUIDs for all files of the virtual file system.

Files of the UNIGINE file system can be accessed by using both names or GUIDs: you can obtain a GUID for the specific file, change the file GUID, add/remove the file with the certain name or GUID to/from a blob or a cached file, and so on.

Notice
To manage GUIDs via API use the UGUID class.

If the UnigineEditor is loaded, it will save the generated GUIDs to the data/guids.db file automatically. Otherwise, you can implement the logic of updating guids.db via the code. The guids.db file stores a pair: a GUID of a file and a path to this file relative to the data folder.

As the UNIGINE file system provides mount points for project's extending, there can be several guids.db files within a single project. Each mount point can store its own guids.db: GUIDs for external files aren't written to data/guids.db.

Deleting the guids.db file won't cause any issues: if there is no guids.db file, the Engine will search among .meta files. However, guids.db might be necessary for the final release build of the project, if you aren't going to include the .meta files.

File System Update#

Dynamic Scanning#

Dynamic scanning allows the Engine to form a virtual file system for all files within the data folder (including the ones inside mount points). Dynamic scanning is performed on the Engine start-up. For files physically stored inside the data directory, it enables tracking file changes in real-time and using relative paths to address files.

If changes to the file system are made not by means of UnigineScript, you may need to call the filesystem_reload console command.

Automatic Resource Reloading#

When the UnigineEditor is loaded, the Engine tracks changes made in files at run time: it checks the time of the last modification of such files and updates them in the memory. If the UnigineEditor isn't loaded, the changed files will be reloaded after reloading the world.

Notice
However, it is possible only if the files are added to the virtual file system (dynamic scanning has been performed on the start-up).

When the UnigineEditor is loaded, automatic tracking of changes isn't available for the mount points. Each mount point is updated manually on demand.

Known vs Unknown Files#

If you add new files at run time, the Engine won't know anything about such files (as the virtual file system has been formed on the start-up). Re-scanning the file system is resource-consuming, so, in this case, you can add new files to the virtual file system via API by using addVirtualFile().

Data Directory#

All files that are used by the Engine at run time are stored in the data folder specified via the -data_path start-up option. By default, it is the data folder created automatically on project creation via UNIGINE SDK Browser.

Notice
The path to the data directory can be specified relatively to the binary executable or as an absolute path.
When the file system resolves paths, it tries to concatenate data paths with the specified path and perform a lookup. In case of absolute paths, the file system will use them as-is without any checks.
Notice
When the path to the data directory is relative, the engine switches the current directory to the directory with the binary executable.
For example, if the project folder has the following structure:
  • unigine_project
    • assets
    • bin
    • data
The data path will be unigine_project/data after the application start-up:
Shell commands
bin\main_x64d.exe -data_path "../"
Notice
A path specified in the -data_path option won't be written into the configuration file.

Current Directory#

When the specified -data_path is absolute, the current working directory may differ from the directory with the binary executable. However, when the path to the data directory is relative, the Engine switches the current directory to the one with the binary executable.

When accessing a file outside the data directory via API, the path to such file should be specified relative to the current directory. For example:

Source code (C++)
// cbox.mesh is stored outside the data directory, so the path is specified relative to the current directory
ObjectMeshStatic cbox = new ObjectMeshStatic("../../data/cbox.mesh");

File Packages#

Types#

UNIGINE supports the following types of file archives to save space or pack the production version of resources:

  • UNG (a UNIGINE-native format for archives created with Archiver tool)
    Notice
    The maximum size for a file inside a UNG archive is limited to 2 GB.
  • ZIP
  • Custom C++ packages created via UNIGINE API
Besides saving space, archives also speed up resource loading, as files in an archive are read linearly.

UNG and ZIP archives are loaded automatically if they are found within the data folder. Files are added to the virtual file system just like in case with non-archived files.

Notice
A package cannot store another package. Also a mount point cannot be packed. However, it can refer to a package.

Content Access#

Archives are completely transparent to the Engine. There is no need to explicitly unpack the archives, as their content is automatically handled as not packed. Archived files are addressed as if they are non-archived. For example, if you have data/project/archive.ung and want to address directory/file.txt within it, simply specify the following path: project/directory/file.txt.

Inside the archive, files can be organized in any way. However, in the root of the archive only files with unique names should be placed. Otherwise, the file search will return incorrect results.

Notice
Using GUIDs allows you to avoid the need to control uniqueness of names when working with archived files.

Here is an example of an incorrect file tree for an archive:

  • my_archive.ung
    • my_folder
      • file_2.txt
    • file_1.txt
    • file_2.txt
In this case, there is no problem with file_1.txt, since its name is unique. file_2.txt, on the other hand, will cause problems: it does not guarantee that a non-root file will be returned.

The correct archive structure can be specified as follows:

  • my_archive.ung
    • my_folder
      • file_2.txt
    • another_folder
      • file_2.txt
    • file_1.txt
In this case, the files with the same names are stored in different directories, so the file search will be perfectly correct.

If there is a name collision between an archived file and a non-archived one, the first matching file is returned. The search is performed in the following order:

  1. Non-archived files
  2. Files in UNG archives
  3. Files in ZIP archives

From UNIGINE API, archives are handled using the FilesSystem functions as well.

Extending File System#

The virtual file system can be easily extended by using the mount point feature. It allows you to extend the virtual file system of your project by adding any external folders and packages to the data directory.

Using the mount points in your project allows you to use content stored:

  • In a single folder or repository for several projects.
  • In several folders for one project. You can create as many mount points as required for the project.

A UNIGINE-based project has a single data directory. Here all project assets and runtime files are stored. In addition, the data folder can store mount points created via the Asset Browser (Create -> Create Mount Point) or API.

A mount point is represented on the disk as a *.umount file: a file in the JSON format that stores a reference to an external directory or package as an absolute path or a path relative to the current directory. Also the *.umount file stores version of UNIGINE SDK, in which the mount has been created, and an information on whether the mount is read-only or not. You can also specify exclusive_filters (whitelist) or ignore_filters (blacklist) as a list of wildcards (e.g., "*.jpg", "some_folder_*") to selectively add files of certain types or contents of folders with certain names, or on the contrary, to ignore the ones specified). For example:

Output
{
    “data_path”: “D:/mount_test”,
    “readonly”: false,
	"ignore_filters": ["*.jpg","*.prop"],
    “version”: “2.7.3.0”
}
Notice
Use only forward slashes for paths (Unix convention) as backslashes in JSON files are used as escape charactes.

All folders inside the mount point are treated by the file system as the usual folders with assets inside the data directory.

Inside each mount point, there is a .runtimes folder that stores runtime files generated for assets of the external directory. Note that they aren’t added to the runtimes stored inside the data/.runtimes folder. If you move an asset from one mount point to another, its runtimes will be moved as well.

As well as runtime files are stored per mount point, the guids.db file is also generated and stored per mount point by default: GUIDs for external files aren't written to data/guids.db.

If several team members work with a single mount point, it should be read-only to avoid issues.

The read-only mount point doesn’t allow any changes in the folder or package it is referenced to. It means that such folder must store assets with already generated .meta files and runtimes. Otherwise, they won’t be available in the Asset Browser. The workflow here should be as follows:

  1. The .meta and runtime files for assets are generated once and saved/committed to the folder/repository (if any).
  2. In each project that uses the assets from this folder/repository, the read-only mount point is created. The assets are used "as is", without opportunity to somehow modify them.
Notice
All .meta files and runtimes in the mounted directory/package must be valid and up-to-date. All assets, that were not properly migrated, as well as the ones having outdated .meta or runtime files will be unavailable and won't be displayed.

When working with mount points there are rules to be followed:

  • The mount points can be embedded: a folder referenced by a *.umount file can store another *.umount file, etc. However, looped mount points are not allowed: you cannot create the 2.umount inside the 1.umount that refers to 1.umount.
  • The *.umount files cannot be packed, as well as packages cannot store another packages. However, the *.umount file can refer to a package.
  • The *.umount file should have a unique name. If the data folder contains a folder with the same name as the mount point, this mount point will be ignored.

When the Editor is loaded, automatic resource reloading isn’t available for the mount points. Each mount point is updated manually on demand: in the Asset Browser, right-click the mount point and choose Refresh Mount Point. If the Editor isn't loaded, the Engine reloads all resources, including mount points, after reloading the world, if the resources are added to the virtual file system.

Paths#

The Engine accepts the relative, absolute, network and virtual paths.

Notice
The UNIGINE file system features path normalization: it automatically converts Windows paths to UNIX paths.

Virtual Paths#

The UNIGINE file system is strict. It means that the virtual file system always checks the exact file location instead of searching somewhere inside the data directory. Such approach makes working with project files clear and transparent.

The virtual file system operates with virtual paths to files. A virtual path is a path to a file inside the data folder (including files in the mount points). The Engine always tries to convert any path to the virtual one. There are several types of the virtual paths:

  • Full virtual path - a path to a file inside the data folder.
  • Partial virtual path.
  • Virtual path specified as an absolute one.

When specifying a virtual path to a file inside the mount point, it always includes the name of the mount point. For example, if you have a data/external_image.umount mount point, that refers to D:\external_content, you should access any file in this folder as follows:

Output
external_images/1.tga

Partial#

Using partial paths means that the file system performs a non-strict file search. In this case, only a file name can be provided without a path. Partial paths are allowed for cases where user can enter a path manually, such as:

  • World loading operations. You can specify only a world name and it will be found and loaded.
  • Include paths in the source code.
  • Paths in UI files.
  • Paths in manual material files.
  • Paths to textures in base material files.
  • Paths to font files.
Notice
If a name is not unique, the first found file with such name is loaded.
It is also possible to provide a sub-path that uniquely specifies a file. For example, to load data/project/my_world/my_world.world, you can use my_world.world (if a name is unique) or my_world/my_world.world.

Also you can refer files by GUIDs to uniquely specify a file.

Notice
A partial path can be resolved via the resolvePartialVirtualPath() method of the FileSystem class. It converts the given partial virtual path to the full virtual one, and then it can be processed as required.

Virtual Path Specified as Absolute#

The Engine can resolve paths that are specified as absolute, but actually are virtual. It means that the path to a file can look like absolute, but physically there is no such file by this path.

For example, there is a project stored in the D:/projects/unigine_project/ folder. In the data folder of this project, there is a test.umount that refers to a D:/content/test/ folder with the 1.tga texture. You can specify the path to this texture as absolute as follows and the Engine will be able to return a virtual path for it.

Source code (C++)
getVirtualPath(“D:/projects/unigine_project/test/1.tga”); // returned: test/1.tga
If you try to get an absolute path, the absolute path D:/content/test/1.tga will be returned.
Source code (C++)
getAbsolutePath("D:/projects/unigine_project/test/1.tga"); // returned D:/content/test/1.tga

To get more usage examples on the getVirtualPath() and getAbsolutePath() methods, check the article on the FileSystem class.

Network Paths#

A lot of heavy content used in a project is usually stored on a network drive. To access such content, a network path should be specified in the following format:

Output
//share/content/1.tga
The network paths are successfully resolved by the file system.

You can create a mount point that refers to a network folder. It allows you to avoid unnecessary copying of assets to a local computer and, therefore, saves the disc space.

Relative vs Absolute#

A relative path is a path specified relative to the current directory. It should be used when, for example, you need to write some file in the same folder with the binary executable. If you specify a virtual path, it will be written into the data directory by default.

When relative paths are used, you can relocate your UNIGINE-based application or copy it onto another machine, and all resources will be properly loaded. There is no loading speed penalty as well: it is as fast as loading files by an absolute path due to the virtual file system. It is possible to use absolute paths to load resources outside the data folder, but such project will not be portable.

As file names are added to the virtual file system, usually the same name and path should be used to load and remove the file when accessing from your source code by using FileSystem functions:

  • For default resources, functions return full paths relative to the data folder.
  • If you load a file and specify a relative path, use a relative path to delete the resource.
  • If you load a file using an absolute path, use an absolute path to delete the resource.

Notice
Using GUIDs allows you to avoid the need to control paths when working with files.

You can check whether the path is absolute or relative via the isAbsolute() function.

Also the file system allows you to get a path to a file relative to the data folder by using the getVirtualPath() function.

Loading Priorities#

The Engine resolves any path as follows:

  1. First, it tries to convert the path to a full virtual one.
  2. Then, it tries to get the current mount point by this path.

By using the obtained information on the path, the Engine will be able to get a real path to the file on the disk (or blob/package/cache).

A virtual path can represent up to four entities at the same time: it can be a file on the disk, a file stored in a package, a file added to a cache and to a blob.

Notice
A GUID is generated for a virtual path: neither a file on the disk, a package file, a cached or a blobbed file can have a GUID.
For example, the textures/white.dds path has only one GUID, however, it can represent the following at the same time:
  • Within the project folder, you can have both core/textures/white.dds and core.ung/textures/white.dds.
  • In addition, in the code, you can have both:
    Source code (C++)
    // the file loaded into a cache
    FileSystem::get()->addCacheFile("core/textures/white.dds");
    // the file loaded into a blob
    FileSystem::get()->addBlobFile("core/textures/white.dds");
Notice
You can check whether the virtual path or the GUID represent a certain entity via the corresponding methods of API.

During read / write operations, the Engine will load the first found entity for such virtual path. The entities will be checked in the following order:

  • For read operations:
    1. The file loaded into a blob.
    2. The file added into a cache.
    3. The read-only file stored on the disk.
    4. The file stored in a package.
  • For write operations:
    1. The file loaded into a blob.
    2. The file stored on the disk.
    Notice
    The cached and packed files aren't checked as write operations aren't allowed for them.

Accessing Assets and Runtime Files#

Working with assets via the Editor is clear and simple, but in order to access your project's files properly you should have a clear understanding of the concepts of asset and runtime files.

Generated runtime files have constant GUIDs and are named as follows:
<GUID>.<extension> (e.g., ab23be4cd7832478edaaab23be4cd7832478edaa.dds).

These files are stored in sub-folders of the data/.runtimes folder. The structure or this folder is optimized for the file system.

Notice
Inside each mount point, there is also a .runtimes folder that stores runtime files generated for assets of the external directory. These runtimes aren’t added to the runtimes stored inside the data/.runtimes folder. If you move an asset from one mount point to another, its runtimes will be moved as well.
A runtime file generated for a non-native asset with a certain GUID will be placed in a folder, that has a name equal to the first two bytes of this GUID.
Notice
A sub-folder of the data/.runtimes folder may contain runtime files generated for different assets (if these assets have matching first two bytes in their GUIDs).

E.g., your non-native asset data/my_textures/1.tga (GUID = "aeb53b44cdbbbbbbbbaaabccc1c1c1c1c1c1c1c1") will have runtime file generated for it in a folder: ./runtimes/ae/aeb53b44cdbbbbbbbbaaabccc1c1c1c1c1c1c1c1.dds

Therefore each runtime file has an alias - a human-readable form of a path used to refer to this file.

Full aliases are constructed as follows: <source_asset_path>/<runtime_alias>

E.g.:

  • 1.tga/1.dds
  • 1.fbx/material/1.mat

In order to simplify access to runtime files, we also use a concept of the primary runtime - a runtime file uniquely assoсiated with the asset. It acts like an implied reference to a runtime file: when we say "model.fbx", we actually mean "model.node". So, that we could write:

Source code (C++)
NodeReferencePtr node = NodeReference::create("model.fbx");
Notice
Primary runtime is actually used instead of an non-native asset, when you refer to it by path.

There are two ways you can access your assets and runtime files:

The file system includes a subsystem for managing assets and runtime files. This subsystem is implemented as a separate class named FileSystemAssets.

You can use assets_info and assets_list console commands to view infomation on non-native assets and runtimes generated for them.

Accessing by Path#

The way of accessing a certain asset by path is determined by its type:

  • Native assets are accessed simply by their name:
    Source code (C++)
    ImagePtr image = Image::create("image.dds");
  • All non-native assets have a primary runtime file. So, when you refer to the asset by its name, this primary runtime file will be actually used. For example, if you specify:

    Source code (C++)
    ImagePtr image = Image::create("image.png");
    The image.dds generated primary runtime file will actually be used.

    You can also directly access any asset source file (not a runtime file). For example, if you need to specify a .png texture, you should write the following:

    Source code (C++)
    ImagePtr image = Image::create("asset://image.png");
    In this case, the runtime .dds file will be ignored, .png source file will be used.
  • Each container asset also has a primary runtime, in case of an FBX asset it is a generated .node file. So, you can use the following reference:

    Source code (C++)
    NodeReferencePtr node = NodeReference::create("teapot.fbx");
    The teapot.node generated runtime file will be used in this case.

    You can access each runtime file of a container asset. For example, an FBX file has .node and .mesh runtime files generated for it. You can access the generated .mesh file in the following way:

    Source code (C++)
    MeshPtr mesh = Mesh::create("teapot.fbx/teapot.mesh");

Accessing by GUID#

You can also access any runtime or asset in your project using GUIDs. In case if a GUID of a file is specified, the exact path corresponding to this GUID will be used:

Source code (C++)
UGUID asset_guid; // GUID of the asset named "1.tga"
const char *asset_path = "1.tga";

ImagePtr image = Image::create(asset_guid); // -> 1.tga
ImagePtr image = Image::create(asset_path); // -> 1.dds

Modifiers#

File modifiers serve to automatically choose which resources to load when a UNIGINE project is run on different platforms or with different localizations. Instead of keeping multiple versions of the same project and copying shared data between them, you can add a custom postfix to file or folder names, and load only required resources on demand.

Warning
This feature is run-time only, and is not supported by the UnigineEditor.

Modifiers are added to file or folder names as a postfix (only one can be specified). Any custom postfix can be used. For example, it could be:

  • File name modifier: file.small.node or texture.eng.dds
  • Folder name modifier: textures.lowres
    Notice
    If a folder has a modifier, files inside of it should not have their own modifiers. Otherwise, file modifiers will be ignored.

Register necessary modifiers in code via addModifier(). When the project is running, resources with the registered modifiers will be loaded automatically. Files without modifiers have the lowest priority (can be used for default resources).

Usage Example#

For example, three localization languages are supported in the project: English (by default), German and French. Depending on the language, different splash textures should to be loaded on the start-up.

To organize your resources, name them using the following file modifiers:

  • data
    • splashes
      • splash.png (this is a default version of the texture. In our case, a texture with an English title)
      • splash.de.png (a German title)
      • splash.fr.png (a French title)

After that, in the code you need to specify which modifier to use via addModifier(). This function is called in the system script (unigine.cpp) since a modifier need to be registered before the world and its resources start to be loaded. For example, to load a German splash screen and the low resolution textures interface:

Source code (UnigineScript)
// unigine.cpp

int init() {
	...
	
	// Register modifier
	engine.filesystem.addModifier("de");
	
	// Set a splash texture
	engine.splash.setWorld("textures/splash.png");   // splash.de.png will be automatically used
	...
	
	return 1;
}

Also you can use -extern_define CLI option to pass the language (for example, if a user chooses a language in the launcher).

Shell commands
bin\main_x64d.exe -extern_define "LANG_DE"

And here is how passed defines can be handled in the code.

Source code (UnigineScript)
// unigine.cpp
string lang = "";

int init() {
	...
	
	// Parse EXTERN_DEFINE
	#ifdef LANG_DE
		lang = "de";
	#elif LANG_FR
		lang = "fr";
	#endif
	
	if(lang != "") {
		engine.filesystem.addModifier(lang);
	}
	
	// Set a splash texture: splash.de.png or splash.fr.png will be used if the language is passed
	engine.splash.setWorld("textures/splash.png");	// otherwise, splash.png
	...

	return 1;
}

Asynchronous Loading#

The UNIGINE Engine allows you to control asynchronous loading of files by means of the AsyncQueue class. All file-related methods of this class will load a file and add it to the file system as a cached one.

Last update: 2018-12-27