File System

The UNIGINE engine has its own file system module used to access files and folders. It has a few peculiarities you should be aware of, when loading resources or organizing the structure of your UNIGINE-based project.

In the UNIGINE file system, each file has a GUID (Globally Unique Identifier), which defines a virtual path to this file (not a file on a disk). Using GUIDs provides more flexible file management: you can abstract from file names (which can be the same in different folders of the project). For example, you can change a path to the file while keeping the same GUID.

The Engine generates GUIDs for all files of the virtual file system.

Files of the UNIGINE file system can be accessed by using both names or GUIDs: you can obtain a GUID for the specific file, change the file GUID, add/remove the file with the certain name or GUID to/from a blob or a cached file, and so on.

If the UnigineEditor is loaded, it will save the generated GUIDs to the data/guids.db file automatically. Otherwise, you can implement the logic of updating guids.db via the code. The guids.db file stores a pair: a GUID of a file and a path to this file relative to the data folder.

As the UNIGINE file system provides mount points for project's extending, there can be several guids.db files within a single project. Each mount point can store its own guids.db: GUIDs for external files aren't written to data/guids.db.

Deleting the guids.db file won't cause any issues: if there is no guids.db file, the Engine will search among .meta files. However, guids.db might be necessary for the final release build of the project, if you aren't going to include the .meta files.

Dynamic Scanning#

Automatic Resource Reloading#

Known vs Unknown Files#

All files that are used by the Engine at run time are stored in the data folder specified via the -data_path start-up option. By default, it is the data folder created automatically on project creation via UNIGINE SDK Browser.

• unigine_project
  • assets
  • bin
  • data

bin\main_x64d.exe -data_path "../"

Current Directory#

// cbox.mesh is stored outside the data directory, so the path is specified relative to the current directory
ObjectMeshStatic cbox = new ObjectMeshStatic("../../data/cbox.mesh");

Types#

Content Access#

The virtual file system can be easily extended by using the mount point feature. It allows you to extend the virtual file system of your project by adding any external folders and packages to the data directory.

Using the mount points in your project allows you to use content stored:

• In a single folder or repository for several projects.
• In several folders for one project. You can create as many mount points as required for the project.

A UNIGINE-based project has a single data directory. Here all project assets and runtime files are stored. In addition, the data folder can store mount points created via the Asset Browser (Create -> Create Mount Point) or API.

A mount point is represented on the disk as a *.umount file: a file in the JSON format that stores a reference to an external directory or package as an absolute path or a path relative to the current directory. Also the *.umount file stores version of UNIGINE SDK, in which the mount has been created, and an information on whether the mount is read-only or not. You can also specify exclusive_filters (whitelist) or ignore_filters (blacklist) as a list of wildcards (e.g., "*.jpg", "some_folder_*") to selectively add files of certain types or contents of folders with certain names, or on the contrary, to ignore the ones specified). For example:

{
    “data_path”: “D:/mount_test”,
    “readonly”: false,
	"ignore_filters": ["*.jpg","*.prop"],
    “version”: “2.7.3.0”
}

All folders inside the mount point are treated by the file system as the usual folders with assets inside the data directory.

Inside each mount point, there is a .runtimes folder that stores runtime files generated for assets of the external directory. Note that they aren’t added to the runtimes stored inside the data/.runtimes folder. If you move an asset from one mount point to another, its runtimes will be moved as well.

As well as runtime files are stored per mount point, the guids.db file is also generated and stored per mount point by default: GUIDs for external files aren't written to data/guids.db.

If several team members work with a single mount point, it should be read-only to avoid issues.

The read-only mount point doesn’t allow any changes in the folder or package it is referenced to. It means that such folder must store assets with already generated .meta files and runtimes. Otherwise, they won’t be available in the Asset Browser. The workflow here should be as follows:

1. The .meta and runtime files for assets are generated once and saved/committed to the folder/repository (if any).
2. In each project that uses the assets from this folder/repository, the read-only mount point is created. The assets are used "as is", without opportunity to somehow modify them.

When working with mount points there are rules to be followed:

• The mount points can be embedded: a folder referenced by a *.umount file can store another *.umount file, etc. However, looped mount points are not allowed: you cannot create the 2.umount inside the 1.umount that refers to 1.umount.
• The *.umount files cannot be packed, as well as packages cannot store another packages. However, the *.umount file can refer to a package.
• The *.umount file should have a unique name. If the data folder contains a folder with the same name as the mount point, this mount point will be ignored.

When the Editor is loaded, automatic resource reloading isn’t available for the mount points. Each mount point is updated manually on demand: in the Asset Browser, right-click the mount point and choose Refresh Mount Point. If the Editor isn't loaded, the Engine reloads all resources, including mount points, after reloading the world, if the resources are added to the virtual file system.

The Engine accepts the relative, absolute, network and virtual paths.

Virtual Paths#

external_images/1.tga

getVirtualPath(“D:/projects/unigine_project/test/1.tga”); // returned: test/1.tga

getAbsolutePath("D:/projects/unigine_project/test/1.tga"); // returned D:/content/test/1.tga

Network Paths#

//share/content/1.tga

Relative vs Absolute#

The Engine resolves any path as follows:

1. First, it tries to convert the path to a full virtual one.
2. Then, it tries to get the current mount point by this path.

By using the obtained information on the path, the Engine will be able to get a real path to the file on the disk (or blob/package/cache).

A virtual path can represent up to four entities at the same time: it can be a file on the disk, a file stored in a package, a file added to a cache and to a blob.

• Within the project folder, you can have both core/textures/white.dds and core.ung/textures/white.dds.
• In addition, in the code, you can have both:
  Source code (C++)

  // the file loaded into a cache
  FileSystem::get()->addCacheFile("core/textures/white.dds");
  // the file loaded into a blob
  FileSystem::get()->addBlobFile("core/textures/white.dds");

During read / write operations, the Engine will load the first found entity for such virtual path. The entities will be checked in the following order:

• For read operations:
  1. The file loaded into a blob.
  2. The file added into a cache.
  3. The read-only file stored on the disk.
  4. The file stored in a package.
• For write operations:
  1. The file loaded into a blob.
  2. The file stored on the disk.
  Notice

  The cached and packed files aren't checked as write operations aren't allowed for them.

Working with assets via the Editor is clear and simple, but in order to access your project's files properly you should have a clear understanding of the concepts of asset and runtime files.

Generated runtime files have constant GUIDs and are named as follows:
<GUID>.<extension> (e.g., ab23be4cd7832478edaaab23be4cd7832478edaa.dds).

These files are stored in sub-folders of the data/.runtimes folder. The structure or this folder is optimized for the file system.

E.g., your non-native asset data/my_textures/1.tga (GUID = "aeb53b44cdbbbbbbbbaaabccc1c1c1c1c1c1c1c1") will have runtime file generated for it in a folder: ./runtimes/ae/aeb53b44cdbbbbbbbbaaabccc1c1c1c1c1c1c1c1.dds

Therefore each runtime file has an alias - a human-readable form of a path used to refer to this file.

Full aliases are constructed as follows: <source_asset_path>/<runtime_alias>

E.g.:

• 1.tga/1.dds
• 1.fbx/material/1.mat

In order to simplify access to runtime files, we also use a concept of the primary runtime - a runtime file uniquely assoсiated with the asset. It acts like an implied reference to a runtime file: when we say "model.fbx", we actually mean "model.node". So, that we could write:

NodeReferencePtr node = NodeReference::create("model.fbx");

There are two ways you can access your assets and runtime files:

• Using a GUID
• Using a path

The file system includes a subsystem for managing assets and runtime files. This subsystem is implemented as a separate class named FileSystemAssets.

You can use assets_info and assets_list console commands to view infomation on non-native assets and runtimes generated for them.

Accessing by Path#

Accessing by GUID#

UGUID asset_guid; // GUID of the asset named "1.tga"
const char *asset_path = "1.tga";

ImagePtr image = Image::create(asset_guid); // -> 1.tga
ImagePtr image = Image::create(asset_path); // -> 1.dds

File modifiers serve to automatically choose which resources to load when a UNIGINE project is run on different platforms or with different localizations. Instead of keeping multiple versions of the same project and copying shared data between them, you can add a custom postfix to file or folder names, and load only required resources on demand.

Modifiers are added to file or folder names as a postfix (only one can be specified). Any custom postfix can be used. For example, it could be:

• File name modifier: file.small.node or texture.eng.dds
• Folder name modifier: textures.lowres
  Notice

  If a folder has a modifier, files inside of it should not have their own modifiers. Otherwise, file modifiers will be ignored.

Register necessary modifiers in code via addModifier(). When the project is running, resources with the registered modifiers will be loaded automatically. Files without modifiers have the lowest priority (can be used for default resources).

Usage Example#

// unigine.cpp

int init() {
	...
	
	// Register modifier
	engine.filesystem.addModifier("de");
	
	// Set a splash texture
	engine.splash.setWorld("textures/splash.png");   // splash.de.png will be automatically used
	...
	
	return 1;
}

bin\main_x64d.exe -extern_define "LANG_DE"

// unigine.cpp
string lang = "";

int init() {
	...
	
	// Parse EXTERN_DEFINE
	#ifdef LANG_DE
		lang = "de";
	#elif LANG_FR
		lang = "fr";
	#endif
	
	if(lang != "") {
		engine.filesystem.addModifier(lang);
	}
	
	// Set a splash texture: splash.de.png or splash.fr.png will be used if the language is passed
	engine.splash.setWorld("textures/splash.png");	// otherwise, splash.png
	...

	return 1;
}

The UNIGINE Engine allows you to control asynchronous loading of files by means of the AsyncQueue class. All file-related methods of this class will load a file and add it to the file system as a cached one.