Jump to content

[SOLVED] Improve engine startup times


photo

Recommended Posts

Hi,

We've switched from 2.5 to 2.9 lately and now big engine startup times really make it very hard to quickly iterate when doing native code (compile, run engine, test, stop, recompile, restart engine etc). From my tests it seems that the 2.9 main startup bottleneck is the fact that now the engine parses and loads all materials at startup.

Is there a way to bypass this (or optimize it) and have the materials being loaded and recompiled at first usage (during rendering)? 

I now this is meant to decrease the stalls during runtime (and we will be using it for final build) but for developing this is a very time consuming step to do all the time.

Regards,

Adrian L.

Link to comment

Hi,

In hybrid build (sort of debug) it takes approx 4 mins to load (doesn't help that i have to run from an encrypted hard drive). Also texture cache loading takes considerable time. From my debug of the 2.9 source it seems that it takes all files from the virtual file system and if any has mat extension it loads it. So basically it loads all our development projects materials (I am talking here about dev setup, not final build). While engine core materials are few, we have over 8000 materials. Also I can see now way to skip some paths unless I modify the source code. Is there a way to postpone materials loading until they are used? Or some other way to optimize this?

Kind Regards,

Adrian L.

Link to comment

Hi Adrian,

Could you please also specify:

  • Total amount of files inside data directory
  • Total data folder size
  • Disk type that you are using (HDD / SSD) and model

I'm assuming that you are currently using Windows for the tests, right? Windows Defender application with enabled real-time protection can significantly slow down the file access operations (and also consume some CPU time).

image.png

Adding your project's data and all the mounts directories to the exclusion list can give you a huge boost in Engine and Editor loading times (up to 2-5x). However, it's not really noticeable on a small projects (less than 10K files in total). You can also set exclusion directory it in elevated Powershell by running following command:

  • powershell -inputformat none -outputformat none -NonInteractive -Command "Add-MpPreference -ExclusionPath 'C:\Unigine Projects\My Project 1\data'"

After the first engine launch (let's call it cold) all the files should be available in the OS Standby Memory Cache and secondary access to the same file (second engine launch) in theory should be much faster (even with real-time protection from Windows Defender). I'm afraid, other than disabling real-time protection there is no ways to improve engine loading times in 2.9.

We slightly improved loading times for a big projects in 2.11 SDK (which we plan to release within a 2 weeks). If you are interested - we can enabled alpha/beta access for your company in order you to get the latest version and see if the virtual FS speed is the real bottleneck here.

Thanks!

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to comment

We have now cca 6.2k materials in project and start is under 40 seconds. Measured on PCI 4.0 NVMe drive capable of 4GB/sec read/write. I strongly recommend to use something like this, for good workstation you need it anyway :)

Link to comment

Hi everybody and thank you for the support,

It turns out antivirus was the fault (mainly the default defender from windows, other, 3rd party are faster, but still slow down things). I also had some issues with the file cache on some systems and second runs (warm cached) were similar with the first one (cold).

Now, the engine starts under 35 secs with all the materials loading. I still think loading all materials, without a configurable way to indicate exceptions (maybe a solution involves using mounts done only when needed?), is not a perfect solution for development systems.

Kind Regards,

Adrian

Link to comment

All the materials needs to be loaded at once because there might be a parent-relation connection between them and you never know if the loaded random material from disk will require any other material to be loaded altogether.

Also, to be more clear: material (.mat) file loading doesn't mean that it will compile shader or something else, it's just a material file (XML) parsing and filling the materials list with parent-child hierarchy, so the engine would know which materials are depends on other materials.

We didn't find any better way to do it rather than on engine start-up (since it's the one-time operation).

If you know that for debugging you don't need such amount of materials you can hide them from the engine virtual filesystem by moving to another mounting point, but you need to be careful and watch for the parent-child hierarchy to be valid.

Thanks!

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to comment
  • silent changed the title to [SOLVED] Improve engine startup times

Hi Silent and thank you for the detail explanation.

How was this done in 2.5? I think it didn't load them all at once, at startup.

And yes, I know it doesn't compile the shaders. But from my memory debugging I also saw that the memory footprint for materials is quite big (from memory it is approx 300MB-400MB for the 8000 materials we load). It may not seem much in the current hw context, but we are very memory restricted and actually run multiple programs and instances of Unigine on the same machine. While the final build doesn't include all these materials, it makes testing with similar conditions as the final from the dev project impossible (without having to do actual packaging).

Anyway, for now, the current situation is okish for me, as the loading times improved based on your suggestions.

Kind Regards,

Adrian

Link to comment

If your main concern here is about the overall memory usage of the process, you may also found useful to switch from the Development engine builds to Release. If you are using SDK Browser, you can do it here: SDK Browser My Projects → <Project Name> → Other Actions → Configure Project → Engine: Development (Change to: Release).

Materials in 2.5 were organized in material libraries and they were also loaded at the engine start-up, so there were no significant changes made.

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to comment
×
×
  • Create New...