Comparing Unigine Editor 2.8 performance with Unity and Unreal

[morbid]

Why benchmarking Editor
The less time you’re spending looking at progress bars the better experience you have, the faster you’ll move on with your tasks. Thanks to our customers we have collected a number of high load usage scenarios that helped us to improve overall performance of import, assets, and scene management subsystems. To understand how current Unigine Editor version performs we ran a number of tests inspired by real cases and compared results with some popular game engines.

 

Test cases and methods
We tested assets import, bulk nodes operations and asset systems. Detailed description of how the tests were run is provided in the corresponding parts of this article. Here will be described general principles we’d followed during the research:

1. Every test was performed in the default Editors environment, no plugins attached, no settings changed.
2. General criteria of a test completion is the equal result. For instance, some editors compress 8k textures to 2k by default, so we had to manually reconfigure import settings to get the identical 8k texture in every tested editor.
3. If the test ended with an application crash or hang we nullified the result.
4. All measurements were done multiple times to verify values.
5. In order to make the tests closer to the real life, we used parts of the real projects from our customers and some complex enough test assets from open sources.

All measures were done in seconds, then converted to the number iterations per hour. All graphs are built with iteration/hour value.

 

Test rig
We decided to use a somewhat typical development hardware:

• CPU: Core i5-9600K @ 3,7 GHz
• GPU: GTX 1080 - 430.64
• RAM: 32 GB (4x8 GB Corsair Vengeance LPX @ 2133 MHz)
• MB: Gigabyte GA-B360M-DS3H
• Storage: Samsung 960 EVO SSD 

Software:
All tests were run on Windows 10 Professional (build 17763.503). We used the following versions of the engines:

• UNIGINE Engine 2.8.0.1
• Unity Engine 2019.1.2f1
• Unreal Engine 4.22.1

 

Test 1. Geometry only import
The first test files (FBX and OBJ) with polygon count varied from 1 to 13 millions, and number of meshes varied from 1 to 46 000+.

It was hard to distinguish a straightforward conclusion due to a different editors behaviour in the performed tests. However, there are some things that can be stated: Unigine Editor loses performance with simple models with low number of stored objects (from 1 to a couple of dozen). We regain speed in extreme loads with thousands of objects in a single model. The more meshes are imported the higher delta of time spent on the process.

You may notice that Unreal Editor’s results aren’t represented in all charts. This is happened due to a crash or insufficient RAM volume.

The major inference here is that we have to reevaluate our approach to static geometry import, because that’s where the speed was bottlenecked.

Iterations per hour and polygon count

 

Test 2. Importing materials
Here we checked how fast Unigine Editor handles import of thousands of materials. At first, we verified that the number of objects does not interfere with material import speed: fbx files with 5000 materials and different number of meshes were imported at the same speed in every tested application. The test files can be found here:

Here we did our best: in every case Unigine Editor imports thousands of materials much faster than the other contestants.
 

Iterations per hour and number of materials

 

Test 3. Geometry and textures
From synthetic test  we moved to the more realistic use cases. FBX and OBJ files were divided into two groups: one with embedded textures and another with external textures. Polygon count varied from 100 000 to 10 M, textures number from 5 to 1 199. Most of the models were kindly provided by our customers, so they are representative for real projects, but we can't share them. However, there’s the Amazon Lumberyard Bistro (two OBJ files with external textures) which is Creative Commons (CC BY 4.0) licensed.

Importing external textures made it difficult for Unity Editor. All tests were considered to be failed because only geometry with materials but not a single texture were imported. 
 

Iterations per hour and number of external textures

Embedded textures were handled successfully by all of three tested editors. Unigine prevailed in 3 of the 5 cases, however, increasing the number of textures up to 1000 slowed import down.

Iterations per hour and number of embedded textures

 

Test 4. Importing cubemap textures
This test was mostly focused on 16k Cubemap textures, but as exclusion we used one 15k texture. All textures were obtained from https://hdrihaven.com. Getting a usable 16k cubemap texture in Unigine Editor is almost 4 times faster than in Unity. Unreal dealt with 15k texture only, bigger resolution caused crashes. However, Unigine experienced issues with RGBA32F EXR and 17k RGB32F HDR textures. We’ll figure out why these files wasn’t processed.

Iterations per hour and image megapixels

 

Test 5. Importing 2D textures
We used the range of textures from 32x32px to 16k. One test included 400 textures of different size and file format. All files were compressed from PNG, TGA, JPG to the runtime-ready format (DDS). The most noticeable breakaway seen with 4k textures: Unigine workflow brings up to 11 times faster asset processing.

Iterations per hour and texture resolution

 

Test 6. Nodes manipulation in Editor
By nodes manipulation we mean a number of general tasks every user does in the scene editor: creating instances, cloning, transforming and undoing changes. We used two sets of nodes: 5k and 15k nodes in the scene.

Every contestant editor managed with 5 000 of nodes in the scene. Unigine performed much better in the number of undo cases and node transformations.

Iterations per hour and test cases

Situation drastically changes with 15 000 of nodes. Unreal was dropped out because all cases resulted in hang or crash. We significantly lost in nodes selection, but still prevail in undo. Transform and creating duplicates.

Iterations per hour and test cases

 

Test 7. Assets manipulation in Editor
Ideologically similar test was performed for the asset system. For the 5 000 assets minor Unigine’s drawback is selection time, while other operations are completed quite fast.

Iterations per hour and test cases

Increasing assets number respectively scales the required time for operation completion.

Iterations per hour and test cases

 

Final thoughts
The work we’d done to the moment made Unigine Editor fast and reliable tool for 3D scene building. Of course, the tests showed us further optimization opportunities: massive selection, small meshes import and cubemap formats widening. The major spotlights of Unigine Editor are:

• Competitive content import system, fastest in a number of use cases
• Stable scene management under extreme load
• Effective and reliable workflow with big projects

Please share your thoughts on UnigineEditor performance!
 

[david.sanda]

Asset loading at the start is still something to think about a lot... during each loading. I bought M2 specifically because of that and it still takes about 40 seconds for our ESQ project. It would be nice to have some option in editor to not to recheck this during the start and just go with the cached data.

[binstream]

1 hour ago, david.sanda said:

Asset loading at the start is still something to think about a lot... during each loading. I bought M2 specifically because of that and it still takes about 40 seconds for our ESQ project. It would be nice to have some option in editor to not to recheck this during the start and just go with the cached data.

We will consider this idea with the tools team.

[davide445]

Choosen Unigine for that (and others) reason, being Unreal crashing just trying to import huge architecture projects. 

Never tested Unity since some years, but seems is more effective than Unreal in dealing with big files and various formats. 

[morbid]

@davide445, Unreal needs a lot of RAM for import. After the first tests with 16Gb we decided to upgrade our rig to 32Gb just because of Unreal.

[demostenes]

On 7/29/2019 at 8:56 AM, davide445 said:

Choosen Unigine for that (and others) reason, being Unreal crashing just trying to import huge architecture projects. 

Never tested Unity since some years, but seems is more effective than Unreal in dealing with big files and various formats. 

Unity is by far worse of all major engines, when dealing with bigger files, projects, etc....Even half year ago it was randomly crashing on such basic thing as importing 2GB+ unity package (bought on asset store) into empty project. Unity is usable for smaller/mobile projects, but anything bigger is just crashing and crashing.

Edited July 30, 2019 by demostenes

[davide445]

@morbid my PC does have 32GB RAM, still after 5h time past importing the asset, the Unreal editor crashed attempting to insert it in the viewport. Asked for support they told me fbx it's not good for industrial pipeline and need to wait for Datasmith managing the ifc format. Curiously Twinmotion (based on UE) does managed in my tests much better the same huge asset.

@demostenes last time used Unity in 2016, when I decided was too basic for my ambitions. Used Unreal so far for learning, experimenting and small things, till I discovered Unigine last year, my choice for the first really challenging project.

Edited July 30, 2019 by davide445