Impostors and performance

[david.norlin]

I am using OSM data to place landcover objects, and the results are very impressive.  I am able to get excellent details, but I am also encountering some performance problems.  

To provide a bit of background - we are building a test island so we can implement all features on this island, refine processes, and get a full understanding before moving to our 600x800 sq. km. project.  Performance is very important, but we must also achieve good ground detail, especially at low-medium altitudes.  Our test area is the island of Bornholm - not very big, but large enough to have good variation for testing.

I am able to place nice-looking impostors without an issue.  In the pic I attached, I am using two different types to provide variation, and both of them have some height spread to further increase visual diversity.  

Density on both grass objects is set to 0.015, and we need this so that the landscape tool will place enough objects so that tree lines can be identified between farmfields.  If anything, I would like to increase this slightly, so that we have a bit more sharply defined tree lines - our eventual project location has many farmfields and this is a distinctive rural feature.  Visibility distance is set to 10000 to represent the level of draw distance we hope to achieve. 

The problem is that doing this across the whole island is really hurting performance.  I can't even look at the center of the island without FPS dropping into the 15-20 range.  It seems that this type of impostor-based generation is something Unigine is very good at, and I am sure I am doing something wrong.  On both grass objects, I have turned on "Culled by Clutter Tree" and "Culled by Occlusion Query" and turned off "Collider Object", but this hasn't helped.  

Also - does the impostor grass object size matter?  If I make the grass object size much larger, with that allow for fewer mesh clutter objects placed, improving performance?  

As always, thanks for the help.

[ulf.schroeter]

Simply extending the grass view distance up to the horizon for visualization of tree areas is NO scalable solution (as already indicated by your current performance problems). Some more sophisticated LOD mechanism will be required. One standard approach ist to replace individual tree billboard patches with coarse, outline-matching tree area polgon box geometries at larger distances and usage of LOD fade/blending to have smooth transitions between both tree area representations. At even larger view distances the tree area polgons could also be offline-baked into the coarse terrain texture (if you are using ObjectTerrein) actually elimination the  complete geometry overhead for tree area rendering. Also I would recommend to have a look at the Unigine Valley demo source as this demo already handles large-scale, high-detail terrain rendering at high framerates.

[david.norlin]

Ulf,

Thanks for your reply.  I will take a look at the Unigine Valley demo (I had been looking at the tank demo before).  Also, is actual mesh more performance friendly than a billboard?  I thought impostors were supposed to be the most efficient.  Maybe you can show an example of what you're talking about? 

I did realize that my step and subdivision values for the impostor grass were completely wrong, so I've edited that according to the formula in the documentation (it fixed the placement issues and got rid of the straight lines). As a test, I used the exact values they use as an example - step 1600, subdivision 16, max visibility at 6400, and my grass object size at 400x400 (my choice on the size - too small?  too big?).  But now I'm faced with the same performance spiking issue they describe in that article.  I get spikes between about 20-30 fps to 90-100 fps, also seen on the performance profiler.  It seems like one of the values is not correct.

Edited July 7, 2018 by david.norlin

[morbid]

Hello David,

from the first glance looks like you created extremely dense objects. Advice Ulf has given is relevant. With current tools you can imitate trees with detail layers, you don't need to render polygons (even for impostor objects) on a large scale distance where "tree" could be smaller than 1 pixel on a screen.

I suggest you the following:

• reduce the density of your grass objects
• create detail layers that will imitate forest colors, just use the same mask you've used for vegetation: https://developer.unigine.com/en/docs/2.7.1/objects/objects/terrain/terrain_global/details/
• after details are ready, reduce the visibility distance for your impostors
• if you're not using mesh clutter (geometry) then you don't have to change subdivision. By the way, there's an unobvious rule - step should divide size without remainder, check please, this could affect the visual appearance of impostors. You can try these settings: visibility distance = 3000, subdivision = 1.

If nothing helps, send us your scene via ftp or another file hosting.

ftp://files.unigine.com
user: upload
password: iejeePheeH3che

Thanks!

[david.norlin]

Hi Morbid,

I did reduce density from .015 to .010, and this seemed to help a bit.  It actually breaks up the trees a little bit, adding some nice variation to the placement and it looks quite good.

I am trying to follow the rules at the bottom of this page of the documentation, but one thing that is not mentioned is how large the grass clutter object should be.  What is best for performance?  Before I first posted, was using 1000x1000, but I changed it to 400x400 and the objects render more quickly when they pass into view of the frustum.  This seems better, however, if I move the camera rapidly, I still see grass impostor objects disappearing and being forced to re-render...quite distracting.  Maybe a smaller object size will be better?  I'd like to know how to calculate it so it fits best according to the other parameters mentioned in the documentation. 

Also, yes - I am using mesh clutter trees at close range.  This is important for us to do, as our product will need to support good up-close detail for user-controlled helicopters and vehicles. 

Lastly, based on what you and Ulf mentioned, do you simply mean a darker texture with the outline of the trees that follows the mask of the forest?  This might work at very long ranges, perhaps around 4-6,000 units of distance.

Thanks. 

[morbid]

Hi David, 

1 hour ago, david.norlin said:

how large the grass clutter object should be

technically there are no limitations on ObjectGrass size. For instance, Landscape tool can create up to 32k*32k objects. However, the object scale will impact overall performance. Usually, you need to find optimal step size for your grass object. If the cell is too big - it will be rendered slowly, especially with high-density values. If it's too small - grass generation also will be visible for the observer.

I suggest you take a look at vegetation settings in Oil Refinery demo, it could serve you as an example of grass and clutter parameters.

I'm afraid that the rule for step calculation is outdated described in the article is outdated (we'll take a closer look at this and fix the article). What should you keep in mind when synchronizing grass and clutter (besides seed and size):

• clutter step * clutter subdivision = grass step
• size and step for both objects should be divided without remainder

1 hour ago, david.norlin said:

if I move the camera rapidly, I still see grass impostor objects disappearing and being forced to re-render...quite distracting.

I think we should separate in editor camera movements and in a final build camera. You mentioned that there will be helicopters and other vehicles, are they capable of such rapid movements? If yes, it could be an issue and this will require additional settings tweaking. If no - then everything should be fine. 

1 hour ago, david.norlin said:

do you simply mean a darker texture with the outline of the trees that follows the mask of the forest

Ulf proposed some optimizations for deprecated ObjectTerrain, but the logic is still the same - you don't need any geometry on the far plane. I meant that you can significantly reduce visibility distance without impact on overall picture quality. The forest colors could be imitated with detail layers. On the attached screenshot the settings are the following:

• camera far clipping plane = 200 000
• impostors visibility distance = 3 000

 

[david.norlin]

Hi Morbid,

Thanks for the recommendations.  I am currently looking at the Refinery demo and trying to use some of those settings.  I saw your comment on this thread, and I understand that it will be an interesting challenge to optimize our very large terrain.  I think based on your suggestions I will try to greatly simplify dense forests, not place down grass in these areas, and instead focus on putting detail into open areas.  

Thanks for your help.  I'll see how some of these adjusted settings work in the editor. 

[david.norlin]

Morbid,

I have followed many of the recommendations you gave, including lowering density, adjusting some of the grass step and subdivision, and reducing visibility distance to 3000.  This has helped a great deal, and the world is now running at about 70-90 fps when not over the island's dense forest.  Over the forest, it lowers slightly to about 50-70, but this is still a massive improvement and a good starting place for further refinement and optimization I think.  

However, one thing I cannot get rid of is the huge delay in how long it takes for the engine to draw the clutter objects.  It can take almost a whole minute sometimes for a tile to show up, all while FPS is around 50-70 fps, sometimes even higher.  I opened the performance profiler and I'm seeing huge spikes in "Render CPU"/"Waiting GPU", which seems like some kind of bottleneck.  I am developing this on a fairly high-end system with a recent model i7, a 1080ti, and Unigine is installed on an SSD, so this leaves me to believe that I am still configuring some option incorrectly.  I've attached a couple screencaps of what I see.  

Any ideas would be appreciated.  Thank you for the help already.  It has improved things quite a bit. 

[silent]

David,

Is it possible to get this scene somehow? That will surely help us to get to the root cause much faster.

You can upload your project  (please zip it with no compression) to our FTP or any other file-hosting service. You can send me PM with a link to the archive.

Our FTP server credentials:

• ftp://files.unigine.com
• user: upload
• password: iejeePheeH3che

Thanks!

[david.norlin]

Will do.  I will PM you when the upload has finished.  Thanks.

[morbid]

Hello David,

thank you for the test scene.

I see that grass cells (impostors) generation really takes a lot of time. This is happening due to a couple of reasons:

• a lot of grass objects in the scene
• grass cell is too big (1600 units) for this terrain

Here are my suggestions:

• The number of grass objects generated by Landscape tool depends on the density of your landcover file. I know, this isn't obvious. So you can do the following trick: use a less dense file for vegetation, I'd say it worth trying 10-15 mppx, and more detailed one for landcover masks (if you need them). There's always an equation with optimal terrain detailing, visibility distance and the number of objects in the scene. You need to find a good working solution for this, as long as I'm not aware of final requirements, I can only make some suggestions for performance optimisation.
• Find a lower combination of step and subdivision for grass objects. Now each cell (1600*1600) has tons of triangles and it takes a decent time to generate the whole cell at one time. It becomes worse when you try to create cells for a number of overlapping grass objects. Probably, 400 units cell will give you a better speed even now.
• Reduce visibility distance for 3rd LOD of mesh clutter. Now you're using 350 units, it should be smaller considering the scale of your terrain.

I have to say I hadn't a lot of time to run deep tests with your content, we're all busy with 2.7.2 release, but all these recommendations should boost the performance. Please, keep us posted with your results.

Thanks!

[david.norlin]

Thank you Morbid.  I will work with this and let you know how it goes.

[demostenes]

It also helps to have impostor texture as full as possible. So try to squeze sprites as much as possible, the less pixels GPU has to filter away, the higher performance it has. This is universal rule for all alfa test/blend surfaces.

[david.norlin]

On 7/23/2018 at 1:55 AM, demostenes said:

It also helps to have impostor texture as full as possible. So try to squeze sprites as much as possible, the less pixels GPU has to filter away, the higher performance it has. This is universal rule for all alfa test/blend surfaces.

I assume you just manually edit the texture? 

[david.norlin]

On 7/19/2018 at 7:48 AM, morbid said:

You need to find a good working solution for this, as long as I'm not aware of final requirements, I can only make some suggestions for performance optimisation.

Morbid,

I have tried many combinations of settings, including your own recommendations, and I am still getting slightly uneven performance over trees (including performance spikes), including the "patchy" loading of clutter and grass objects.  I am willing to accept some of this for now, but I think it is probably just easier at this point to explain what we are going for and show some examples of what we hope to achieve, and perhaps that can guide your recommendations. 

Our flight simulation project is set in central Europe - a large swath of terrain stretching from one end of Germany to the other (parts of Poland, Czechoslovakia, Belgium, France), .  Dense forests are a key part of this terrain.  German forests are very very dense and occasionally quite vast, and we are hoping to portray this, at least to some extent.  Obviously, we must scale back density in order to keep performance high - this is understood.  We must also have reasonably good visibility distance.  For this, we hope to have something in the range of 7-8 miles at least, before they fully fade out of visibility.  Given that these forests will be visible at altitude and from a high speed vehicle, keeping them visible "into the distance" is quite important for both the look of terrain and gameplay.   

I have seen many recommendations for using textures for LOD, and I think this is a good suggestion we will use.  We have also thought about using "blocks" of mesh that fit the shape of the forest to simulate the height and color of trees at great distance - providing depth at low cost (hopefully).  How ever we wind up doing it, the appearance of physical trees is important, especially out to a significant distance, and as you can see from the attached pictures...actually, a lot of the terrain can be nothing but dense forest.  So, this is a significant visual characteristic for us to completely master.  Indeed, it is almost the foundation of our terrain, because of level of coverage and density in the real life location.    

I will continue working with this, and I look forward to the new patch as it seems there major improvements to the impostor creation process.  Thank you for the help so far, and hopefully the description of our project can help provide some more context. 

[demostenes]

On 7/27/2018 at 5:58 AM, david.norlin said:

I assume you just manually edit the texture? 

You can change aspect ratio (grabber tool) of grabbed billboard, to use texture more efficiently. In 2.7.2 billboard system was refactored, but currently it creates lots of empty space between sprites, previous system was much more efficient from this point of view. Unigine team told us, they will do something with it.

You cant just delete empty space, because sprites must correspond to UV mapping of billboard/depends how shader reads it. So for these automaticaly generated billboards you cant do much. If you manually create some BB LOD, it is no problem to do it in most efficient way.

Other advice for vegetation first LODs (leaves, branches...), alfa test is done against geometry, so for same shapes is faster to use sligtly more polygons to better copy shape of texture, because GPU has to fillter away less pixels. This vastly depends on use case, you have to do performance test, what approach is faster. Lots of ugly work, but at the end for objects, which are displayed many times each milisecond is precious.

 

 

 

 

Edited July 30, 2018 by demostenes

[david.norlin]

On 7/27/2018 at 2:21 AM, demostenes said:

In 2.7.2 billboard system was refactored

It makes me wonder if in this new patch this issue will be solved.  If I manually create my own BB LOD, does the shader still detect how many textures there are?  

On 7/27/2018 at 2:21 AM, demostenes said:

alfa test is done against geometry, so for same shapes is faster to use sligtly more polygons to better copy shape of texture, because GPU has to fillter away less pixels

Also, this is an interesting idea about using more geometry for trees.  I noticed that the mesh for the spruce models is mostly rectangles, for all branches.  Like you drew in your picture, this is a large filtering load for the GPU.  Very good thinking. Thank you for this tip.  For now we are using the default spruce models included with Unigine, but we will be doing our own models when the time comes and I will be sure to remember this.

[demostenes]

3 hours ago, david.norlin said:

 If I manually create my own BB LOD, does the shader still detect how many textures there are?  

 

It is parametr in BB material, which can be manually adjusted. But without knowing how exactly is sprite UV mapped on BB it would probably takes lots of attempts to make it fit...