Unigine SDK 2011-11-11

[frustum]

ObjectGrass and WorldClutter will not stall engine on mask loading anymore.

Zone based loading via NodeSector is ready.

[binstream]

Denis, will tree impostors be based on ObjectGrass as in the older city demo or is it planned to use an optimized tree impostor system for blending between 3D trees and billboards ?

As far as I remember there are some improvements.

[ulf.schroeter]

Thanks Alexander for the explanation. So this means that NodeSector child node structures will always be loaded, right ? Is this the final streaming approach ? For large/complex worlds even node structures itself will use far too much memory amounts.

 

PS: maybe its also a good idea to include WorldOccluderTerrain mask to cover all current node types with masks

[alexei.garbuzenko]

frustum, while you are in this topic, please consider one more minor feature request for the next build: when cluster/clutter/grass objects are spawned over terrain, ideal LOD/Flatness values are used (= default height map) that results in visual gaps between objects and terrain when you use higher flatness/smaller LOD values. Same applies to physics and intersections - high curvative terrain surfaces are visually 'relaxed' even at Flatness 50. At the same time using low Flatness and high LOD values gives too much polygons and we can't change them without losing physics and visual precision. Simplest solution would be to apply some smoothing to terrain height map before spawning height-dependent objects and performing intersection checks.

[ulf.schroeter]

NodeSector should provide corresponding unloadMeshes/Textures/Images at least for immediate clearing of queued FileManger file load request in case application logic decides that sector data is no longer required. Otherwise these now unnessessary file request would delay loading of other file requests for newly required NodeSectors.

[steve3d]

why not use another strategy, all those advanced rendering function requires dx10/11, in dx9, those features just has nothing to render, so we can keep dx9 for older hardware.

[frustum]

Thanks Alexander for the explanation. So this means that NodeSector child node structures will always be loaded, right ? Is this the final streaming approach ? For large/complex worlds even node structures itself will use far too much memory amounts.

 

PS: maybe its also a good idea to include WorldOccluderTerrain mask to cover all current node types with masks

 

It's not a final streaming approach. I will add a node which loads piece of world from .node file without keeping a whole nodes hierarchy in the memory in future. Such scenarios requires huge amount of dataset.

 

WorldOccluderTerrain can hide big parts of world which can be visible and streamed while WorldOccluderTerrain loads their heightmap.

[frustum]

Same applies to physics and intersections - high curvative terrain surfaces are visually 'relaxed' even at Flatness 50.

 

I will try to remove Peter panning of grass and clutter nodes.

[frustum]

NodeSector should provide corresponding unloadMeshes/Textures/Images at least for immediate clearing of queued FileManger file load request in case application logic decides that sector data is no longer required. Otherwise these now unnessessary file request would delay loading of other file requests for newly required NodeSectors.

 

Unloading of resources is not always helpful. We can remove a file which is required for different sector. And we will have delay on this resource. We should remove file queries with respect to other node sectors. I will add unloadMeshes|Textures|Mask() function which receives arrays of currently loaded node sectors as an argument.

[frustum]

why not use another strategy, all those advanced rendering function requires dx10/11, in dx9, those features just has nothing to render, so we can keep dx9 for older hardware.

 

I'm trying to keep this strategy for whole rendering pipeline. For example we can use Bokeh DOF on DX10 level hardware, tessellation on DX11 level hardware. Tessellation doesn't require different asset resources and doesn't break the assets pipeline. In same time I can't create stable terrain rendering with up to 4 textures per surface because that requires hardware support of texture arrays and use of different texture assets.

[frustum]

NodeSector should provide corresponding unloadMeshes/Textures/Images at least for immediate clearing of queued FileManger file load request in case application logic decides that sector data is no longer required.

 

The following functions for safe resources removal from the loading queue.

 

void clearMeshes(const Vector<NodeSector*> &sectors);

void clearTextures(const Vector<NodeSector*> &sectors);

void clearImages(const Vector<NodeSector*> &sectors);

[frustum]

frustum, while you are in this topic, please consider one more minor feature request for the next build: when cluster/clutter/grass objects are spawned over terrain, ideal LOD/Flatness values are used (= default height map) that results in visual gaps between objects and terrain when you use higher flatness/smaller LOD values. Same applies to physics and intersections - high curvative terrain surfaces are visually 'relaxed' even at Flatness 50. At the same time using low Flatness and high LOD values gives too much polygons and we can't change them without losing physics and visual precision. Simplest solution would be to apply some smoothing to terrain height map before spawning height-dependent objects and performing intersection checks.

 

The only way to achieve that is to create terrain relaxing tool. Which simulates terrain LOD reducing. Physics problems will be removed also. This tool will be available today.

[alexei.garbuzenko]

 

The only way to achieve that is to create terrain relaxing tool. Which simulates terrain LOD reducing. Physics problems will be removed also. This tool will be available today.

 

Sounds great! What about intersection checks? Are you going to construct relaxed data on the fly according to terrain settings?

[frustum]

The pipeline is ugly:

* export terrain heights map from the editor

* run relax tool: terrainrelax -h 70 -r 32 heightmap.png

* import terrain heights map in the editor

 

The problems with intersections will be gone because all small LOD levels will be relaxed to the upper ones.

[alexei.garbuzenko]

The pipeline is ugly:

* export terrain heights map from the editor

* run relax tool: terrainrelax -h 70 -r 32 heightmap.png

* import terrain heights map in the editor

 

The problems with intersections will be gone because all small LOD levels will be relaxed to the upper ones.

 

That is acceptable solution, thank you!

Also could you think of using several height maps in one terrain to switch between them depending on different quality settings (i.e. I'm going to use lowest detail map on low-end hardware, etc.)? It can be done also by using different terrain objects, but it would take much more space and some more efforts for re-parent grass, clusters and clutters to them..

[frustum]

This is a terrain relax tool. The single command line option is a flatness threshold from the Unigine editor. You can use 1.0 as "Flatness" parameter after height map regeneration. There is a small terrain LOD popping in the current SDK, it will be fixed in the upcoming SDK.

Terrain with increased step is better from the performance point of view. This terrain can be generated automatically via simple script.

TerrainRelax.zip

[alex.dovgodko]

1. We have a lot a of players with low-end hardware. Supporting them is necessary for us. But if we could have an OpenGL solution for low-enders, and DX11 - for high-enders, and our game could run smoothly for both (of course with different graphics settings) - we would be absolutely happy without DX9! Just make it run stably without freezing please.

 

2. "Like" to all that community said and to fast wise reaction of Unigine. Thank you guys for speeding up the streaming creation! Praying that it will fulfill our needs. We had to freeze engine version long time ago to save our developers time on upgrades while the streaming was not ready (we got lot's of engine customizations / optimizations / fixes which require much time to merge to newer versions).

 

3. If the static geometry will run fast in a huge scale open world, then crowd rendering native support will be the next thing every MMO needs. Might be a good choice to include in a next year roadmap? We will be absolutely happy if it will be available only for PC (no mobile and PS).

 

P.S. Trying not to give any advices, just telling my IMHO. I'd bet on positioning an engine as an MMO-specialized. IMHO nowadays single player engines (like UDK/CryEngine/Unity3D) are based on tool set features (complex editiors give a way to make everything fast and easy) much more then on render features. It requires a great amount of work to implement such a tool set, so competing them is a hard choice. But as for the MMO-games - developers got hunger on very special features like: high performance, big seamless worlds, crowd rendering, etc. rather then the ease of use. And known to me solutions are too expensive for small studios.

[alexei.garbuzenko]

This is a terrain relax tool. The single command line option is a flatness threshold from the Unigine editor. You can use 1.0 as "Flatness" parameter after height map regeneration. There is a small terrain LOD popping in the current SDK, it will be fixed in the upcoming SDK.

Terrain with increased step is better from the performance point of view. This terrain can be generated automatically via simple script.

Works like a charm! Thank you!

[Guest shane.ploenges]

On regards to Streaming with node Sector - how is distance of each node to be loaded or unloaded specified? we are currently designing a tiled world approach where all terrain and associated nodes are under a single Node Sector each- but we are unsure how we specify at what distance each of the node sectors is loaded or unloaded from memory in relation to the camera.

 

Also we are developing a tool to import our terrain tiles and automatically place them plus setup all associated nodes and maps for height, diffuse, detail, masks, grass and Cluster under an associated sector node for each one. Our final editor scene will consist of at least 10 x 10 4097 x 4097 tiles.. will this pose a memory problem in the editor attempting this, or will the streaming also work in the editor? If so should we be adopting some sort of strategy for importing and setting up properties for all of these tiles and maps like this?

 

Sorry if these are stupid questions, i am an artist not a programmer - but i am part of the team designing the concept for our world and graphics management and not at work right now so cannot consult with the engineers.

 

Thanks in advance.

[ulf.schroeter]

There is no automatic distance-based streaming with NodeSector implemented at the moment. Actually it has to be handled explicitly by your application via UNIGINE script (so your programmers will be needed).

 

A simple but effective approach might be some NodeSectorManager class controlling the streaming activities of all NodeSectors in the world based on their distance to current editor/player camera. Most simple splitting of your world would be same-sized regular world tile grid layout, where every grid tile has a top-level NodeSector node for streaming control.

 

Such an approach should both work in editor and player mode if NodeSectorManager selects its streaming pivot point during frame update alternatively based on editor or player camera position.

[Guest shane.ploenges]

Thanks for the info Ulf. This sounds reasonable.

 

making a series of tiles all under a world sector node is exactly our plan - which brings me to some information you (and others) may be interested in:

 

Our company has paid for some custom enhancements to the world machine software (which we came across from a recommendation you made in a different post)

 

The next beta will have selective dem tile import, 3d spline adjustment in Layout view and selective tile export so that you can export only the tiles you select from the tile setup window. I can also tell you the new beta has Multiple windows for views : it is a fantastic new enhancement to this great software.

[ulf.schroeter]

Our company has paid for some custom enhancements to the world machine software (which we came across from a recommendation you made in a different post)

 

The next beta will have selective dem tile import, 3d spline adjustment in Layout view and selective tile export so that you can export only the tiles you select from the tile setup window. I can also tell you the new beta has Multiple windows for views : it is a fantastic new enhancement to this great software.

Sounds great.

[ulf.schroeter]

Also we are developing a tool to import our terrain tiles and automatically place them plus setup all associated nodes and maps for height, diffuse, detail, masks, grass and Cluster under an associated sector node for each one. Our final editor scene will consist of at least 10 x 10 4097 x 4097 tiles.. will this pose a memory problem in the editor attempting this, or will the streaming also work in the editor?

 

Just another thought: maybe using a finer tile splitting (e.g. 40x40 tiles with 1x1K texels or even finer) is more appropriate for smooth streaming around viewer position. Just guessing, but allocationg/uploading one or multiple large 4x4K textures per frame might cause rendering stalls.

[Guest shane.ploenges]

This will definately require some testing, i would rather work with as few tiles as possible, however you are right in that using these maps with associated detail, grass masks, detail maps etc could be a problem.

Has anyone alse tried something similar that can comment? Is there any official Unigine comment?

 

Also if we are aiming for say a 3 or 4km view distance, which is quite normal in a train simulator in some cases, don't we need to load the same amount of data, but just in more tiles? I would have thought it better to load less sectors than try load say 25 1k sectors at once?

 

Thanks

[manguste]

Several general recommendations on terrains can be found here.