Jump to content

[SOLVED] Retrieve "complete mesh data" from terrains


photo

Recommended Posts

Hello UNIGINE team,

 

in order to get my navigation mesh plugin up-to-date I need to rasterize the complete terrain data and make my calculations on top of that. So far I started with ObjectTerrainGlobal and get stuck on some basic ideas behind the new system and improving overall performance to maximize processing speed. For the old terrain-system I am getting the triangle-data by following piece of code:

Mat4 objectTransform = terrainPtr->getWorldTransform();
float step = terrainPtr->getStep();									//get the step-size of the terrain

BoundBox tempBoundBox = terrainPtr->getBoundBox(0);					//getting the bounding box of first surface == the actual terrain data
tempBoundBox.setTransform(terrainPtr->getWorldTransform());


for (int i = startX; i < endX; ++i)
{
	for (int j = startY; j < endY; ++j)
	{
		Vec3 v0,v1,v2,v3;

		v0 = objectTransform * Vec3(step * i, step * j, terrainPtr->getHeight(i, j));
		v1 = objectTransform * Vec3(step * (i + 1), step * j, terrainPtr->getHeight(i + 1, j));
		v2 = objectTransform * Vec3(step * i, step * (j + 1), terrainPtr->getHeight(i, j + 1));
		v3 = objectTransform * Vec3(step * (i + 1), step * (j + 1), terrainPtr->getHeight(i + 1, j + 1));

		/**
		Further processing of our terrain data
		**/
	}
}

 

Thats, (of course) not that easy any more for the new terrain systems. I created a new terrain (ObjectTerrainGlobal) from the "Arbitrary Terrain Generation" tutorial and followed the video tutorial. During my code-implementation the following questions raised:

  1. The overall bounding information from the object didn't work. Neither getBoundBox() nor getWorldBoundBox() returns a proper result and seems to be initialized with default values. Therefor I tried to proceed with getting/calculating the bounding box from the tileset data. I noticed, that only the height_lods parameter have proper values set for bounding-box information of each tile in the tileset-list. The other ones (normal_lods and masks_lods) have default parameters as well. For the height_lods I can only get the bounding box information on the last list-item (least accurate one). The previous ones all have smaller bounding-values than the overall terrain size. Is this intended and if so, I am more than interested for the details :). Second, are there also an more easier way to get a proper bounding box value?
  2. For retrieving actual height-data the function fetchTopologyData() seems to work but I am concerned about performance. Is this the only way to get proper height data from the terrain? In order to get ALL triangles from the terrain I am afraid to use that function thousands/millions of time.
  3. While getStep() is not available any more I guess I have to retreive the smallest density-value of my TerrainGlobalLod-objects of the terrain, right? Because for calculating a navigation mesh I am interested in getting the most accurate mesh data of my terrain.

 

Regarding the implementation of ObjectLandscapeTerrain I am considering the same amount of complexity fo the implementation. Retreiving height data not only from the standard one but also for their insets (LandscapeLayerMap) seems to increase the complexity as well. Also, using the LandscapeFetch-class is the only way to get proper height data? In case of performance implementation are there also another way to deal with it?

I hope my questions are not overwhelming but will help speed up my implementation for sure. :)

 

Best regards and many thanks in advance

Christian

 

 

 

  • Like 1
Link to comment

Hello Christian,

Both TerrainGlobal and new LandscapeTerrain objects are drastically different when compared to the old ObjectTerrain, I assume you already discovered this :)

The problem with bound box is, more likely, unsolvable, because they are designed to be streamed on demand and have no "real" limits. Each frame engine streams tiles that were caught in the frustum, so the engine doesn't know where's the terrain's borders.

You can use API (fetchTopologyData() for TerrainGlobal and LandscapeFetch) to get height data and build a mesh, but it can cause major performance drops.

Can you describe the general approach you want to use with the navigation, so I can discuss it with the team?

Thanks.

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to comment

Hi morbid,

 

thanks for your answer. Already thought that there wouldn'be an easy way to solve that problem. ;) I created an plugin for UNIGINE to fully support dynamic navigation meshes, based on Recast/Detour (old thread here: https://developer.unigine.com/forum/topic/3034-recastdetour-meets-unigine/). It got quite some attention and there were and currently are interests from various UNIGINE customers to have that work with UNIGINE Engine 2.0. That's why I reworked the system and upgraded my code base (API for now, Editor integration later). So far my migration went pretty smoothly and I did some improvements and performance optimizations.

The overall idea is the following: You can add any mesh you want to my NavMeshBuilder and it automatically generate an navigation mesh for traversing any kind of agents through the world. While not only working in editor mode you are completely free to change your navigation mesh at runtime. Therefor I need to get access to every mesh data of the supported object. While getting mesh triangles from ObjectMeshStatic and ObjectMeshDynamic is pretty easy, current terrain system isn't. :)

A dynamic navigation mesh is split up in multiple chunks, layed out as a 2D-grid (same as Tileset used for the terrain). When adding an object to the NavMeshBuilder, the system have a first look with chunk will cover the newly added mesh and (if too small) extend automatically the 2D grid. After that, it did some processing steps, where only the necessary triangles are cached. When a chunk is build (a navigation mesh tile is generated), the system rasterizes all the cached triangles and a proper navigation mesh is calculated.

Back to the supported terrain system: I first need to calculate the overall terrain size and check, if the bounds fit with the NavMeshBuilder. If not, than I need to enlarge the mesh. That's why I need access to the objects bounding box. In this specific case I don't care if all the terrain data are streamed in or only a part of it. Thats only crucial for the second part, the processing. Here I need access to the whole mesh data (with height information) to rasterize the landscape triangles successful.

 

If needed I can share with you the current version of the plugin. You can also contact me directly (I guess silent still have my telegram and skype information. :)).

 

Edited by christian.wolf2
  • Like 1
Link to comment

Hi,

I've got an update on your question from the engine team. Unfortunately, there's still no easy way, and we can' help you much from the API side.

You need to design your own mesh builder for TerrainGlobal and LandscapeTerrain. More likely both should be processed offline: build a mesh once and use it for navigation purposes. Real-time processing will take too much time and will drastically drop the frame rate.

We did something similar for collision detection, but our approach didn't require bound box or a full mesh. Engine creates a simplified small mesh each time the collision must occur. In general, we can't afford mesh approach with new terrain systems because real geometry will drain memory too fast.

As I understood, you already found a method for TerrainGlobal. More likely, you'll have to optimize the resulting mesh, depending on its density and size.

In case with LandscapeTerrain two methods can help find you bounding box: 

LandscapeLayerMap::getExtremumHeight

LandscapeLayerMap::Size

First will give you min/max height values, the second — map size. With this you'll be able to get a real bounds of the object. Then you can go with LandscapeFetch.

As I mentioned above, this looks like an offline task, because retrieving mesh in real-time just wouldn't work for these objects.

Thanks.

  • Like 1

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to comment

Hi morbid,

many thanks for the reply and your detailed answer. Building an partial mesh, like for your collision approach is definitely what I am looking for, but many parts for the complete mesh. :)

Quote

As I understood, you already found a method for TerrainGlobal. More likely, you'll have to optimize the resulting mesh, depending on its density and size.

After describing that topic more detailed last week I tried to implement it with that approach but there are some API-limitations for that moment.

grafik.thumb.png.30ce7595707c3d4fefc9c67958e100a1.png

For the ObjectTerrainGlobal-object there is internally an variable called max_height" stored (green box). Should be same as LandscapeLayerMap::getExtremumHeight(), right? But exposing this value via an function is currently not provided. Same with bounding_box values for the TerrainGlobalLod (green box). It would be nice to expose those one as well.

Anyway, two additional questions raises during more investigations. In your first post you have written, that the terrain tiles are streamed in and out and therefor the bounding box-related problem can't be solved easily. While understanding the streaming part I don't really get it for bounding box. If I am following the workflow with the landscape tool, overall input data (height texture, albedo and masks) are limited by the user input. For example I tried the arbitary landscape tutorial and have to set "Area settings" in the parameter tab. So for my current understanding the values can be calculated and transformed into engine coordinates. Are there use-cases where input data is not that trivial and not known during setup? Also, populating that large world with other meshes like static meshes, clutters and cluster must be positioned and scattered as well, and for them bounding boxes exists. Sorry if I am asking so much, but just want to understand the design choice behind that one. :)

Second question, if those tilesets are not loaded to the system, fetchTopologyData() still gets the right values, because it loads the data to memory, right? Is it unloaded right afterwards or will those data still be available some time for other uses? Because calling this function hundreds of time to generate my simplified, small mesh will still be slow when loading and unloading those data during each call.

 

Thanks again for answering the question and looking forward to this one. Much appreciated. :)

Best

Christian

 

EDIT: Maybe some more explanation details for regarding the "offline processing". I did the same thing with the old terrain system. Therefore the implementation is split up in two parts: process() and rasterize(). In the first function the algorithm fetches all vertex and index data from the object that needs to bei considered for mesh calculation on later use. Second function pushes all the needed triangles to the mesh builder and (with some magic) you retrieve an robust navigation mesh. Different UNIGINE objects require different handling. For example static and dynamic objects are more performance friendly fetching indices during process() and only getting vertices triangles in rasterize(). For terrains it is more efficient to get start and end index of each chunk and in rasterize() calculating detailed triangle data with two iterations and retreived height information. So, I actually do "offline processing" only during this step but discard those information right afterwards. The only thing is, I need exact start and end information of the whole terrain and the most detailed density-parameter.

Hope this makes something a little bit more clearer!

 

Edited by christian.wolf2
Link to comment
On 3/30/2020 at 10:47 PM, christian.wolf2 said:

For the ObjectTerrainGlobal-object there is internally an variable called max_height" stored (green box). Should be same as LandscapeLayerMap::getExtremumHeight(), right? But exposing this value via an function is currently not provided. Same with bounding_box values for the TerrainGlobalLod (green box). It would be nice to expose those one as well.

These values are correct only for the streamed chunk of data, it can't be used for the whole terrain. I guess that's not what you're looking for.

On 3/30/2020 at 10:47 PM, christian.wolf2 said:

While understanding the streaming part I don't really get it for bounding box. If I am following the workflow with the landscape tool, overall input data (height texture, albedo and masks) are limited by the user input.

You, as a user, do know real limits of the terrain (texture size and density allows you easily calculate this). But the engine doesn't need this data. It just loads data from the tileset as long as there is some data. Knowing texture size of a map for LandscapeTerrain might be helpful here.

And I have another remark on TerrainGlobal. This object is near its end. We're going to replace it partially with LandscapeTerrain and partially with planet (in development). I'd say, it'll be deprecated by the end of 2020 or the beginning of 2021 if nothing will drastically change in our plans.

On 3/30/2020 at 10:47 PM, christian.wolf2 said:

Second question, if those tilesets are not loaded to the system, fetchTopologyData() still gets the right values, because it loads the data to memory, right?

After you do this request data is being cached. Cache keeps recently used data only.

On 3/30/2020 at 10:47 PM, christian.wolf2 said:

EDIT: Maybe some more explanation details for regarding the "offline processing". I did the same thing with the old terrain system.

I wanted to say that this task (retrieving terrain mesh) doesn't sound like a something possible in real-time. If you already knew this, I hadn't added anything new :)

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to comment

Hi morbid,

 

I just want to update you about this topic again.

 

On 4/1/2020 at 12:10 PM, morbid said:

And I have another remark on TerrainGlobal. This object is near its end. We're going to replace it partially with LandscapeTerrain and partially with planet (in development). I'd say, it'll be deprecated by the end of 2020 or the beginning of 2021 if nothing will drastically change in our plans.

That was a good hint for me to drop the current integration of TerrainGlobal for this plugin. :) So I went straight to LandscapeTerrain and the implementation of this one was very easy. Performance is still okay, even with forcing the LandscapeFetchData to get his results immediately. The rasterization highly depends on the input data, but choosing an larger value than the terrain geometry polygon size speed up the process and still get great results.

grafik.thumb.png.9943a9f886d52bebd1b83cc0cffafbf6.png

 

Also, a good thing to add is that I was able to distribute calculation on multiple threads using your Thread-class wrapper. So performance in this demo world was mainly about 200-300 FPS while multiple threads building the navigation mesh asynchronously. There is still some more work to do but all in all the intial problem could be solved.

Many thanks to you!

 

Best

Christian

 

  • Like 1
Link to comment
  • morbid changed the title to [SOLVED] Retrieve "complete mesh data" from terrains

Thanks silent. Will soon update that post again with the current state of developement. Currently switching to full multithreaded support and preparing some demos.

  • Like 2
Link to comment
×
×
  • Create New...