height data interpretation in Unigine


photo

Recommended Posts

Hi there,

I've noticed some differences with straight elevation data interpretation  (32 bit float), rasters are geotif encoded with pixel is point not area, between Unigine and other tools I use (eg Cityengine, or when making a Delaunay triangulation or surface drapping with FME), when creating some ObjectTerrainGlobal. On a same geo position, eg at center of an elevation pixel, values of Unigine terrain is sometimes over or under real height values I can't see why. In some similar situation with other engine, this came from the fact data is interpreted as pixel is area, hence the discrepancies. Any hint where to look to ensure consistencies over all the data? Thx 

Link to post

Hello Romain,

We handle all georeferenced data via GDAL. By default, this library uses top-left corner as a coordinates anchor. We had no requests to change this behavior so everything was left as is. Another thing that affects the result is shader. We build terrain mesh with GPU and this process is not fully controllable.

Is it a major issue for you? May I ask for an example?

Thanks.

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to post

Hi Morbid,

this can be a major issue because in my pipeline procedural buildings are not handled with points but as tiled pre-regrouped (or batched or clustered if you prefer) meshes of varying sizes depending on poly density and dimensional aspects (not the same for high features, low features, colored scattered features etc), and of course LOD. For the sake of performance and workflow.Well so basically sometimes buildings are largely under the terrain mesh, sometimes flying, even if source data (shapefile ) are altimetrized onto the same source elevation data (geotif or other georeferenced format). I'm used to this kind of problem so I usually add some geometry under my procedural buildings to fill the gap when they are too much above the ground. This works when the delta is .5 or 1m, but I have some case where the difference is 2 or even 5m or more between what it should be and what it is (same lat long in unigine height sampling and sampling in a GIS tool, off course I'm telling this at LOD0 where terrain equals raster density. I'll try to give you an example on some simple data asap.

One other possibility would be to generate features individually, import  and drap them at the right Z onto the real Unigine terrain and do the regrouping/clustering in unigne but I don't know what it will induce to in terms of performance and workflow usability when dealing with hundreds of thousands of FBXs. 

On that, a side question what is the best strategy in Unigine to load nodes in the background or stream nodes based on player loc (and field of view), I tried the WorldLayer feature some time ago but I don't know if it's still OK to do that in 2.11. 

thanks

Link to post

I think that I mislead you with this.

Landscape Tool applies reprojection to all raster and vector sources and this will always result in a slightly different terrain when compared to the other software.

You can override this with *.prj file. The projection should be similar to the one you've used in City Engine.

You can extract such files with GlobalMapper or QGis from one of the raster/vector sources.

Please check this and write me back on the result.

Thanks!

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to post

Hi Morbid, I 'm already using exact same projection with the same prj so lat long are the same, only Z is somewhat different

I mean buildings are on the right place (when play area and its center are matching with consistency the CE export global offsets, in the same projection)

 

Edited by romain.janil
Link to post
  • 2 weeks later...

Sorry, I lost this thread in a feed.

Can you send me the test content? a couple of meshes and the heightmap with prj file? I'd like to see how it works in the engine.

Thanks.

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to post

Hi Morbid, no problem, I'll try to do that asap, I'm currently moving my office so I'm not fully operational right now :) I'll tell you when samples will be ready.

Link to post
  • 2 months later...

Hi, returning to this old question of mine, however problem's not solved. Here's a little snapshot showing the height interpretation variation between tools: for instance, same lat-long in GM or Qgis gives not exactly same elevation in CE (projection is constant and stay the same between tools, it's an UTM zone with a WGS84 datum, same prj definition is used for the terrain generation in Unigine). So there's a ~1m difference between vanilla GIS tools and how CE interprets the data albeit min-max and pixel size being almost accurately the same, all tools using a bilinear filtering to ramp between pixels. In Unigine, same data, same gaming center, same custom projection, almost same pixel size (which is very big, 38m) so lat-long positioning of imported buildings visually look correct. Anyway, height is totally different in quasi all situations, sometimes over, sometimes under...at this spot it's almost a 5 m delta...so either buildings are flying, or buried into the ground debug_elev.thumb.jpg.b1f6d6fa2f17e62dc965b70bdf74cb9b.jpg

So could this is to do with how Unigne (or GDAL) is set to interpret geotif rasters (pixel is point or area then sampling) ? Thanks for inputs.

R

 

Link to post

Hi Romain,

If I understand you correctly, you are trying to place buildins on a terrain with 38m/px density? Could you please send us an example of prj + landscape sources (geotiff) + building examples (in FBX)?

It would be interesting to see what is causing this behavior.

Thanks!

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to post

Hi Silent, yes but pixel size shouldn't be the problem. (source data is regular 1 arc res SRTM). Anyway, i can't share this particular example. I was in the progress of making another one with some other data elsewhere, all was good till I tried to use a RGF93 Lambert projection as custom proj when I got this in Unigine (which seems to be encoded in WKT2 :) :( (All the data I wanted to build my test on are on this projection.image.png.b852f0ed45ecf0ea55f287917e5a415e.png  So I need to extract a bit of the data and make the proj Wkt1 compatible in the meantime. (Other question would be if it could be possible to support Wkt2 specifications, I thought it would be handled trough Gdal tho, so I don't get why this error happen). 

I get back to my test data package I will send it when it's ready.

 

Link to post

Wkt2 currently is not supported due to the used GDAL version in current Landscape tool. We are currently in the middle of Landscape Tool refactoring process so Wkt2 support is already implemented in the internal dev branches.

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to post

Ok good to know.

I managed to set a little other example in a simple wkt1 UTM proj. So here I build a terrain with landscape too using same data everywhere and trying to workaround precision issues.I've made an archive with the rasters (elev and alb). Center of the playing area is set at centroid of the elevation data, and terrain is generated with the same custom projection (UTM etc using one of the .prj. Buildings are imported with -y as forward axis They are generated with an offset in CE UTM projection corresponding to the center of the playing area, so 0.0.0 of the FBX matches the 0,0,0 of Unigine. This is the case for x and y, but the Z (height is totally wrong). How can I send you the archive (~25 mb) ?debug_2.thumb.jpg.7823e2d6d7ea4bea5a6fff9b5886b597.jpg

Link to post

You can upload your files to our FTP server (login and password are specified in my signature). Btw, which software are using to generate FBX files?

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to post

this is straight procedural generation exported in FBX from CityEngine 2020.0 

Link to post

sorry can't find login and password :(

Link to post

ok I've just uploaded DEBUG_ELEV

Link to post

btw I tried to artificially increase px density at lod0 (manual settings) but it doesn't change anything, terrain is more tessellated but intrinsic data and Z values remain equal logically (and wrong it seems). Another weird thing is I tried to put a dummy node or a sphere or a cube primitive at the bottom of a building to measure the Z but it gives inconsistent results: pivot widget is off  and located under the mesh. So it's difficult to visually tell what the generated height values of Unigine terrain are.image.thumb.png.d3b8814d79ecb969fba9147ccd22dc19.png

Anyway, since both the terrain and the meshes are in the same projection and since the Z is encoded with absolut values into the geometry (checking Z (or Y values since the export is Yup) in maya gives expected results, bottom most vertices are lying around 85.98). It might be a datum interpretation  problem too...

I exported the CE terrain as a mesh too (no simplification, so Z values are a direct regular triangulation of the height raster rows and columns at it's pixel size: image.thumb.png.78a861663774df5876672042fb94e159.png result is correct. 

So my question is how is the source data interpreted in Unigine (not regarding it might be an ellipsoid problem) ? Is there a Z threshold definable besides pixel density? I guessed a triangulation at source pixel density should yield to a regular grid where Z values would match source pixel's centers?  

 

Link to post

Ok thanks a lot.

Link to post

I've also uploaded the terrain mesh in FBX exported from CE as expected reference at Lod0 (ie no Z simplification  from source data) to the FTP.

thanks again

Link to post

Hello Romain,

Thanks for the assets. We can confirm visible discrepancy with mesh and terrain heights. At the moment I can't point out the reason for this. Might be the GDAL feature or some internal specific algorythms (or both). I did a research and filed this case. It doesn't looks like something that can be solved quickly, unfortunately.

The good news is we're working on a revamped Landscape Tool and the team will investigate this behavior to avoid further problems.

What can be done now?

  • I think the easiest way is using the exported mesh. This will work only if you wanted to keep sattelite image as a major texture. I assume you can export mesh tiles with the texture from City Engine.
  • Experimenting with higher quality elevation data. You already mentined that you've tried tesselation with the same source, I guess the result could be different with real data.
  • Combining Unigine's terrain with City Engine's mesh. You can use decal-based holes (available since 2.7.2) to cut out the city area and put a mesh inside with "Terrain Lerp" turned on. You can see the attached image as a reference.

We currently do not have any specialists working with City Engine so any debugging height data from you is welcome.

Sorry for the inconvenience.

image.png

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to post

HI Morbid, thanks for your answers. 

Indeed that's not really a problem for small area when ground is actually done fully procedurally with CE like in dense urban environment (streets, curbs, lots etc are exported as meshes) because I indeed make some holes in the Unigine terrain and use terrain lerp on some part, but majority of the ground is fully synthetic and does not rely on draped imagery.

But more generally I was thinking driving some tests on really larger scales  (let say 200km200km or more, country or peri-urban areas) where terrain should be Unigine and buildings from CE, because I don't want to use terrain meshes but Unigine terrain. At lod 0, there shouldn't be so many discrepancies between source data and generated terrain mesh. I'll produce some more test data with increased resolution and see what it produces, I'll also make some other tests with flat (or tile based) terrain and Landscape Terrain to see if there's some differences with ObjectTerrainGlobal. Maybe it will give some hints either problem comes from Gdal, or a GPU smoothing filter somewhere. 

So let's continue to dig :)

BEst

Romain

Edited by romain.janil
misspelling
Link to post

I'm on 2.11.0.2 (yet editor tells 2.11.0.0)

Link to post