[SOLVED] sandworm problem (again :))

[romain.janil]

Hi there, I've upgraded/migrated  a project from 2.16.1 to 2.18.1. Original world had a landscapeTerrain generated with sandworm in 2.16 all working perfectly (lots of external geotifs etc).

Then I upgraded the content to 2.18 and now I want to regenerate the terrain (to modify or add some imagery with better res), all in same projections as earlier, but it won't work, I can't add any new data to the sandworm project, console keeps saying "

LayerGeo::onPrepare: can't reproject geo transform in layer(xxxxx).

Can't generate preview of layer.

I even tried to re-add one of the raster which works but same thing. as if the project upgrade had busted something. 

Any clue ?

thanks

[romain.janil]

hmm tried also in a new sandworm project...and neither of the previously working data work at all (elevation, img,masks, all in geotif with 2154 epsg projection). Are there some major changes between 2.16 and 2.18 which could lead to this that I'm not aware of ???

[romain.janil]

re hmmm, just tried also in a fresh new unigine 2.18. project with a new sandworm project and no more luck. What's going on ?...

[romain.janil]

and continuing monologue, just tried same data in a fresh 2.16 project, and of course it works there. So basically I can't generate a lanscapeTerrain in 2.18 with sandworm at the moment... Any idea ???

Tried also in 2.17 and it's like in 2.18, I can't add any geotifs (external files) which used to work in 2.16, so something changed between 2.16 and 2.17 apparently...

Edited September 5 by romain.janil

[romain.janil]

so something definitely changed (maybe in gdal lib) between 2.16 and 2.17. None of my previous geotif work as-is like they used to, but if i re-export one of them with gdal (via qgis), I'm able to load it in sandworm. Originally these files were exported from global mapper. For the moment I won't re-export all of them (hundreds of gigs to process) but It might be interesting to look at the differences between 2 files to see why those won't load in sandworm anymore (while they load perfectly well in any gis tool, including gdal)...

However since all the previously added files still work in sandworm (just can't add new ones directly exported from gm), my workaround is to encode new ones with gdal in qgis.

Edited September 5 by romain.janil

[romain.janil]

On another topic, would it be possible to have a per-source data export quality setting ? For the moment, this is applied globally (ie: quality set to 50% export all sources layers to 1/2 pixel density, 5mpx source outputs an lmap of 10mpx, 0m8px gives 1m6px etc ). It'd be cool if this could be applied per fle for faster iteration... 

[bmyagkov]

Hello Romain,

We're sorry to hear that you're facing this issue. Based on our research there haven't been any changes to the import system for quite some time. Even our test assets have remained unchanged for a long period. The first thing that comes to mind is a issue we discussed earlier covered in this topic: https://developer.unigine.com/forum/topic/8478-solved-sandworm-crash-when-create-a-new-project

In short it might be linked to the "GDAL_DATA" or "PROJ_LIB" variables set in your system's "Environment Variables" which could be affecting this behavior. Could you please check these parameters on your end first?

If that doesn’t seem to be the cause it would be helpful if you could share any of the tiles you're having issues with so we can try to reproduce the problem on our side.

However, since SDK 2.18.1 was released quite some time ago and we haven’t found any similar issues mentioned in our internal bug tracker we believe this is likely a local problem that we need to troubleshoot together.

Thanks!

[romain.janil]

For sure I need and have a GDAL_DATA system variable set since I use Postgis and postgres for other projects, but no PROJ_LIB variable. I'll temporary delete the first one and see if it solve the problem...however these might be problematic for correct use of postgis, I'll see.

[romain.janil]

just had a quick test removing GDAL_DAT var, reboot, and tried again but the issue is still here... I'll send you some data to test on your side when I won't be in such a hurry next week.

[bmyagkov]

1 hour ago, romain.janil said:

just had a quick test removing GDAL_DAT var, reboot, and tried again but the issue is still here... I'll send you some data to test on your side when I won't be in such a hurry next week.

It's unfortunate that it didn't help. However, the behavior seems unusual and should be relatively straightforward to address. The key now is for us to identify the root cause. We'll be waiting for the sources to investigate further.

Thanks!

[romain.janil]

sure I'll extract a small data set asap (ie: a geotif that won't load and another ok one, in sandworm 2.17/18 (haven't tested in 219)

[romain.janil]

Hi just uploaded on the ftp an example of one geotif which used to work in sandworm 2.16 but can't be added as source in 2.17+ (proj is 2154), could you tell me if you can retrieve it ?

[bmyagkov]

Hello!

10 hours ago, romain.janil said:

Hi just uploaded on the ftp an example of one geotif which used to work in sandworm 2.16 but can't be added as source in 2.17+ (proj is 2154), could you tell me if you can retrieve it ?

Yes, thank you for the additional data. We have successfully downloaded and shared it with our colleagues from the QA department to determine the cause of this behavior. This may take some time but we hope to get back to you with an answer soon!

Thanks!

[silent]

It appears that the issue is caused by the new GDAL version 3.6.2 (Proj 7.2.1); it cannot properly reproject your files. In version 2.16, we used GDAL 3.2, which functioned without any problems.

It is likely a regression in GDAL itself, and tracking it down and fixing it quickly could be quite problematic. We will keep an eye on GDAL updates and search for the alternative options.

As a quick solution, you can try reapplying the projection to your *.tif files.

In GIS you can do:

• QGIS -> Raster -> Assign projection...  -> Desired CRS = EPSG:2154 ->  Run

or use GDAL/OGR console call:

• gdal_edit.py -a_srs EPSG:2154 ***.tif

Please apply this to all your sources, and Sandworm should be able to read them again :)

Thanks!

[romain.janil]

Hi Silent, yes that's what I suspected as I also tried the manip in QGIS (reprojected one file with latest GDAL version and it was "accepted" by sandworm (and behind the hood GDAL 3.6.2)). Thanks for investigating around this issue. I'll batch process all the sources.

[silent]

Not a problem at all.

You're very welcome! :)