2.14 Sandworm crash (or hang) when "a lot" of tiles are added


photo

Recommended Posts

Hello,

I almost always get Sandworm to hang ("process not responding") and eventually crash when "a lot" of tiles are added (around 200). I understand that loading a lot of tiles with a high resolution can be a lengthy operation, but there could be some workaround to implement:

  • don't actually load the tiles at the full resolution but limit to maybe some configurable values (or even just load the bounds)
  • load only a few tiles and then just the bound (or only load them when they are clicked in the view?)
  • allow sub-projects in sandworm, so we can work on a small subset at a time. As we can have a manually fixed origin now, this can maybe be done already?
  • at generation time, only load the relevant tiles, and then unload them as soon as possible

With the current state, I can only work with small insets (maybe 50x50km) when our goal is more or less 200x200.

Please note that when I'm lucky to get "all" tiles loaded, I often get a crash at generation time, wasting a lot of time :(  Hence I resolved to only work on small subsets, but I don't know how to merge all these subsets into one large terrain.

Thanks!

Link to post

Hi Stephane,

Indeed there are a number of problems which we have been known and mention this in the release notes at "known issues" part. Can you specify what extension you are uses with that 200 tiles? Is that JP2 or ECW?

Many difficulties appeared after the implementation of ECW and JP2 support related to the internal structure of the GDAL3 library which we use "as is"

Crashes are most likely connected due to the fact that the RAM is out, since all Imagery & Elevation data continue to be stored in memory and even if we are completely turn off display the preview, problems will still remain at the generation stage and that part much more difficult to solve at this moment

Now we are working on a solution to make it possible to work with this type of data comfortable and fast, but as you correctly noted, the operations will still lengthy since the data sizes are actually large. We hope all changes will be ready for the next hotfix update.

Thank you very much for the suggestions, will see which of this can be used to improve the behavior.

Link to post

Hi,

I'm currently using ASC for elevation and JP2 for imagery. I can regenerate into other formats but it's a loooong operation. And geotiff is so much heavier compared to JP2. Which formats would you advise?

I think the best way would be to define small sandworm projects and a "master" sandworm that would reference the others (viewing bounds only). This way we could update/regenerate only part of the terrain, and each would create it's own layermaps. Maybe there is a way to already do that somehow? (multiple worlds and copy-paste layers?)

As a sidenote, can you confirm we can have a Landscape as large as 200x200 with 5mppx density (elevation)?

Thanks!

  • Like 1
Link to post
3 hours ago, Amerio.Stephane said:

I think the best way would be to define small sandworm projects and a "master" sandworm that would reference the others (viewing bounds only). This way we could update/regenerate only part of the terrain, and each would create it's own layermaps. Maybe there is a way to already do that somehow? (multiple worlds and copy-paste layers?)

This is more likely sounds like a feature request and it will definitely not be done in the current release but may appear in the future updates after some discussion and design. Thanks!

Quote

And geotiff is so much heavier compared to JP2. Which formats would you advise?

The thing is that the JP2 are very well compressed and we do the work to decompressing them on our side while generate things like preview or terrain itself where geotiff are ready to go out of the box

I made my own research and measured difference between those two file formats:

JP2 was converted to geotiff using QGIS and its size was 1,8GB after decompressing from 100MB source

1 ASC tile (5m/px) + 1 JP2 tile (1m/px) generated on 20 km2 area size will allocate ~10gb RAM and takes ~4 min approx. (Running or Ryzen 9 3900X 12c24t)

1 ASC tile (5m/px) + 1 geotiff tile (1m/px) generated on 20 km2 area size allocate ~4gb RAM and takes ~2 min approx. (Running or Ryzen 9 3900X 12c24t)

So, until we improve the processing and interaction with compressed data like JP2 I would personally suggest to go for geotiff since task will be done much faster and will require less RAM which is good. As a bonus - preview image appears immediately after adding sources to the project.

(also I would suggest to turn off microprofile before generation by using starting argument "microprofile_enabled 0" this will save you extra 7-8gb of RAM)

3 hours ago, Amerio.Stephane said:

As a sidenote, can you confirm we can have a Landscape as large as 200x200 with 5mppx density (elevation)?

Should not be a problem. But imagery .lmap asset might requires a lot of disk space because does not have compression at this moment. With 20 km^2 area in example above it was 3,7GB for imagery asset and for elevation it was only 100MB.

Thanks!

  • Like 1
Link to post