sandworm : very long landscapeTerrain generation time

[romain.janil]

Hello,

I'm facing a major difficulty trying to generate some portion of landscapeTerrain at resolution which weren't problematic with terrainGlobal where lods production was very fast.

exemple: "small" 50km*50km extract which is 1/8 of targeted extent...

elevation 5mpx (geotif) (tiled)

imagery 0.2mpx (jp2) (tiled)

4 masks 1mpx for the finer resolution (geotif) (tiled) for landuse details

1 big mask constant color for background mainly details masked by albedo (5mpx)

 

problem is actually triple:

-1 exported data estimated to ~250 GB

-2 cache : ~500 GB

-3 estimated time of generation (still rough estimation based on previous smaller test) : almost 2 days (maybe even more) on a recent fast workstation (xeon bi-cpu/128 GB, ssd etc)

 

I guess at the moment I should wait for upcoming alpha to get smaller exported data and cache also (hopefully), but what about generation times?

Do you have any advice on how to shorter those, what would be reasonable resolution/density that one should use:

is 0.20mpx for albedo too fine ? is problem coming from jp2 decoding (for  generation times)

any advice for masks density ratio compared to albedo? should we use bigger densities (20mpx, 40mpx ...).

Thanks for some insights on metrics you could share.

[silent]

romain.janil

Cache can be quite big due to the initially highly compressed formats (like JP2 in your case). Our cache is already compressed (with fast LZ4 compression for more performance).

Exported data (lmap) can be compressed in 2.15 SDK with high efficiency. After compression you can expect storage space required for LMAPs to be reduced (4x-8x reduction), but in some cases we can achieve 100x reduction (mostly due to uniform terrain colors).

Generation time should be faster in the alpha version. Your estimations can be also not very accurate, try to generate to see :)

Thanks!

[romain.janil]

hi Silent,

ok thanks for inputs, as for generation time unfortunately it's not estimation right now, it's pure observation, I hit the generate button day before yesterday afternoon...hopefully it'll get faster because generation time counted in weeks has to be carefully planned...

Do you know if generation is faster with uncompressed source data?

[silent]

Quote

Do you know if generation is faster with uncompressed source data?

Generally it should be, since we don't need to decompress it (as in case of JP2) format. But I can't tell you exact numbers.

[romain.janil]

that's what I guess too, thanks.