romain.janil Posted September 22, 2021 Share Posted September 22, 2021 Hello, I'm facing a major difficulty trying to generate some portion of landscapeTerrain at resolution which weren't problematic with terrainGlobal where lods production was very fast. exemple: "small" 50km*50km extract which is 1/8 of targeted extent... elevation 5mpx (geotif) (tiled) imagery 0.2mpx (jp2) (tiled) 4 masks 1mpx for the finer resolution (geotif) (tiled) for landuse details 1 big mask constant color for background mainly details masked by albedo (5mpx) problem is actually triple: -1 exported data estimated to ~250 GB -2 cache : ~500 GB -3 estimated time of generation (still rough estimation based on previous smaller test) : almost 2 days (maybe even more) on a recent fast workstation (xeon bi-cpu/128 GB, ssd etc) I guess at the moment I should wait for upcoming alpha to get smaller exported data and cache also (hopefully), but what about generation times? Do you have any advice on how to shorter those, what would be reasonable resolution/density that one should use: is 0.20mpx for albedo too fine ? is problem coming from jp2 decoding (for generation times) any advice for masks density ratio compared to albedo? should we use bigger densities (20mpx, 40mpx ...). Thanks for some insights on metrics you could share. Link to comment
silent Posted September 22, 2021 Share Posted September 22, 2021 romain.janil Cache can be quite big due to the initially highly compressed formats (like JP2 in your case). Our cache is already compressed (with fast LZ4 compression for more performance). Exported data (lmap) can be compressed in 2.15 SDK with high efficiency. After compression you can expect storage space required for LMAPs to be reduced (4x-8x reduction), but in some cases we can achieve 100x reduction (mostly due to uniform terrain colors). Generation time should be faster in the alpha version. Your estimations can be also not very accurate, try to generate to see :) Thanks! How to submit a good bug report --- FTP server for test scenes and user uploads: ftp://files.unigine.com user: upload password: 6xYkd6vLYWjpW6SN Link to comment
romain.janil Posted September 22, 2021 Author Share Posted September 22, 2021 hi Silent, ok thanks for inputs, as for generation time unfortunately it's not estimation right now, it's pure observation, I hit the generate button day before yesterday afternoon...hopefully it'll get faster because generation time counted in weeks has to be carefully planned... Do you know if generation is faster with uncompressed source data? Link to comment
silent Posted September 22, 2021 Share Posted September 22, 2021 Quote Do you know if generation is faster with uncompressed source data? Generally it should be, since we don't need to decompress it (as in case of JP2) format. But I can't tell you exact numbers. How to submit a good bug report --- FTP server for test scenes and user uploads: ftp://files.unigine.com user: upload password: 6xYkd6vLYWjpW6SN Link to comment
romain.janil Posted September 22, 2021 Author Share Posted September 22, 2021 that's what I guess too, thanks. Link to comment
Recommended Posts