landscape quality parameter details?

[romain.janil]

Hi, 

would it be possible to have some more details about "quality" param ?  and how does it affect height/imagery/vectors/masks ?:

Object Landscape Terrain

Export quality is from 1 to 100%. The percentage relates to the compression of the source data quality for the output. 100% quality means 0 compression, and reducing the quality value increases the compression.

 

[romain.janil]

A bit related, so I continue in this thread:

what data is actually in .sandworm_cache? 

Is it all the source data loaded in the project and then at generation time, process picks in that data inside Export Area extent ?

Or is it only cached data needed for current Export Area.

In other words, does it make sense to pre-cache all data needed for a big project extent, in order  then to generate various small Export Areas to drive some tests, as opposite of building specific sandworm projects loading source data only related to these small areas.

Don't know if I make myself clear enough...I could elaborate.

 

[bmyagkov]

Hello!

12 hours ago, romain.janil said:

would it be possible to have some more details about "quality" param ?  and how does it affect height/imagery/vectors/masks ?:

Speaking of quality slider it will simply cut source resolution and for example, if you set 50% there, then the original value will be halved (see attache image below)

This rules applies to any raster data such imagery, elevation and masks too. Vector data are not affected and always used "as is"

Using the quality slider is convenient for quick iterations to roughly estimate what the result will be

Quote

In other words, does it make sense to pre-cache all data needed for a big project extent, in order  then to generate various small Export Areas to drive some tests, as opposite of building specific sandworm projects loading source data only related to these small areas.

Well, It's a bit complicated.

If you set polygon for "Export Area" than the cache will be generated only by specified area, that means if next time you decide to move area in some other place - the whole process of generating caches will be repeated for the new location.

And if new selected area partially touch the area where cache has already been generated before, then the data will be reused and speed up generation processes a bit.

You can also not specify areas for generation at all, that means the cache will be created for all sources that were added to the project. Such data can be reused again if you decide to restrict area by using "Export Area"

 

[romain.janil]

Ok thank. Would it be possible eventually that these quality setting could be setable by source type or even more accuratly by source (ie parameter under "file" field?

 

[romain.janil]

One more things: when dealing with very large area and source data at very fine mpx, what's best approach in order not to hit RAM problems or crashes during generation?

Is it possible to generate elevation in one sandworm project, imagery in another, masks in  another, then backup generated .lmaps, and then reassociate these layers  under an ObjectLandscapeTerrain?

[romain.janil]

developing on my last remark in order to find best workflow/dataflow and thinking out loud a bit: when dealing with real world location elevation/imagery won't have to be iterated over much, only in some detail areas.

But more generally masks or vector for details will be iterated a lot . So I guess this is the whole purpose of the landscapeTerrain after all, just want to be sure.

Each layermap has same structure right? Any lmap contains data for height/albedo/masks/settings, they are generated/serialized as individual files only if resolution and density of source data differ? I didn't try that but I guess if I generate with all height/albedo/masks source data at same resolution/extent and mpx density, I should get one .lmap output? 

[bmyagkov]

3 hours ago, romain.janil said:

Ok thank. Would it be possible eventually that these quality setting could be setable by source type or even more accuratly by source (ie parameter under "file" field?

At the moment, you can setup export quality settings only globally. It might be a good idea to do this separately, but this has not been discussed yet. I will add this to the list of  suggestions as well.

3 hours ago, romain.janil said:

One more things: when dealing with very large area and source data at very fine mpx, what's best approach in order not to hit RAM problems or crashes during generation?

Well, I would advise few things and first one it is disabling microprofile feature, you need to run Unigine Editor with external argument "-microprofile_enabled 0" that will save you at least 5-6GB of ram. Check yourself by typing "microprofile_info" from console, it should return "microprofile feature is not compiled" if everything went right.

The second thing is it use a computer with a lot of RAM, preferably at least 32GB, even better 64.

And I would also like to warn you to be careful with vector data. If you suddenly encounter any problems during generation, first of all try to remove the vector layer from the project, because it may consuming a huge amount of memory, since such data does not have its own resolution and uses values from near sources to form its own grid.

Quote

Is it possible to generate elevation in one sandworm project, imagery in another, masks in  another, then backup generated .lmaps, and then reassociate these layers  under an ObjectLandscapeTerrain?

You can try, but most likely, it will be very difficult to bring the data together, especially if sources have different resolution it may requires a lot of time.

Quote

Each layermap has same structure right? Any lmap contains data for height/albedo/masks/settings, they are generated/serialized as individual files only if resolution and density of source data differ? I didn't try that but I guess if I generate with all height/albedo/masks source data at same resolution/extent and mpx density, I should get one .lmap output? 

Yes, if all the source data is located within the same geographic location and have the same resolution, then the output should be one .lmap asset. But as I said above, if source data are different, then it will be difficult to assemble such a project by manually generating the data separately. Closest resolution values will merge into one .lmap asset.

Thanks!

[romain.janil]

Quote

The second thing is it use a computer with a lot of RAM, preferably at least 32GB, even better 64.

all workstations here have 128 GB so I guess problem was elsewhere (not disk space though) (it was an overnight generation test but in the morning Unigine had vanished): by the way is there a command or argument I should enable may be to trace more log or verbose in a text file besides what's output in the console during sandworm generation?

One last thing, for the moment .lmap are not compressed right ? What is the linear order of relation between extent(m)*mpx and lmap size in bytes (or mb or gb :)

? 

[bmyagkov]

12 minutes ago, romain.janil said:

all workstations here have 128 GB so I guess problem was elsewhere (not disk space though) (it was an overnight generation test but in the morning Unigine had vanished): by the way is there a command or argument I should enable may be to trace more log or verbose in a text file besides what's output in the console during sandworm generation?

One last thing, for the moment .lmap are not compressed right ? What is the linear order of relation between extent(m)*mpx and lmap size in bytes (or mb or gb :)

? 

Truth be told there is only /bin/editor_log.txt collects all the information, unfortunately there is no additional tools provided to monitoring generation and we are working to ensure that the logs contain more data about generation in future releases.

Even though I still recommend you to disable microprofile feature for the generation period and prepare data separately, adding layers step by step, since it will be easier to find out what the problem may cause crashes.

Quote

One last thing, for the moment .lmap are not compressed right ? What is the linear order of relation between extent(m)*mpx and lmap size in bytes (or mb or gb :)

Yes, but good news is that we are already testing data compression for the Landscape Terrain and that changes might be expected in the autumn release this year too.

There is no linear dependency here, but you can always get an approximate value by adding a sources to the project and check how much storage will be required before generating on following tab