WorldManager Issues

[ulf.schroeter]

In current WorldManager image management there seems to be some issues and possibilities for render stalls

 

Problem A

 

WorldManager::clear() does not get called on world unload/reload. This leads to ever increasing managed resource entry counts even if resource ref counter == 0 (might also be a problem for upcomming world zone streaming when zones gets paged out, but resource refs remain in WorldManager list).

 

Fix A

 

void World::clear() {
   ....	
   engine.worlds->clear()	// clear world resources

   engine.sound->clear();
   engine.physics->clear();
   ....
}

 

 

Problem B

 

Not a big deal, but WorldResourceImage::update() unnecessarily calls clear() even if already cleared

 

Fix B

 

class WorldResourceImage : public WorldResource<Image> {
....
  virtual void update() {
       if(engine.frame - frame > WORLD_MANAGER_IMAGE_LIFE && isLoaded()) {
          clear();
}
....
}

 

 

Problem C

 

This one is a little bit more complicated. ObjectGrass mask image access frame will only be updated in case of new grass cell creation (via call to getMaskImage() from Grass::create_cell() called by Grass::update_cells()). If player movement does not trigger at least one cell creation within WORLD_MANAGER_IMAGE_LIFE frames, ObjectGrass mask image gets unloaded.

 

On next create_cell() event mask image file has to be reloaded from disk by main thread. As images are currently not loaded by backgroud filesystem thread they also do not get cached in filesystem memory cache, so each time there will be OS file operations. Especially for large mask image files this might cause recurring render stalls.

 

Same problem for WorldClutter where mask image frame only gets updated within get_density() called by create_positions()

 

Fix C

 

Mask image should be 'touched' each frame from within Grass::update_cells() and WorldClutter::spawnNodes() to avoid unload/reload cycles. This at least ensures that mask image stays in memory while player is within object range.

 

Still this does not solve general render stall problem caused by WorldManager image loads from main thread when camera (re-)enters object range. Solution might be using same background loading approach as RenderManager where deferred background resource load gets triggered by resource ptr access during rendering.

 

This would also provide faster loading of worlds with large number of object mask image resources. At the moment all mask image resources will be fully loaded by setHeights-/-MaskImageName during initial world load even if they will be immediately cleared afterwards for all objects not within initial camera radius.

 

For general large world zone streaming such WorldManager background streaming is required anyway. For avoiding object popping due to missing resoures when camera changes orientation (e.g. visible in WorldClutter demo) current camera frustum based resource 'touching' should also be replaced with radial object distance to camera criteria as discussed here and here

[ulf.schroeter]

Any update on these issues ?

[manguste]

Problems A and B are fixed (already included in this SDK update). As for problem C, Frustum will see what could be done about it. Can't give the exact ETA though, sorry.

[manuel.gysin]

Problems A and B are fixed (already included in this SDK update). As for problem C, Frustum will see what could be done about it. Can't give the exact ETA though, sorry.

 

Any news on C?

[ulf.schroeter]

I was already thinking about providing a UNIGINE patch for modified WorldManager and RenderManager resource caching to avoid above problems and also some other render stalls. Nevertheless I am not sure if it is worth the effort or would interfere with UNIGINE activities for required/upcomming more severe re-design for unlimited node-file streaming...

[steve3d]

Frustum said that re-design takes about 1-3 months, so why not provide a patch for this :)

[manuel.gysin]

It would be very nice to have a patch. We have currently everywhere grass!

With mask loaded stalls have to much impact.

[ulf.schroeter]

With some re-thinking found a simple work-around to 'lock' mask images of ObjectGrass or WorldClutter in memory. This will avoid render stalls caused by current WorldManager non-background resource loading. See here

 

Still hoping that we get much more sophisticated streaming support for really unlimited worlds as described above.

[frustum]

I have fixed stalls problem via getting the images in Node::flush() function as you propose.

[ulf.schroeter]

Good idea as a quick-fix. Nevertheless I would expect that for large image resources or multiple mask image loads (e.g. player teleportation) current sychronous file loading triggered by mask_img_ptr.get() still might cause render stalls ? Why not also include asynchronous image loading by FileSystem as implemented in RenderManager and mesh/texture_ptr.render() ?

 

Actually for a more general streaming solution it would make sense from my point of view to implement a single ResourceManager for centralized resource management/background streaming as discussed above and referenced posts. Also distance-based resource pre-caching (not only-frustum-based) is essential to avoid popping on fast viewer direction changes, right ?

[ulf.schroeter]

I have fixed stalls problem via getting the images in Node::flush() function as you propose.

 

Hi Alexander, not sure, but Node::flush() will only be called for nodes marked for update (manually or based on view frustum intersection), right ? If so, render stalls might still occure when nodes with unloaded masks become visible.