[Solved] enabled node not always rendered

[Coppel.Damien]

Hello,

I am trying to add and remove fbx imported models from my scene but it doesn't work. I have a vector of NodePtr called "models" containing the models I want toe handle in my scene.  I am simply calling this code every AppWorldLogic::update() :

int idx = update_ctr % models.size();
int prev_idx = (update_ctr - 1) % models.size();
models[prev_idx].getNode()->setEnabled(false);
models[idx].getNode()->setEnabled(true);
models[idx].setPosition(targetCenter);

The goal of this code is to simply test if I can add one model and remove the previous one each frame. With this code there is 0 model that gets added in the image in the first X frames (X value varies). After that, it sometimes work in the render display but models are not rendered in the screenshots I save using

viewport->addCallback(Render::CALLBACK_END, MakeCallback(this, &AppWorldLogic::image_ready_callback)); // In the AppWorldLogic::Init

viewport->renderTexture2D(camera, texture); // In AppWorldLogic::update

void AppWorldLogic::image_ready_callback(Renderer *renderer) {
    ImagePtr screenshot_image = Image::create();
    texture->getImage(screenshot_image);
    if (!Render::isFlipped())
        screenshot_image->flipY();
    screenshot_image->convertToFormat(Image::FORMAT_RGB8);
    if(saveImage) {
        std::string path = imageDir + "_" + std::to_string(ctr) + ".png";
        screenshot_image->save(path.c_str());
    }
}

 

The render seems extremely inconsistent :

In this saved screenshot an Helicopter should appear on the heliport (it is displayed on UNIGINE Engine window).

Also, in the first rendered frames after launch the image is not fully rendered (no sun light) and heliport is blurred (heliport is a decal).

Do you have any suggestion to improve my results and consistency of the saved screenshots ?

Let me know if it is not clear.

 

Edited October 8, 2020 by Coppel.Damien
solved

[silent]

Hi Damien,

I would recommend to use a dedicated Viewport to render models for the screenshots. You can also set the SKIP_STREAMING flag so the whole model will be loaded (including the textures and materials) before the frame starts to render.

Inconsistency probably caused by the async streaming delay. Generally speaking, you can't expect that model will be fully loaded the next frame.

If your main purpose is not to generate screenshots with the specific models loaded - could you please give us more details about your use-case and what you want to achieve?

Thanks!

[Coppel.Damien]

My goal is to create datasets of images for training neural networks. My dataset needs to be as diverse as possible. The images must contain objects (added models) that may be detected by a neural network. I move the camera, change the lights, the models and several other parameters in the scene, each frame.

I don't need real time streaming but I would like to have screenshots of the best possible quality.

I already use a dedicated Viewport (I think ?).

// In the init
texture = Texture::create();
texture->create2D(img_width, img_height, Texture::FORMAT_RGBA8, Texture::FILTER_POINT | Texture::USAGE_RENDER);
viewport = Viewport::create();
viewport->setSkipFlags(Unigine::Viewport::SKIP_STREAMING);
viewport->addCallback(Render::CALLBACK_END_AUXILIARY_BUFFER,
	MakeCallback(this, &AppWorldLogic::gbuffers_ready_callback));
viewport->addCallback(Render::CALLBACK_END,
	MakeCallback(this, &AppWorldLogic::image_ready_callback));

// In update
viewport->renderTexture2D(camera, texture);

The auxiliary buffer is used to generate a 2D mask of every added object in the image (Masks are used as labels to train neural networks). I added the SKIP_STREAMING flag as you suggested although I didn't really see any difference in the results yet.

Thank you for your time !

[Coppel.Damien]

It works fine if I don't change the models

Is there a way to force the complete render of the frame including the new enabled model(s) ?

[silent]

Normally SKIP_STREAMING is enough to render a single object. Have you tried to add more delay between node setEnabled and image grab?

If this doesn't help, could you please send us a minimal working test scene (with couple of models that are not loading and other scene paramters are being changed)?

Thanks!

[Coppel.Damien]

I tried to swap back and forth between 2 objects and it doesn't work even with the SKIP_STREAMING flag. Adding delay between setEnabled and image grab is not really practical because I need to know exactly what's on my image to label it.

I could update the scene (setEnabled) every N frame and save a frame only every (N - 1) frames with N being the delay. I don't like this solution because the delay would be completely empirical and the problem could reappear if I change more things it the scene.

I find it weird to no be able to force the full render and to have to wait frames to render a good image. I will try to send you a test scene tomorrow.

[Coppel.Damien]

I added a few frame of delay before saving images after a scene change.

I also added a first step of initialization in which each of my models is rendered individually in one frame. Some models were never rendered even after several frames when I enabled them for the first time before I added this step.

But it seems to work fine now. Thank you for your answers.