[NoesisGui] Integration of GuiObject-like feature

[bertrand.remondin]

Hi everyone,

 

I'm having quite a problem with the integration of Noesis Gui, a UI framework. Here is how it works in a DirectX11 application (it's very similar in OpenGL):

• Initialize the NoesisGui object and update it each frame (no problem with that)
• At render time:
  • Set the RenderTargetView
  • Pass a draw call to the NoesisGui object

The NoesisGui framework will then render on top of the RenderTargetView, which can be the backbuffer (for HUD elements) or another texture. It is especially usefull to draw UI in 3D scene (in a Unigine::ObjectGui fashion): the texture can be used as ShaderResourceView to texture a mesh.

 

My problem is that I cannot find a correct way to pass that ShaderResourceView to Unigine in order to use it in a 3D scene. I've found this post, showing the integration of Coherent UI but what it does is copying the UI element texture twice so that it can be used as Unigine::Image2D by the engine. That seems dramatically sub-optimal to me and I was hoping I overlooked some part of the documentation that would prevent such manipulation.

 

Is there a better way to perform what I'm trying to achieve?

 

Thanks,

[unclebob]

Hey Bertrand!

 

On C++ side you're able to use Unigine::Texture class, it allows you to even get a ShaderResourceView and pass it somewhere like NoesisGui. After that you can assign that texture to material (https://developer.unigine.com/en/docs/1.0/cpp_api/reference/api_material_class#c_setImageTexture_int_constTexturePtrn) or widget like WidgetSprite https://developer.unigine.com/en/docs/1.0/cpp_api/reference/api_widgetsprite_class#c_setRender_constTexturePtrn_int).

[bertrand.remondin]

Thank you for your quick answer!

 

Here is what I've done so far:

ID3D11Device* pDevice = static_cast<ID3D11Device*>(Unigine::App::get()->getD3D11Device());
            ID3D11DeviceContext* pContext = nullptr;
            pDevice->GetImmediateContext(&pContext);
            if (pContext != nullptr)
            {
                ID3D11RenderTargetView* pTexNoesisRTV = static_cast<ID3D11RenderTargetView*>(pTexNoesisPtr->getD3D11RenderTargetView());
                //pContext->OMSetRenderTargets(1, &pTexNoesisRTV, nullptr);
                const Noesis::Gui::RenderCommands& commands = pRenderer->WaitForUpdate();
                if (commands.offscreenCommands != 0)
                {
                    pRenderer->Render(commands.offscreenCommands.GetPtr());
                }
                pRenderer->Render(commands.commands.GetPtr());
                //pContext->OMSetRenderTargets(0, nullptr, nullptr);
            }

These lines are just after the Unigine::Engine::render call. There effect is to render the UI element on top of the viewport. But as soon as I uncomment the lines 7 or 14, only the first frame is rendered correctly. I would have thought that the render targets would be set again at the next Unigine::Engine::render but I'm probably wrong.

 

Do you see how I could solve the problem?

[unclebob]

Hey Bertrand,

 

When you set your render target you should take care of what was before. That means, you need to get old render target if it was set, then set your render target and after rendering set old render target.

 

Don't forget to deal with depth stencil part as well!

[bertrand.remondin]

Hi Bob,

 

Again, thank you for your quick answer.

 

It might be working but I'm having a hard time verifying the result. In order to confirm that feature once and for all, I would like to print the resulting texture in a file. Here are some code snippets.

 

Just after Unigine::Engine::render:

ID3D11Device* pDevice = static_cast<ID3D11Device*>(Unigine::App::get()->getD3D11Device());
ID3D11DeviceContext* pContext = nullptr;
pDevice->GetImmediateContext(&pContext);
if (pContext != nullptr)
{
    ID3D11RenderTargetView* ppOldRtv = nullptr;
    ID3D11DepthStencilView* ppOldDsv = nullptr;
    pContext->OMGetRenderTargets(1, &ppOldRtv, &ppOldDsv);

    ID3D11RenderTargetView* pTexNoesisRTV = static_cast<ID3D11RenderTargetView*>(pTexNoesisPtr->getD3D11RenderTargetView());
    pContext->OMSetRenderTargets(1, &pTexNoesisRTV, nullptr);

    const Noesis::Gui::RenderCommands& commands = pRenderer->WaitForUpdate();
    if (commands.offscreenCommands != 0)
    {
        pRenderer->Render(commands.offscreenCommands.GetPtr());
    }
    pRenderer->Render(commands.commands.GetPtr());

    pContext->OMSetRenderTargets(1, &ppOldRtv, ppOldDsv);
}

I think Noesis does its job, if I don't change the render target, I can see my UI element on top of the viewport, which is what is expected.

Just after the main loop, when closing the program:

Unigine::ImagePtr pNoesisImage = Unigine::Image::create();
if (pTexNoesisPtr->getImage(pNoesisImage) == 1)
{
    pNoesisImage->save("Noesis.DDS");
}
pNoesisImage.clear();

The file is created and has the correct size, but it seems completely blank (RGBA=(0,0,0,0)). I suspect my understanding the link between Texture and Image, but I'm not sure. Can you see something wrong in my code?

 

Thank you again.

[unclebob]

Bertrand,

 

You must create Texture with USAGE_RENDER flag in order to use the texture as a render target.

 

If that won't work, try to use Unigine::TextureRender instead of Unigine::Texture. That way you don't have to expose low level DirectX stuff, just call TextureRender::enable before Noesis rendering and TextureRender::disable after.

[bertrand.remondin]

I've tried both methods and succeeded with both, thank you very much once again :)

[unclebob]

Awesome!

[bertrand.remondin]

Hi!

 

I have another question about the integration.

 

The mouse cursor is drawn before the Noesis Gui and as a result, it is displayed behind the actual UI.

 

Is there a way to delay its drawing?

 

Thanks,

[unclebob]

Hi Bertrand!

 

Consider rendering noesis gui before everything. :)

[bertrand.remondin]

Hi,

 

Alright, I hoped for some lazy solution to that problem.

I guess I can render to a WidgetSprite and attach it to the Unigine Gui, it should do the trick.

 

Edit: It works, indeed. :)

 

Thank you!

[bertrand.remondin]

In fact, I've just noticed it is not working exactly as expected.

 

I cannot find a way to clear the Texture displayed in the WidgetSprite each frame. I've found a clear method in TextureRender and Texture. The first one makes the program crash as soon as the TextureRender instance is used. The second prevent the UI from being displayed at all. I guess that's not the way to do it.

 

Here is the way I'm creating the TextureRender, Texture and WidgetSprite, maybe there's a problem with that:

_fullscreenTexture = Unigine::Texture::create();
_fullscreenTexture->create2D(screenWidth, screenHeight, Unigine::Texture::FORMAT_RGBA8, Unigine::Texture::USAGE_RENDER);
    
_fullscreenTextureRender = Unigine::TextureRender::create();
_fullscreenTextureRender->create2D(screenWidth, screenHeight, Unigine::TextureRender::DISCARD_DEPTH);
_fullscreenTextureRender->setColorTexture(0, _fullscreenTexture);

_hudWidget = Unigine::WidgetSprite::create(Unigine::Gui::get());
_hudWidget->setRender(_fullscreenTexture);

Then I render the UI each frame in _fullscreenTextureRender.

 

Can you see what the problem might be?

Thank you again!

[unclebob]

Hello, Bertrand!

 

TextureRender::clear and Texture::clear methods will wipe out all texture data so that's why you had crashes. What you need to do instead in to get texture render and clear its contents via glClearColor or ID3D11DeviceContext::ClearRenderTargetView.

[bertrand.remondin]

Thank you!

 

I've made some more research to try to avoid using directly the graphic Api's. I could use blend function instead. Here is what I'm trying to do:

myTexRender->enable();

int OldSrcBlendFunc = Unigine::State::get()->getBlendSrcFunc();
int OldDestBlendFunc = Unigine::State::get()->getBlendDestFunc();
Unigine::State::get()->setBlendFunc(Unigine::State::BLEND_ZERO, Unigine::State::BLEND_ONE);

/** Render Noesis Things **/

Unigine::State::get()->setBlendFunc(OldSrcBlendFunc, OldDestBlendFunc);

myTexRender->disable();

That does not make any change apparently. I've tried to change the functions after enabling the TextureRender but that did not make any difference. Am I doing something wrong?

[unclebob]

Bertrand,

 

Oh, I totally forgot about Unigine::State thingy, my bad! Once you enabled texture render you can just call Unigine::State::clearBuffer (https://developer.unigine.com/en/docs/1.0/cpp_api/reference/api_state_class#c_clearBuffer_int_constvec4n_float_int) to clear your texture contents.

[bertrand.remondin]

It looks like it's working, thank you again :)

[bertrand.remondin]

I was recently confronted to a bug that was due to the fact Noesis did not restore properly the OpenGL context after rendering.

Is there a proper and API-agnostic way of fully saving and restoring the graphic context in Unigine?

[unclebob]

Bertrand,

 

The only way you have is to use Unigine::State class (https://developer.unigine.com/en/docs/1.0/cpp_api/reference/api_state_class) and store its values directly in some place and restore them after Noesis rendering.

[azanin]

Hi,

 

I still have one detail to solve for the integration. A problem appears which might be due to the fact I don't bind a stencil buffer before rendering the GUI. To do so, I've done the following:

TexturePtr _colourTexture = Unigine::Texture::create();
_colourTexture->create2D(inWidthPix, inHeightPix, Unigine::Texture::FORMAT_RGBA8, Unigine::Texture::USAGE_RENDER);

TexturePtr _stencilTexture = Unigine::Texture::create();
_stencilTexture->create2D(inWidthPix, inHeightPix, Unigine::Texture::FORMAT_RGBA8, Unigine::Texture::USAGE_RENDER);

TextureRenderPtr _hudTextureRender = Unigine::TextureRender::create();
_hudTextureRender->create2D(inWidthPix, inHeightPix);
_hudTextureRender->setColorTexture(0, _colourTexture);
_hudTextureRender->setDepthTexture(_stencilTexture);

And then use enable/disable methods on the TextureRender. It's not working as expected, but it might be due to the GUI-framework. I just would like to check I'm not missing something here.

 

Thank you (and thank you for your last answer as well, I've just noticed I did not answer to that before, sorry ;) )

[unclebob]

Hi Alexandre,

 

You just used wrong texture type for depth/stencil texture. Use Unigine::Texture::FORMAT_D24S8 instead.

[azanin]

Oh I see, it works, thank you again.

[azanin]

About saving and restoring the OpenGL context, I'm trying to use Unigine::State as you mentionned earlier, but all I got are 0's for everything.

 

I don't think that's the correct behaviour, because I think one of my previous issues with the integration by setting the blending functions to BLEND_SRC_ALPHA and BLEND_ONE_MINUS_SRC_ALPHA. I should get these by calling getBlendSrcFunc and getBlendDestFunc, right?

 

All of this happen right before Unigine renders, each frame. I got the State instance by calling Unigine::State::get().

 

Here's what happens exactly code-wise:

Unigine::State inState = Unigine::State::get();

int _depthFunc = inState->getDepthFunc();
int _stencilFunc = inState->getStencilFunc();
int _stencilPass = inState->getStencilPass();
int _stencilRef = inState->getStencilRef();
int _blendSrcFunc = inState->getBlendSrcFunc();
int _blendDestFunc = inState->getBlendDestFunc();
int _alphaFunc = inState->getAlphaFunc();
int _polygonFront = inState->getPolygonFront();
int _polygonCull = inState->getPolygonCull();
float _polygonFill = inState->getPolygonFill();
float _polygonBias = inState->getPolygonBias();
Unigine::ShaderPtr _shader = inState->getShader();
Unigine::MaterialPtr _material = inState->getMaterial();

textureRenderInstance->enable();
inState->clearBuffer(0xFFFFFFFF, Unigine::vec4(0.0f, 0.0f, 0.0f, 0.0f));

// Noesis rendering

textureRenderInstance->disable();

inState->setDepthFunc(_depthFunc);
inState->setStencilFunc(_stencilFunc, _stencilPass, _stencilRef);
inState->setBlendFunc(_blendSrcFunc, _blendDestFunc);
inState->setAlphaFunc(_alphaFunc);
inState->setPolygonFront(_polygonFront);
inState->setPolygonCull(_polygonCull);
inState->setPolygonFill(_polygonFill);
inState->setPolygonOffset(_polygonBias, _polygonSlope);
inState->setShader(_shader);
inState->setMaterial(_material);

[unclebob]

Alexandre,

 

I had a look at engine's rendering code and looks like it clears all states once it finished rendering. So I think you're able to set any render state before Noesis rendering such as blending mode.

 

Just get current rendering state, set proper values to it and call flushStates method, the engine will do the rest.

[azanin]

Actually it does not fix my problem. I still need to call

Unigine::State::get()->setBlendFunc(Unigine::State::BLEND_SRC_ALPHA, Unigine::State::BLEND_ONE_MINUS_SRC_ALPHA);

after the UI rendering.

 

EDIT:

Just to make things clear, the only thing I change in the graphic state is calling TextureRender::enable before Noesis rendering and TextureRender::disable once it's done rendering on that texture. What happens in between is Noesis business. That's why I'd like a way to restore the state the engine wants.

Setting the blending functions fixed one of the problem we had when rendering UI but I got these value by using RenderDoc and see what's different with and without UI rendering. Having a way to make sure everything is exactly the way the engine expects it to be would be very valuable.

I thought calling Unigine::State::flushStates after Noesis rendering would solve it, but by reading your message again, I'm thinking I might have misunderstood your solution.

 

Thanks.

[maxi]

Hi, Alexandre.
 
The solution was something like this:

myTexRender->enable();

Unigine::State state = Unigine::State::get();
state->setBlendFunc(Unigine::State::BLEND_ZERO, Unigine::State::BLEND_ONE);
state->flushStates();

/** Render Noesis Things **/

myTexRender->disable();