Implementation NoesisGUI GUI sample

[Klimczak.Jan]

Hi,

At the last time I make a few tests about implementing NoesisGUI GUI framework into Unigine (just display layer and mouse movement and left button down).

This is just prototype, needs refactoring (especially see header file) and maybe some optimize.

It's not documented anywhere, so I would like to share with it:

AppWorldLogic.h:

#include <NoesisGUI.h>
#include "D3D11RenderDevice.h"

#include <windows.h>
#define WIN32_LEAN_AND_MEAN
#include <d3d11.h>
#include <UnigineApp.h>
#include <UnigineTextures.h>
#include <UnigineGame.h>
#include <UnigineMaterial.h>
#include <UnigineMaterials.h>
#include <UnigineLogic.h>
#include "UnigineWidgets.h"
#include "UnigineGui.h"
#include "UnigineTextures.h"


using namespace Noesis;

using namespace Unigine;

class AppWorldLogic : public Unigine::WorldLogic {
	
public:

	Noesis::Core::Ptr<RenderDevice> device;
	Noesis::Core::Ptr<VGContext> context;
	ID3D11Device* pDevice;
	ID3D11DeviceContext* pContext;
	ID3D11RenderTargetView* pTexNoesisRTV;
	Noesis::IRenderer* renderer;

	ID3D11RenderTargetView* ppOldRtv = nullptr;
	ID3D11DepthStencilView* ppOldDsv = nullptr;

	Noesis::Core::Ptr<Noesis::IView> g_XamlView;
	Noesis::Core::Ptr<FrameworkElement> xaml;

	Unigine::TexturePtr pTexNoesisPtr;
	Game *game;

	WidgetSpritePtr hud;
	TexturePtr my_texture;
	WidgetLabelPtr widget_label;

	void createLabel();
	void createHUDWidgetSprite();
	int SetWidgetSpriteTexture(Unigine::WidgetSpritePtr sprite);

	bool is_click = false;

	AppWorldLogic();
	virtual ~AppWorldLogic();
	
	virtual int init();
	
	virtual int update();
	virtual int render();
	virtual int flush();
	
	virtual int shutdown();
	virtual int destroy();
	
	virtual int save(const Unigine::StreamPtr &stream);
	virtual int restore(const Unigine::StreamPtr &stream);
};

AppWorldLogic.cpp:

#include <NoesisGUI.h>
#include "D3D11RenderDevice.h"

#include "AppWorldLogic.h"
#include <UnigineApp.h>
#include <UnigineTextures.h>
#include <UnigineGame.h>
#include <UnigineMaterial.h>
#include <UnigineMaterials.h>
  
using namespace Noesis;
  
  
// Error handler are invoked for fatal errors. You must never return from here
void NoesisErrorHandler(const NsChar* filename, NsSize line, const NsChar* desc, NsBool fatal)
{
	Log::message("Blad noesis %s \n", desc);
}

int AppWorldLogic::init() {
	// Write here code to be called on world initialization: initialize resources for your world scene during the world start.

	Noesis::GUI::Init(NoesisErrorHandler);
	Noesis::GUI::SetResourceProvider(".");

	xaml = Noesis::GUI::LoadXaml<FrameworkElement>("PasswordBox.xaml");
	g_XamlView = Noesis::GUI::CreateView(xaml.GetPtr());
	g_XamlView->SetSize(1600, 900);
	g_XamlView->SetAntialiasingMode(Noesis::Gui::AntialiasingMode_MSAA);

	game = Game::get();

	Materials *materials = Materials::get();
	MaterialPtr m = materials->findMaterial("noesis_mat");
	int num = m->findTexture("albedo");

	pDevice = static_cast<ID3D11Device*>(Unigine::App::get()->getD3D11Device());
	pContext = nullptr;
	pDevice->GetImmediateContext(&pContext);

	// Initializes renderer. This could be done in a render thread
	device = *new Noesis::Render::D3D11RenderDevice(pContext);
	context = Noesis::GUI::CreateVGContext(device.GetPtr(), Noesis::VGOptions());
	g_XamlView->GetRenderer()->Init(context.GetPtr());

	renderer = g_XamlView->GetRenderer();

	pTexNoesisPtr = Unigine::Texture::create();
	pTexNoesisPtr->create2D(1600, 900, Unigine::Texture::FORMAT_RGBA8, Unigine::Texture::USAGE_RENDER);

	createHUDWidgetSprite();
	createLabel();
	SetWidgetSpriteTexture(hud);

	return 1;
}

void AppWorldLogic::createLabel()
{
	GuiPtr gui = Unigine::Gui::get();
	widget_label = WidgetLabel::create(gui, "Label text");
	widget_label->setToolTip("This is a label");
	widget_label->arrange();
	widget_label->setPosition(10, 10);
	gui->addChild(widget_label->getWidget(), Unigine::Gui::ALIGN_OVERLAP | Unigine::Gui::ALIGN_FIXED);
}

int AppWorldLogic::SetWidgetSpriteTexture(Unigine::WidgetSpritePtr sprite)
{
	my_texture = Unigine::Texture::create();

	const int width = int(800);
	const int height = int(600);
	int flags = Unigine::Texture::FILTER_LINEAR | Unigine::Texture::USAGE_RENDER;
	my_texture->create2D(width, height, Unigine::Texture::FORMAT_RGBA8, flags);

	sprite->setRender(pTexNoesisPtr);
	return 1;
}

void AppWorldLogic::createHUDWidgetSprite() {
	GuiPtr gui = Unigine::Gui::get();
	hud = WidgetSprite::create(gui);
	hud->setPosition(0, 0);
	hud->setWidth(1600);
	hud->setHeight(900);
	hud->setLayerBlendFunc(0, Unigine::Gui::BLEND_ONE, Unigine::Gui::BLEND_ONE_MINUS_SRC_ALPHA);

	gui->addChild(hud->getWidget(), Unigine::Gui::ALIGN_OVERLAP);
}

// start of the main loop
int AppWorldLogic::update() {
	// Write here code to be called before updating each render frame: specify all graphics-related functions you want to be called every frame while your application executes.
	
	int mouseX = App::get()->getMouseX();
	int mouseY = App::get()->getMouseY();
	g_XamlView->MouseMove(mouseX, mouseY);

	if (App::get()->getMouseButtonState(App::BUTTON_LEFT))
	{
		if (!is_click)
		{
			g_XamlView->MouseButtonDown(mouseX, mouseY, MouseButton_Left);
		}
	}

	ControlsPtr controls = Game::get()->getPlayer()->getControls();
	return 1;
}

int AppWorldLogic::render() {
	// The engine calls this function before rendering each render frame: correct behavior after the state of the node has been updated.
	
	float ifps = game->getIFps();

	if (pContext != nullptr)
	{
		ppOldRtv = nullptr;
		ppOldDsv = nullptr;
		pContext->OMGetRenderTargets(1, &ppOldRtv, &ppOldDsv);

		ID3D11RenderTargetView* pTexNoesisRTV = static_cast<ID3D11RenderTargetView*>(pTexNoesisPtr->getD3D11RenderTargetView());
		pContext->OMSetRenderTargets(1, &pTexNoesisRTV, nullptr);

		// Updates view passing global time
		g_XamlView->Update(ifps);

		// Performs rendering operations. Note that the renderer associated to a view is intended
		// to be used in the render thread. In this simple application it is the main thread
		renderer = g_XamlView->GetRenderer();

		// Applies changes to the render tree
		renderer->UpdateRenderTree();

		// Renders offscreen textures. This step must be done before binding the main render target
		if (renderer->NeedsOffscreen())
		{
			renderer->RenderOffscreen();
		}

		renderer->Render();

		pContext->OMSetRenderTargets(1, &ppOldRtv, ppOldDsv);
	}

	return 1;
}

Also you will need for e.g. D3D11RenderDevice.h and D3D11RenderDevice.cpp files that come with NoesisGUI (I can't share it here because of license agreement).

For work it needs an object with "noesis_mat" material at scene. Also it needs PasswordBox.xaml file from sample delivered with NoesisGUI in data folder.

It will display a Noesis GUI at the screen /xor at object in scene.

Edited February 7, 2018 by Klimczak.Jan

[morbid]

Great!

Thank you, Jan. I'll ask our tech writers to take a look at your sample.

[Klimczak.Jan]

Hi,

I have a little problem with NoesisGUI 2.1. I move my implementation form main app file to AppSystemLogic file (to work in Editor2).

The problem is that each rendered frame of NoesisGUI contains last frame too, I think soo. The problem is that transparency object of NoesisGUI is transparent just for a few first frames and then stay more opaque and full opaque at last. Also when I hide Noesis gui object it is still displayed in the GUI.

my render loop looks like that (in AppSystemLogic render):

	if (!view)
		return;

	ppOldRtv = nullptr;
	ppOldDsv = nullptr;
	pContext->OMGetRenderTargets(1, &ppOldRtv, &ppOldDsv);

	ID3D11RenderTargetView* pTexNoesisRTV = static_cast<ID3D11RenderTargetView*>(my_texture->getD3D11RenderTargetView());
	pContext->OMSetRenderTargets(1, &pTexNoesisRTV, nullptr);

	view->GetRenderer()->UpdateRenderTree();
	view->GetRenderer()->RenderOffscreen();

	view->GetRenderer()->Render();

	pContext->OMSetRenderTargets(1, &ppOldRtv, ppOldDsv);

and update just contains:

	view->Update(Noesis::HighResTimer::Seconds(time - startTime));

*view is a g_XamlView from the previous topic.

Do you have any idea how to fix it ?

NoesisGUI describe this implementation like that (GLUT sample):

void DisplayFunc(void)
{
#ifdef NOESIS_GUI
    // Update view (layout, animations, ...)
    _view->Update(glutGet(GLUT_ELAPSED_TIME) / 1000.0);

    // Offscreen rendering phase populates textures needed by the on-screen rendering
    _view->GetRenderer()->UpdateRenderTree();
    _view->GetRenderer()->RenderOffscreen();
#endif

    // If you are going to render here with your own engine you need to restore the GPU state
    // because noesis changes it. The framebuffer and viewport are restored here
    glBindFramebuffer(GL_FRAMEBUFFER, 0);
    glViewport(0, 0, glutGet(GLUT_WINDOW_WIDTH), glutGet(GLUT_WINDOW_HEIGHT));

    glClearColor(0.0f, 0.0f, 0.25f, 0.0f);
    glClearStencil(0);
    glClear(GL_COLOR_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

#ifdef NOESIS_GUI
    // Rendering is done in the active framebuffer
    _view->GetRenderer()->Render();
#endif

    glutSwapBuffers();
    glutPostRedisplay();
}

Edit:

I attach my implementation of NoesisGUI as a library (library project). Because of license agreement I do not include D3D11RenderDevice.h, D3D11RenderDevice.cpp and Render.D3D11RenderDevice.cpp files which you can add from NoesisGUI download pack (such files are not modified). So You can see how it is works and use it for your own (Unigine I give you all rights to do whatever you want with it).

To work with Unigine project you have to implement it in AppSystemLogis as:

#include "NoesisGUI.h"

NoesisGUI* noesisGUI;
	
int AppSystemLogic::init() {
{
	noesisGUI = &NoesisGUI::getInstance();
	noesisGUI->Initialize();
}	
	
int AppSystemLogic::update() {
	noesisGUI->Update();
	return 1;
}	

int AppSystemLogic::render() {
	noesisGUI->Render();
	return 1;
}

then you can use it somewhere like that:

		NoesisGUI* noesisGUI = &NoesisGUI::getInstance();
		noesisGUI->LoadUI("my_file.xaml");

		// garaz
		Noesis::Button* button1 = noesisGUI->xaml->FindName<Noesis::Button>("button_quit");
		button1->Click() += [](BaseComponent* sender, const RoutedEventArgs& args)
		{
			App::get()->exit();
		};
		
		...

I attach Transparent.xaml view file which you can use to see problem with transparency. Where I hope you will able to help me to resolve it :)

 

 

NoesisIntegration.zip

Transparent.xaml

Edited March 25, 2018 by Klimczak.Jan

[Klimczak.Jan]

To help for community with integration of NoesisGUI and to help fix problem with transparency I created project at GitHub:

https://github.com/researchdeveloping/NoesisGUI-Unigine/

Also I ask NoesisGui to look into it. So I hope we will improve this integration!

 

[Klimczak.Jan]

16 hours ago, Klimczak.Jan said:

To help for community with integration of NoesisGUI and to help fix problem with transparency I created project at GitHub:

https://github.com/researchdeveloping/NoesisGUI-Unigine/

Also I ask NoesisGui to look into it. So I hope we will improve this integration!

 

Noesis found the problem with transparency. I pushed fix into GitHub.

After fix NoesisGUI is rendered on top of Unigine.

 

Edit:

The fix stooped displaying NoesisGUI by AppSystemLogic implementation.

To work NoesisGUI it is necessary to have such loop:

	while (!pEngine->isDone())
	{
		pEngine->update();
		noesisGUI->Update();
		pEngine->render();
		noesisGUI->Render();
		pEngine->swap();
	}

which is possible with main.cpp file. But this loop is not used by Editor2 so it is not working within Editor2.

As I see the AppSystemLogic work in different way as I provided code earlier. The update() and render() should be done in different way.

It is possible to achieve such loop:

// Offscreen rendering phase populates textures needed by the on-screen rendering
_view->GetRenderer()->UpdateRenderTree();
_view->GetRenderer()->RenderOffscreen();

// If you are going to render here with your own engine you need to restore the GPU state
// because noesis changes it. The framebuffer and viewport are restored here
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glViewport(0, 0, glutGet(GLUT_WINDOW_WIDTH), glutGet(GLUT_WINDOW_HEIGHT));

glClearColor(0.0f, 0.0f, 0.25f, 0.0f);
glClearStencil(0);
glClear(GL_COLOR_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

// Rendering is done in the active framebuffer
_view->GetRenderer()->Render();

in AppSystemLogic ?

or implement use of own loop by Editor2 from main.cpp file (which I provided earlier) ?

 

Edited March 29, 2018 by Klimczak.Jan

[morbid]

Hello Jan,

we're working on 2.7 and I'm afraid we can't provide you with any solutions for this issue right now.

Our devs may take a look at this case after 2.7 release.

Thanks!

[Klimczak.Jan]

24 minutes ago, morbid said:

Hello Jan,

we're working on 2.7 and I'm afraid we can't provide you with any solutions for this issue right now.

Our devs may take a look at this case after 2.7 release.

Thanks!

Hello Morbid,

Thanks, so please take a look for it after 2.7 release :)

Thanks!