Coherent UI plugin (HTML5 renderer)

[nikola.vasilev]

Hi there,

 

I recently started playing around with Unigine and made a plugin for Coherent UI, a product by the company I work for. If you haven’t heard of it, it’s basically a high-performance HTML5-compliant renderer based on WebKit.

 

The texture that Coherent UI gives the client can be mapped on anything (e.g. a camera or a 3D model). When mapping on a 3D object the texture has to be copied in the RAM (Unigine::Image), though, I couldn’t find a way to access the material’s texture directly (I don't have the full source). There’s also a huge slowdown caused by Unigine::Image::createMipmaps which I’ll try to avoid in a future version of the plugin. [Edit: This is no longer present in the latest version]

 

The plugin itself is still in very early stage so there are probably a lot of parts that can be done better. At the moment JavaScript interaction and input forwarding are not yet implemented. The plugin allows scripting using Unigine Script through the CoherentUIView object.

 

I’m attaching the plugin and sample code for anyone interested. Here’s some screenshots at the bottom (some UI made with Adobe Edge / CSS3 animation / mapping youtube on a rock). I also made this youtube video to demonstrate the animation done with Edge, it's easier to show than with a still image - http://www.youtube.com/watch?v=YSTNm0Do36E

 

If you have any feedback I’ll be glad to hear it :)

 

Edit (19.10.14): Updated the plugin for Coherent UI 2.x and Unigine 2014-10-06

 

Edit (18.03.15): Updated the plugin for Coherent UI 2.x and Unigine 2015-02-17 (Unigine 2). Many performance optimizations, especially for D3D11. 3D and HUD views now have the same performance and mipmapping is no longer an issue. The HUD is now drawn using a WidgetSprite so you can manipulate it easily.

 

Edit (03.06.15): Added "CoherentGT_Alpha1.zip" which is the first version of the Coherent GT for Unigine 2

 

Edit (21.06.15): "CoherentGT_Alpha1.zip" and "Unigine_2.x_CoherentUI_2.x.zip" are now obsolete. Added updated versions with fixed resizing code and sample in "CoherentGT_Unigine2x_150620.zip" and "CoherentUI2x_Unigine2x_150620.zip", respectively.

CoherentUIPluginAndSample.zip

CoherentUIPluginAndSample_2.x.zip

Unigine_2.x_CoherentUI_2.x.zip

CoherentGT_Alpha1.zip

CoherentGT_Unigine2x_150620.zip

CoherentUI2x_Unigine2x_150620.zip

[binstream]

That looks very nice!

 

I look forward for this plugin at the production-ready level. We are working internally on the add-on store for plugins and content, I believe this one will be popular.

[ivan.cuevas]

It's very interesting.

Congratulations :D

[manuel.gysin]

Hello

 

We did the same integration the last weeks. (Your solution seems far more complete)

 

I'm happy to offer the input(mouse, keyboard) part. Maybe you can setup a github branch? (I can offer a private one on github if you wish)

[nikola.vasilev]

Glad you like it so far guys :)

 

@manuel.gysin

I'd make a public repository but I'm not sure if the Unigine license allows that. I can make a private one on Github/Bitbucket tomorrow or use yours, I don't really have a preference. You can share yours with 'gashtio' if it's already made (that's my github/bitbucket username) so I can push the files with all the history from my local repo.

[manuel.gysin]

Hello Nikola

 

I've added you to the repo with full permissions.

Don't be shocked about the some code parts, much of it was for the PoC. 

 

Bindings and fully input support should be in within the next days. (Sadly, coherent is using virtual-key codes)

[ivan.cuevas]

I've already tested the plugin with many of the html5 demos and experiments, I'm impressed, congratulations again :D

It's possible to have access to an updated version plugin with user inputs?

[nikola.vasilev]

Here's the version with Manuel's modifications. I'd like to add a few changes myself but I can't find the time for it right now :) In any case, there are 3 more functions exported to Unigine Script, ""InjectKeyPress", "InjectKeyRelease", "InjectMouse". The key functions need an integer parameter that represents the key code, and the InjectMouse function needs 2 parameters - the X and Y offsets of the top left corner of the app (the position is calculated as "x = unigineApp->getMouseX() - offsetX"). That should be enough for the moment :)

coherentplugin-input.zip

[manuel.gysin]

One more function added:

 

• You can now transfer data from JS to C++ with getting data back, currently following data structure are allowed: ShareJson(std::string caller, std::string json)
  You can use it like:

	function setGame()
	{
		engine.call("ShareJson", "Test", "Test2").then(
			function () {
				$("#meins").text(arguments[0]);
			},
			function (type, message) {
				/* Will be called on error */
				$("#meins").text("ERROR");
			}
		);
	}

From C++ following is called:

std::string CoherentUIView::ShareJson(std::string caller, std::string json)
{
    return std::string(Unigine::Engine::get()->runWorldFunction("setJsonData",Unigine::Variable(caller.c_str()), Unigine::Variable(json.c_str())).getString());
}

BTW:

If anyone is skilled with DX11: For windows there is the need to convert a BRGA to an RGBA texture for correct color displaying. :)

coherentplugin-master.zip

[ivan.cuevas]

Thanks guys. 

[lars.saalbach]

If you want to use Coherent, I'd recommend to use AngularJS mixed up with Coherent.
https://angularjs.org/

Point 1: Dynamic loading of HTML-Templates

Point 2: Multilanguage - https://github.com/angular-translate/angular-translate

Point 3: Services - One instance and nothing more ;) - To handle all your guis.

Point 4: UI-Router - Provide different states (Ingame, Login, etc) - https://github.com/angular-ui/ui-router

 

Via a singelton service you can add dynamic html-templates with a dynamic controller, sample:

var settingsEle =  $('<div ng-show="show" ui-view="yourViewName"></div>');
$("body").append(settingsEle);
$compile(settingsEle)(angularScope);

The rest is handling my "routeProvider" 
 

function RouteProvider($stateProvider, $urlRouterProvider) {
     $stateProvider.state('login', {
     views: {
        "loginView": {
            templateUrl: 'templates/state/login/loginView.html',
            controller: LoginController
        },
         "settingsMenu" :
              {
            templateUrl: 'templates/settings/settings.html',
            controller: SettingsController
        }
    }
    })
     .state("game", {
     views: {
         "settingsMenu" :
              {
            templateUrl: 'templates/settings/settings.html',
            controller: SettingsController
        }
    }
    })

}

Also you can trigger a new instance outside of angular js, just with jQuery/javascript:
 

angular.element(document).injector().get("yourService").createGUI

If you want to go to another state, you can use the $state obj:

$state.go("game");

Greet

[steve.brodie]

Looks good, what is the cost to license Coherent ?

[nikola.vasilev]

Depends on the team size and revenue. You can write to sales@coherent-labs.com to request a quote. We'll also be announcing a low-cost per-developer subscription model (that seems to be so popular now:)) for indie teams very soon.

[steve.brodie]

Thanks, Nikola, Im hoping its at the same price point as offered to Unity.

[manuel.gysin]

Coherent in action:

 

Recorded it with my laptop because of that it's a bit laggy.

[david.garcia]

Hi Manuel,

 

The version that you use in the video, is the same as the last one you added? 

[manuel.gysin]

Hello David,

 

Yes, it's the same with little modifications to the ShareJson function.

Note: I used OpenGL rendering for this video, the DirectX one displays the wrong colors because BRGA is used by coherent and unigine  uses RGBA.

[christophe.meurice]

Hello Manuel,

I'm trying to run the example on linux but I have some problems ...

 

- I had to make some changes in order to compile the plugin :

 

std::string CoherentUIView::ShareJson(std::string caller, std::string json)

{

    return std::string(Unigine::Engine::get()->runWorldFunction(Unigine::Variable("setJsonData"),Unigine::Variable(caller.c_str()), Unigine::Variable(json.c_str())).getString());

}

 

and (I'm running on x86)

 

solution 'CoherentUI'

 

    configurations { 'Release' }

    if os.is('linux') then

        platforms { 'x32' }

    else

        platforms { 'x32', 'x64' }

    end

    

    include 'CoherentUIPlugin'

    include 'CoherentUISample'

 

 

- I'm running the project with the 'CoherentUISample_x86' but it crash like this :

 

 /home/christophe/Devel/Unigine/Projects/x86/lib/libCoherentUIPlugin_x86.so: undefined symbol: InitializeUISystem

 

Symbol seems to be present :

 

 christophe@alderaan:~/Devel/Unigine/Projects$ cat /home/christophe/Devel/Unigine/Projects/x86/lib/libCoherentUIPlugin_x86.so | grep InitializeUISystem

Binary file (standard input) matches

 

 

Any idea ?

 

Thanks,

 

Christophe

unigine-log.txt

[manuel.gysin]

Hell Christope,

 

can you please link it against libCoherentUI.so? (Be aware coheren only works with 64-bit linux)

 

file libCoherentUI.so
libCoherentUI.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=a1ec661c456a353eac7d4ff73b3f9f9c35421f40, stripped

[christophe.meurice]

Ok, I was trying on linux 32 ... I will try on x64 ...

Is it planned to be supported on Android ?

 

Thanks,

 

Christophe

[christophe.meurice]

Hi,

 

Now the application is starting but I can't see the coherent ui. I have the following message continously :

 

Trying to create Coherent UI View... [fail] UI System not ready!

 

Any idea ?

 

 

 

To be able to run it, I had to edit the CoherentUIPlugin/premake4.lua

 

From (line 60)

 

elseif os.is('linux') then

libdirs {

'../CoherentUI/lib/Linux'

}

links {

'GL',

'pthread',

'rt'

}

buildoptions {

'-std=gnu++0x',

'-fvisibility=hidden',

'-fvisibility-inlines-hidden'

}

 

To 

 

elseif os.is('linux') then

libdirs {

'../CoherentUI/lib/Linux'

}

links {

'GL',

'pthread',

'rt',

'CoherentUI'

}

buildoptions {

'-std=gnu++0x',

'-fvisibility=hidden',

'-fvisibility-inlines-hidden'

}

 

 

Thanks,

 

Christophe

[nikola.vasilev]

Hi Christophe,

 

Thanks for the fix, I'll update the premake file.

 

As for an Android version, we're currently working on a completely in-process UI solution (based on WebKit, again) that's going to support consoles and Android (no iOS for now, Apple won't allow using a WebKit port other than theirs). I'd say that a final production version of that is not going to be available before the holidays, though. There are no plans to port the multiprocess version of Coherent UI for mobile. We have a mobile version that uses the Android/iOS-specific WebViews that has been released for Unity3D, but it has a limited feature set.

[christophe.meurice]

Hello Manuel,

 

No idea about the problem I have ?

 

(Trying to create Coherent UI View... [fail] UI System not ready!)

 

Thanks,

 

Christophe

[demostenes]

Hi Christophe,

 

Thanks for the fix, I'll update the premake file.

 

As for an Android version, we're currently working on a completely in-process UI solution (based on WebKit, again) that's going to support consoles and Android (no iOS for now, Apple won't allow using a WebKit port other than theirs). I'd say that a final production version of that is not going to be available before the holidays, though. There are no plans to port the multiprocess version of Coherent UI for mobile. We have a mobile version that uses the Android/iOS-specific WebViews that has been released for Unity3D, but it has a limited feature set.

 

Simple advice, dont waste time with mobile, Unigine is not Unity3D and majority of people here is doing serious projects. Also my opinion is, that 3d engine mobile market is bubble, just look at top 10 mobile games of this year. How many is 3d? Reality is, that nobody will ever play more complex game on mobile. For mobile people like simple 2d games, which can be played 50 seconds in public transport and then forgotten. Everything more complex will have huge problem to succeed.

[manuel.gysin]

Hello Manuel,

 

No idea about the problem I have ?

 

(Trying to create Coherent UI View... [fail] UI System not ready!)

 

Thanks,

 

Christophe

 

Ahh I think I had to deploy libdbus.so.0 to get it running. Try LDD on the host binary and look what is missing.