UNIGINE 2.0 Release Candidate: IDE Integration, PBR Improvements, Bugfixes

[binstream]

Key Improvements

• PBR as the default material (with full DirectX support).
• Improved LightWorld shadows.
• Better IDE integration.
• Faster builds on Windows.
• Creation of C++/C# projects via SDK Browser.
• Improved FBX support.
• Performance optimizations for UnigineEditor.
• A lot of bugfixes.

This version is considered stable enough for production use.

 

 

The UNIGINE 2.0 release will be presented (together with a new demo) at SIGGRAPH 2015 in Los Angeles (August, 9-13) at the UNIGINE booth #853.

 

Renderer

• Improved PBR (Physically-Based Rendering) and SSR (Screen-Space Reflections) materials:
  • Added full support for DirectX 11.
  • Migrated from the add-on format to the core material libraries. Instructions on how to upgrade your projects can be found here.
  • mesh_pbr_base is the default material moving forward.
• Improved detection of AMD GPUs.
• Increased Visualizer performance.
• Fixed crashes with AMD video driver under Linux.
• Implemented smooth transitions between LightWorld shadow splits on PBR materials: render_use_shadow_lerp console option (enabled by default).

 

IDE Integration

 

Improved IDE integration: introduced solution and improved project files for Visual Studio (2010 / 2013): engine, plugins, main application (can be found in source/engine folder).

 

 

XCode project for the engine has been updated as well.

 

 

Additionally, SDK Browser now generates IDE-friendly C++/C# projects (see below).

 

World Management

• Fixed incorrect placement of WorldLayer nodes.
• Fixed crash on nested WorldLayers import.
• Added configurable debug logging for data streaming via console variables:
  • world_log_async (0 - disabled; 1 - log loading/unloading; 2 - log all actions)
  • filesystem_log_async (0 - disabled; 1 - log loading/unloading; 2 - log all actions)

GUI

• Fixed incorrect tooltip size on window resize.
• Fixed image scale defined in rich text tag.
• Fixed "show" callback defined in .UI file.
• Fixed memory leak in WidgetCanvas::setPolygonImage.
• Fixed incorrect click handling with GUI_ALIGN_OVERLAP flag on.
• Fixed black rectangles under tooltips.

Sound

• Fixed crash on audio playback from video (OGV) files.
• Fixed incorrect timing on repeated sound playback.

Editor

• Added "Import" main menu section, "Node File Reference" and "Node File Contents" items were moved there.
• FBX import plugin has been integrated into the editor core: "Import -> FBX File".
• Added "Import tangent space" option on FBX import.
• Added an option of merging multiple static meshes from FBX file into a single mesh.
• Fixed crash in FBX import if it was canceled after we selected an FBX file.
• Removed annoying prompt during FBX texture import, now default textures from core.ung will be used for missing textures.
• Added an option of choosing a workflow for the physically based materials (if any) on FBX import.
• Fixed root material & root node name import from FBX files.
• Greatly increased performance of surface selection.
• Added selection of a mesh file and "Reload" button for DecalDeferredMesh nodes.
• Fixed undo of assigning a physical body to an object.
• Fixed crash on material modification undo.
• Fixed loading of world scripts located in extern_path folders.
• Fixed wrong placement of objects contained in a NodeLayer on import.
• Removed "system_editor" option from unigine.cfg: "-extern_define EDITOR" argument is required now.
• Implemented code framework for asset browser system (without UI yet).
• Fixed bug with mouse pointer on terrain editing.
• Fixed merging of material library hierarchy in the Node Export plugin.
• Fixed creation of rudimentary .world_ files after "File -> Save World As" operations.
• Fixed black rectangle bug after closing the rendering settings window.
• Fixed crash with painting by a mask or diffuse+mask on missing materials.
• Removed obsolete file formats (sanim / smesh / spline) from file dialogs.
• Fixed crash on undo and exit from terrain editing mode.
• Animation play/pause works for multiple selected objects.

 

SDK Browser

• Added support for upgrades of SDKs, add-ons, demos and projects.
• Added customization of options for "Run" and "Edit" buttons of the projects.
• More compact notifications view.
• Added check on available disk space before downloading components.
• Windows installer can also install runtime for both VS 2010 and VS 2013.
• Added support for running add-on demos.
• A lot of minor bugfixes.
• Usability improvements all over the place.
• Added generation of C++ / C# projects (for Visual Studio, XCode and GNU Make) into Projects -> Create New Project.

 

Other Changes

• Increased build speed (x2-x3 boost) on Windows thanks to the precompiled headers.
• Increased maximal number of UnigineScript namespaces from 65536 to 262144 (it is recommended to delete old .cache files).
• Exposed WorldIntersection, WorldIntersectionNormal, WorldIntersectionTexCoord classes into the C++ API.
• Added ObjectMeshClusterPtr::setMeshTransform(int num, const dmat4 &transform) into the C++ API.
• Fixed flickering settings window in the GLAppQt sample.
• Removed Sixsense plugin due to the dropped support of Razer Hydra by the vendor.
• Fixed crashes on loading x86 engine plugins under x64 Linux.
• Removed -d command line argument from Archiver.
• Fixed AppWall configurator under Linux.
• Fixed resetting of ObjectParticles simulation on disabling/enabling in the Tracker.
• Fixed crash on adding new MeshSkinned animation parameter in the Tracker.
• Forced visualization of the bones in the Kinect 2 integration samples.
• Restored Windows XP compatibility.

Documentation

• Updated the Execution Sequence article.
• Updated the Memory Management article.
• Updated the Setting Up Development Environment section.
• Updated the Rebuilding the Engine and Tools section.
• Added the SDK Browser Projects article.
• Added articles on the following add-ons:
  • Game Framework
  • Kinect2
  • Materials
  • SFX
  • VRPN
  • Vegetation
• Updated articles on the following objects:
  • WorldLayer
  • Sky
  • Water and WaterMesh
  • Grass
• Updated the following tutorials:
  • Adding Grass
  • Adding Sun, Sky and Clouds
  • Adding Water
  • Setting Up Object LODs
  • Using Tessellation
• Added the article on the Resource tool.
• Added the Matrix Transformations and Basic Object Movements articles.

 

Experimental Features to Come

 

This SDK update is a Release Candidate for final 2.0 release, but there are still some exciting features in the upcoming release version, including:

 

Improved Anti-Aliasing

 

Temporal AA is currently in developement, improving not only still images but also dynamic scenes. The goal is to get full-scene AA faster than with MSAA.

 

 

Editor integration into a Qt application

 

The goal is to provide better usability by migrating tools to Qt widgets (native look and feel, fast) and speed up the development process (so users will get more polished features faster). During the transition period there will be a hybrid UI: old windows with Unigine GUI and new UI elements based on Qt widgets.

 

 

Stay tuned for the release!

 

 

How to Install this Update

 

There are two ways:

• (recommended) via SDK Browser (available in the Downloads section)
  • (if it was already installed) SDKs -> Upgrade (requires update to SDK Browser 1.2 first)
  • (new installation) SDKs -> Install
• Install standalone SDK (available in the Downloads section)

[silent]

Known issues

 

SDK Browser 1.2 (build 2476):

• Licensing checking under Mac OS X 10.10 can work incorrectly. Fixed in build 2510.
• core.ung / editor.ung / scripts.ung are not upgraded after upgrading previous created projects from SDK Browser 1.0. Fixed in build 2510.

Engine:

• Warnings Xml::read_args(): invalid XML, attribute 'version' when saving and parsing any XML file.

C# API:

• Missing grab(), release(),  isOwner() methods. This may result in memory corruptions and engine crashes. Fixed libraries for float engine and double.

[manuel.gysin]

Thank you!

What is the best way to migrate RakNet stuff? Currently we are not sure to port the plugin to 2.0 or switch to another unigine based library?

 

Greets

Manuel

[binstream]

RakNet plugin is not supported for a while, we don't plan to migrate it to UNIGINE 2.0.

[manuel.gysin]

Hello binstream

 

@RakNet

This is so far clear to us, but my questions is whats your suggestion to use.

 

@Feedback

Played around with it a bit, so far I love the SDK Browser.

Is there a roadmap to add custom add-ons for downloading?

 

@Feature Request

Currently all demos and samples are opening in the release mode without unigine editor loaded.

Can you please add the option to load the editor in the samples?

 

Greets

Manuel

[silent]

manuel.gysin

 

@RakNet

This is so far clear to us, but my questions is whats your suggestion to use.

For the RakNet alternatives, please, visit this link: https://developer.unigine.com/forum/topic/3120-solved-alternative-to-raknet/

 

@Feature Request

Currently all demos and samples are opening in the release mode without unigine editor loaded.

Can you please add the option to load the editor in the samples?

Yep, indeed there are some missing options currently. We have plans to extend demos functionality (in UnigineScript samples you can load editor via console command editor_load) in the next SDK Browser update.

 

Thanks!

[manuel.gysin]

manuel.gysin

 

For the RakNet alternatives, please, visit this link: https://developer.unigine.com/forum/topic/3120-solved-alternative-to-raknet/

 

Yep, indeed there are some missing options currently. We have plans to extend demos functionality (in UnigineScript samples you can load editor via console command editor_load) in the next SDK Browser update.

 

Thanks!

 

Thanks a lot silent! :)

[binstream]

@Feedback

Played around with it a bit, so far I love the SDK Browser.

Is there a roadmap to add custom add-ons for downloading?

 

Yes, the goal is to implement a fully-fledged add-ons store with both free and paid add-ons, including submitted from 3rd parties.

[werner.poetzelberger]

 

Editor integration into a Qt application

Looking forward to that!

 

Thank you for the DX version of PBR! (..and all the other stuff as well!)

 

Question: has this been integrated already?

 

https://developer.unigine.com/forum/topic/3312-terrain-diffuse-texture-issue/

 

 

nice.

Cheers

[cinetec_tech]

any road map for this year? Will GI in unigine be considered?

[esger.abbink_]

Better Qt/external windows support would get a big thumbs up here as well!

 

(The current Interface plugin is even with a lot of extra work spent on ironing out kinks still not really usable unfortunately with lots of larger and smaller idiosyncrasies.)

 

 

Due to work overload I have not had a chance to give the recent 2.0 releases more than a fleeting look, most of it looks good and I'll be diving in soon probably. (at the moment we're still on 2014-7-7)

 

I do however want to make a few remarks regarding the new SDK Browser. I can probably see from a binary customer pov where it might be ok or a step forwards (although even then I am not entirely sure the benefits outweigh the cost). For us it presents absolutely no additional value and any effort spent on it is time not spent on those items that for us are crucial: PBR, rendering/physics determinism, ellipsoid earth model, DIS/HLA, and all-around cool next-gen 3D engine stuff. (UnigineScript also falls in the category of time far better spent elsewhere)

 

In fact, the sdk browser and the accompanying change in folder structure and increased number of downloads just made the recurring task of integrating a new unigine release that much more of a pain.

 

While for our projects unigine is a key element it is also just one element of many. We use many other software components, both internal and external. Unigine and some internal libraries are common to many projects, but many project specific components are exactly that: project specific. We use mercurial for versioning in combination with its guestrepo extension to manage relations between repositories. One of our recent projects for instance looks more or less like this regarding its repository layout:

+Project+-+
          +-main internal library+-+
          |                        +-unigine+
          |                                 +-bin
          |                                 +-lib
          |                                 +-docs
          |                                 +-data/demos
          +-internal library
          +-external library
          +-projects application 1
          +-projects application 2
          +-projects application 3

With everything except the project specific applications being common. Other projects can add more external libraries or remove internal ones.

 

Because we do projects with simulation demands that unigine cannot (yet) fulfil, it is occasionally necessary to modify or extend unigine. We do try to avoid it whenever we can, but sometimes there is just no way around it. This means that with every new unigine release a merge needs to happen. In the past this was already a little bit of work because of the split between windows and linux sdk's as we merge them again before doing a new release commit in our unigine repository. Subsequently we merge the new sdk from the release branch to the development branch containing our modifications.

 

The new sdk and browser change a couple of things which make this that much harder. Partly hopefully just once, partly recurring. And this being done manually each time, means it is error prone each time.

 

Therefore a couple of questions/remarks:

 

- can you please provide a complete set of an sdk version in as few downloads as possible? (having to download each demo as separate ung was not ideal, having to go around in the sdk browser and click on each and every demo and add-on is a little bit more tedious again). At the moment afaics the demos and add-ons are not directly downloadable from the website.

 

- the c# dll's and exe's are different between linux and windows sdk, are they in fact interchangeable or not? if not, please consider putting them in a platform specific directory

 

- why were materials/vegetation/sfx split off into add-ons? I'd consider them quite 'core'-y. Whereas that is not necessarily the case for vrpn. However having vrpn included as standard in the sdk poses no real issues, while having core functionality split out among two or more locations does. (see below)

(I can find where they 'belong/came from' but moving them there does not mean everything will work correctly because of hardcoded paths starting with 'add-ons'..)

 

- how future proof is the current layout? Adjusting layout if it is a one time thing is ok, every two releases.. not so much.

 

- Also, having add-ons & demos outside the sdk directory complicates things there a bit and makes no sense for us. I assume the idea is to have the possibility of having multiple sdk versions concurrently, without doubling up on (mostly?) identical demo data? Imho, thats a problem for version control to solve.

Either:

 

1. we leave them in the new locations and I have commonly used data in multiple locations (pain) with very convoluted paths (i.e. 'addons/vegetation_addon_1.0/data/add-ons/unigine_vegetation' instead of something simple as 'vegetation' or even 'addons/vegetation'). Which is added pain since it does not add anything useful but having a version 1.2 or 2.0 in the future only means more pain since then obviously the path changes. Imho, again trying to solve something a versioning tool should.

 

2. we move them to a different location (perhaps sdk/data/addons would work??), while stripping all the outer paths. Not sure that would work correctly in all cases, and lots of manual work each time.

 

- the sdk itself also has a path with its version number in it ('sdks/sim_src_lin_2.0-RC'). Assuming a version 2.1 would change the path name => pain. The source and everything with it should be in a place with a constant path name. Keeping track of different versions of them is the job of a versioning tool.

 

 

In short:

 

Doing anything other than keeping strictly to the new layout may solve issues above, while perhaps still not working 100% correctly and breaks the browser and with it the one very useful function we did quite regularly use the old browser for: quickstart a sample to see how it works & quickstart a sample to see if a bug is present there as well (comparing just (modified) unigine versus larger application).

 

Keeping strictly to the new layout is in a couple aspects a continued exercise in pain. The new layout is in annoyingly at odds with the use of a versioning tool and puts new stuff next to old stuff instead of over it.

 

Now what? :huh:

[esger.abbink_]

A response from unigine regarding the previous post would be appreciated. I am not sure how many source customers there are percentage wise in relation to binary customers, but I imagine other source customers possibly having similar issues.

 

For us, it is holding up upgrading to the latest sdk (coming from 2014-7-7). With several ways to go about, each having their negatives, there is no right choice. Having some of the issues raised answered or addressed would perhaps help us make the 'least wrong' choice.

 

And if for instance the next sdk would again be significantly different, we probably would choose to hold off upgrading until layout has stabilized.

[ivan.cuevas]

We develop for Windows and Linux. All our projecst are based on UnigineScript, but this structure supports custom dll/so and executables. Each of the UnigineScript projects is an independent repository so we are able to create different application using several modules shared between them.

 

Upgrading from last version it's easy apart of addons distribution because it's apart from SDK data folder (please Unigine team consider change this).

+-apps_dir_x64
  +-bin
  +-lib
  +-unigine
    +-config
    +-data
      +-add-ons
      +-core
      +-editor
      +-scripts
      +-project 1 (git)
      +-project 2 (git)
      +-.... (git)

[silent]

Thank you for the feedback. We are totally understand the inconvenience caused by new SDK directories structure and we are now thinking about how to minimize this impact for the future releases.

 

I believe Denis will give you more detailed information soon.

[binstream]

First of all, thank you for your feedback - it's very helpful to shape up the future releases.

 

Better Qt/external windows support would get a big thumbs up here as well!

 

(The current Interface plugin is even with a lot of extra work spent on ironing out kinks still not really usable unfortunately with lots of larger and smaller idiosyncrasies.)

 

Agree. With the transition of UnigineEditor to Qt widgets we have already improved Qt integration a lot, and with future releases it will be even better.

 

 

Answering your questions item by item:

 

- can you please provide a complete set of an sdk version in as few downloads as possible? (having to download each demo as separate ung was not ideal, having to go around in the sdk browser and click on each and every demo and add-on is a little bit more tedious again). At the moment afaics the demos and add-ons are not directly downloadable from the website.

 

If you have installed SDK and some add-ons and demos, all of them will be updated with single "Upgrade SDK" operation (single click).

 

- the c# dll's and exe's are different between linux and windows sdk, are they in fact interchangeable or not? if not, please consider putting them in a platform specific directory

 

They are platform-dependent.

 

- why were materials/vegetation/sfx split off into add-ons? I'd consider them quite 'core'-y. Whereas that is not necessarily the case for vrpn. However having vrpn included as standard in the sdk poses no real issues, while having core functionality split out among two or more locations does. (see below)

 

We are extending these libraries now and they will consume more disk space, while not 100% needed for all the projects.

 

- how future proof is the current layout? Adjusting layout if it is a one time thing is ok, every two releases.. not so much.

 

We plan to apply some more restructuring in 2.0 release based on the customer feedback (including yours) and keep it as stable as possible.

 

- Also, having add-ons & demos outside the sdk directory complicates things there a bit and makes no sense for us. I assume the idea is to have the possibility of having multiple sdk versions concurrently, without doubling up on (mostly?) identical demo data? Imho, thats a problem for version control to solve.

 

That's one of the things under consideration at the moment.

 

- the sdk itself also has a path with its version number in it ('sdks/sim_src_lin_2.0-RC'). Assuming a version 2.1 would change the path name => pain. The source and everything with it should be in a place with a constant path name. Keeping track of different versions of them is the job of a versioning tool.

 

We see customers using multiple SDK versions simultaneously. However, if we find more elegant solution, we'll definitely improve this.

 

 

Summary: the SDK browser introduction is a one-time change, but it is designed to speed up the development.

[binstream]

A response from unigine regarding the previous post would be appreciated. I am not sure how many source customers there are percentage wise in relation to binary customers, but I imagine other source customers possibly having similar issues.

 

Here is another potential option for source licensees: get regular (binary) SDK as it is now (either as a standalone build or via SDK browser) + easy access to source code (some sort of version control repository ideally for easy update / diff view / hotfixes access). What do you think?

[esger.abbink_]

thank you for the detailed response.

 

we'll hold off upgrading for now then, with the next release hopefully addressing some issues (addons outside sdk dir / no version numbering in paths being the biggest). Losing the browser in the sdk dir is a little bit more inconvenient than having it there, but that is surmountable.

 

Any idea when the next sdk will be out? ;)

 

For us the option of using binary and having access to source would not work. We really do need to modify or add engine behavior here and there (time management, determinism, C++ api extension, bug fixing, video format support extension being a few areas of the top of my hat where we have needed to do some work) and we need to keep the versions of the engine in sync with our applications. Keeping the merges with new releases as smooth as possible minimizes chances new errors are being introduced.

[binstream]

If you integrate parts of UNIGINE into your codebase, the required minimum is source code + build system and data/core. // this alone can be delivered in more convenient way from us

 

If you modify sources, you no longer need pre-built binaries from the SDK in your repository.

What is the reason of putting demos into your project repositories as well?

[esger.abbink_]

We're not so much putting them in, as well as relating them. We use mercurial and its guestrepo extension. The guestrepo extension allows one to structure different repositories and link them. But it does this in a loosely coupled way (mercurial subrepos are tightly coupled in contrast).

example repo layout, each node/leaf representing a repository:

+Project+-+
          +-main internal library+-+
          |                        +-unigine sdk+-+
          |                                       +-bin
          |                                       +-lib
          |                                       +-docs
          |                                       +-data/demos
          +-internal library
          +-external library
          +-projects application 1
          +-projects application 2
          +-projects application 3

So if you look at the hierarchy above you can see that bin/lib/docs and data/demos are separate repositories. This is done in part because mercurial starts to become more inefficient when repository size grows (due to large binary files) and partly because they are not normally needed. Because they are loosely coupled we don't need to pull in all of bin or lib if it isn't necessary to do so.

 

The reason that we don't leave bin and lib completely out is that sometimes it is beneficial to have 'known-good' versions around to check if a bug was recently introduced, introduced by us, or already present a couple sdk versions ago. Since there was almost no work involved updating the bin and lib repositories there was no reason not to do it.

 

Another reason is that I prefer to have each sdk release completely in our history since older releases are no longer downloadable from the downloads section. Again, almost no work involved so no reason not to do it. This is one reason why demos are linked as well.

 

A second reason to link demos is that our main internal library contains a stub or example application as well. Often this application will be used just to integrate something new, or the content for a project is not yet quite there and we use a demo just to have a working world with something in it we can work with. For example, when developing our new motion driver for rexroth we needed something just to generate some movement and took the scripted camera from Valley. When integrating an FDM it first flew in Valley as well I think. Since the demos do sometimes change with releases it is beneficial to just have the correct one right there if needed. And again, very little work involved there in the past.

 

The only thing that took a bit of time is the merge of linux and windows versions. But there Winmerge comes in quite handy and I think it is normally 10-20 mins.

 

 

Regarding your comment about binaries, that is correct. We can always build binaries from the release branch to check the things above, it is not necessary that those are built in advance by you. The build system is technically (for us) not even required as well, since we use CMake anyway.

[binstream]

OK, thank you for the detailed explanation.

We will figure out next week how to structure SDKs better.

[ggris]

Hi,

 

First, the recent PBR improvements are greatly appreciated.

 

As we have to customize some shaders, I would say that the new shader structure makes development of GLSL and HLSL shaders a lot easier. Do you plan to use the new shader structure on the other shaders as well ?

 

The mesh_pbr shader works nicelly but, in terms of performance, it takes a lot of recalculation : mostly in sample_base.h . In some cases, it would suffice to do a deferred shading to prevent this problem. Currently, if I am not mistaken when I look at the shaders, this shading is not a PBR shading (but it is somehow consistent since it uses PBR samples of the deferred pass). Do you plan on making a PBR version of the deferred shading ?

 

Also, a lot of usefull shaders are not yet PBR : mesh_triplanar and grass for instance. Do you plan on making PBR versions of these in future releases ?

 

Thanks in advance

[silent]

Hi Guillaume,
 

As we have to customize some shaders, I would say that the new shader structure makes development of GLSL and HLSL shaders a lot easier. Do you plan to use the new shader structure on the other shaders as well ?

Yes, absolutely. If you want to heavily modify shaders for your project - it is better to wait the next SDK update.
 

The mesh_pbr shader works nicelly but, in terms of performance, it takes a lot of recalculation : mostly in sample_base.h . In some cases, it would suffice to do a deferred shading to prevent this problem. Currently, if I am not mistaken when I look at the shaders, this shading is not a PBR shading (but it is somehow consistent since it uses PBR samples of the deferred pass). Do you plan on making a PBR version of the deferred shading?

We are implemeting full deferred render right now (with forward pass for the transparent objects). Deferred shading will be used heavily. More performance optimizations will be available soon for the PBR materials.

 

Also, a lot of usefull shaders are not yet PBR : mesh_triplanar and grass for instance. Do you plan on making PBR versions of these in future releases ?

We are going to simplify the materials settings and reduce the number of predefined materials. mesh_triplanar material can be easily created by enabling only a single checkbox in mesh_pbr_base material (not in the RC). Unfortunately, some old materials settings can't be ported correctly. We will try to minimize this impact and provide a tool for automatic materials conversion (but still some manual work might be required).

Thanks!

[ggris]

Thanks for your reply. Very instructive.

[esger.abbink_]

Any news on sdk structure?

[silent]

Hi Esger,

 

Sorry for the late reply.

 

Any news on sdk structure?

RIght now we are in the middle of a designing a new structure that will not change drastically anymore. More information will be available before the SDK release.

 

Thanks!