[SOLVED] ObjectBillboards/ObjectMeshDynamic memory pre-allocation

[ulf.schroeter]

Problem

 

We are dynamically creating large counts (30000+) of billboard instances via ObjectBillboards::addBillboard() on application startup. This takes very long time and most of this time is spend in addBillboard() most probably due to recurring memory reallocation and data copying. Same problem for dynamic ObjectMeshDynamic vertex data creation.

 

Proposal

 

Addition of something like ObjectBillboards::reserve( int countInitial, int countIncrement ) to pre-allocate larger memory chunks to speed-up data loading. Same for ObjectMeshDynamic e.g. ObjectMeshDynamic::reserveIndices/Vertices( int countInitial, int countIncrement )

[manguste]

Once billboards are created, clearing and recreating them is done without stalls. But the start-up can take long time indeed. Frustum said he'll investigate this issue, and if it helps will add reserve() function.

 

P.S. Ulf, can you share your opinion whether you need access to billboards via C++ API, for example, for your project?

[ulf.schroeter]

At the moment I am just using UNIGINE script including billboards for visualization of some global communication data. To be honest, I don't think that current very limited UNIGINE C++ API support is a good approach at all. I know that the original idea was to provide a very robust programming environment based on UNIGINE scripting.

 

But for "serious" projects full C++ API access with defined extension points (OGRE is a good example for such a philosophy) is more or less mandatory from my point of view, not only due to performance reasons, but also to have a full-fledge debugging environment (I still don't know how you managed OilRush debugging, but excessive usage of printf or log.message is 70th programming style...)

 

With current approach you quickly end up in some mixed UNIGINE source code extensions (which interferes with your code development), C++ API access (which only offers a very limited functionallity) and huge amounts of scripting (which has no serious debugging support)...call it challenging or call it a nightmare depending on your basic life attitude

[steve3d]

Agree with ulf, why unigine use some wrapper class to export C++ functions in engine?

In the first time I get into the C++ api, I'm really shocked by this design, all these modern opensource engine use directly exported classes in engine like ogre, irrchit, even the ancient Virtools, they all export the classes directly, then by using C++ to extend the functionality is very easy, and a C++ plugin can access almost any aspect of engine.

 

I've never think of writing some huge script or bigger plugin, because plugin can access only few of engine, and debuging a huge script system is a nightmare. I have to say that you unigine guys are really a group of superman to make OilRush debuging work.

 

In fact creating about 4000 billboards in ObjectBillboards also need about 2-3 seconds on my computer.

[manguste]

Ulf and Steve,

 

First of all, a feedback from our Oil Rush developers: script debugging is neither a nightmare, nor even a serious challenge. Well, yes, UnigineScript debugger is not a visual one, it's more of gdb style. But all the necessary functionality is there. You can add a breakpoint, see a call stack, etc. - everything you need to perform step-by-step debugging.

 

Developers that created Oil Rush say that for games (at least), it is actually more convenient to use the following set for debugging: visualizer, log.message, debugger and profiler all together. Yes, you are right about the fact that log.message is used, but the reason for it is actually different: unlike debugger, it allows to see how the picture changes dynamically, which is sometimes makes the issue easier to grasp. During Oil Rush development the debugger was also used quite heavily. It and message logging simply fit different purposes and fit pretty good in the opinion of our developers.

 

 

As for C++ API and wrapper classes to export engine functions, it all boils down to the matter of design. And by design Unigine decided to go with scripting system rather than giving direct access to its C++ part. Profit scripting brings:

• Painful C++ memory management is out of the way. With scripts it's much easier to avoid app crashing due to memory issues.
  
• Scripts are cross-platform out of the box, again, no headache whether your C++ code is platform-specific or not. If suddenly there's a need to port to another platform, it's already done for you.
  
• Fast iterations. That is, one-click world reloading instead of having to recompile the app each time you change the code.
  
• Additional language features like yield, wait, fast callbacks, etc.
  

The current C++ API is a logical extension of this approach. The idea is to perform CPU-intensive computations on C++ side, pass the results to scripts, for the implemented high-level script system to handle it. In-house, we've created 3 implementations of the network and added support of external GUI framework via C++ API (and the fact that C++ code and scripts need to be debugged separately did not hinder it in any way). C++ API is tailored for this kind tasks - it is not designed to replace the scripts completely.

 

 

 

Given this premise, you can make all sorts of modifications with a source SDK. The full source code on hand is a green light and a free road for customization.

So to sum it up, we see such design as a chance for seasoned developers as well as less experienced ones to effectively use Unigine engine and create projects on its base.

[ulf.schroeter]

I like this last post, nice summary of different UNIGINE design aspects. In our simulation use-case we were of course "driven" by quite complex external legacy C++ simulation state which had to be passed through to the scripting environment and from there back into the UNIGINE C++ core (we used UNIGINE 'only' as render frontend, not as a stand-alone game engine). For huge amounts of external object states to be synchronized this led to quite some performance problems. For 'regular' games the UNIGINE scripting approach might be a much better solution