[SOLVED] Importing a custom model

[istvan.szennai]

I'm trying to import one of our models into Unigine. Currently I use the following C++ method to create objects from individual elements:

Vector<ObjectMeshDynamicPtr> objects;

void processElement(MyElement* elem)
{
    for( int i = 0; i < elem->nummeshes; ++i )
    {
        MyMesh& mesh = elem->meshes[i];
        ObjectMeshDynamicPtr object = ObjectMeshDynamic::create(0);

        for (int j = 0; j < object->getNumSurfaces(); j++) {
            object->setMaterial("mesh_base",j);
            object->setProperty("surface_base",j);
        }

        object->clearVertex();
        object->clearIndices();

        for (int j = 0; j < mesh.numvertices; ++j)
        {
            MyVertex& vert = mesh.vertices[j];

            object->addVertex(vec3(vert.x, vert.z, vert.y));
            object->addTexCoord(vec4(0.0f));
        }

        for (int j = 0; j < mesh.numindices; j += 3)
        {
            object->addIndex(mesh.indices[j + 0]);
            object->addIndex(mesh.indices[j + 2]);
            object->addIndex(mesh.indices[j + 1]);
        }

        object->updateBounds();
        object->updateNormals();
        object->updateTangents();
        object->flush();

        objects.append(object);
    }
}

While it loads fine, rendering runs very slow compared to our current implementation. I suppose the engine can't utilize its full power because I gave it "just single meshes".

My question is how should I tell the engine (through the C++ API!) that:
- meshes are static so it can create large batches from them
- tell it where the portals are (doors, windows)
- tell it that it should be streamed


Thanks
Steve

[sebastianbah]

Hello,

Is it important to use DymanicMeshObject? If all meshes are static you can use engine's native file format for static meshes:
https://developer.unigine.com/en/docs/1.0/code/file_formats#mesh
https://developer.unigine.com/en/docs/1.0/cpp_api/reference/api_mesh_class

This tips about multi-threading also can be useful:
https://developer.unigine.com/en/docs/1.0/principles/world_management/#multithreading

[istvan.szennai]

Hi,

no its not important at all, I just used what I found :)
Is there a way to use the Mesh class similarly that the ObjectMeshDynamic? (not using the file format, but building it from code)

[sebastianbah]

Constructing dynamic objects from code is really slower then loading static meshes. But if you want to construct a model using c++ code, static mesh is not suitable.

 

May be, if you clarify all the process and describe in more detail what exactly you want, we can help to find another way.

[istvan.szennai]

I have a custom model format (architectural elements, materials, lightmaps, etc.) and a bunch of C++ code to parse it. I would like to pass the parsed geometry/material data to Unigine. All the elements are static, so I would like to tell the engine that it should create large batches (grouped by material perhaps). I also would like to tell it which one of the elements are windows/doors, so it can use them in its portal-based algorithms. My problem is that the documentation is not too detailed about the C++ part of the engine. But if you can tell a solution with the script language that will suffice too. Now this is the basic thing I would like to do (as a simple test).

The more advanced topic would be how to tell the engine that it should not put the data into memory, but it should cache it and stream it based on where the player is. Can the engine do this automatically, or I should do something about it?

 

[sebastianbah]

It seems, that if it's really necessary not to create additional files, ObjectMeshDymanic is the only solution for the present moment.

If you can afford creating of new *.mesh file for every unique model, you can write export plugin for your data format and export it to native *.mesh format.
Then for adding batch of elements you can create clusters:
https://developer.unigine.com/en/docs/1.0/scripting/library/objects/class.objectmeshcluster
https://developer.unigine.com/en/docs/1.0/scripting/library/worlds/class.worldcluster

or clutters:
https://developer.unigine.com/en/docs/1.0/scripting/library/objects/class.objectmeshclutter
https://developer.unigine.com/en/docs/1.0/scripting/library/worlds/class.worldclutter

For doors and windows you can add new surfaces and set properties (for example, collision) for every surface.
And if needed, locate portals/sectors accordingly the surface coordinates.

[istvan.szennai]

Thanks, I will look into it.

[istvan.szennai]

Okay I converted every element to .mesh (seems ok because double click opens them perfectly in the viewer).
Now I want to load them :)
 

MeshPtr mesh = Mesh::create();

if( 0 == mesh->load(filename.c_str()) )
{
    std::cout << "Could not load mesh\n";
    return;
}

objects.append(mesh);

But the engine doesn't render anything (checked with PIX). How do I add the meshes to the scenegraph? (or anything)
 

[sebastianbah]

Please, check that all surfaces of your mesh have own assigned material (for example mesh_base).

[istvan.szennai]

Ok, so it works from script only. :)

 

Next question: is there a simple way to create a material? I see that there are methods like engine.material.inheritMaterial, but it cannot inherit to the default library (0). So do I have to create an own XML file? Isn't there a simpler method? (I would expect something like MakeMaterialForMePlease("mymat1")).

 

Also, which one of the materials' members is the diffuse and specular color? :)

 

Thanks

[sebastianbah]

The easiest way to add new materials is to inherit it from the default library.

 

You can add libraries to your world using engine.materials.addWorldLibrary(string filename) function.

Detailed info you can find in docs here: https://developer.unigine.com/en/docs/1.0/scripting/library/engine/engine.materials#engine.materials.addWorldLibrary_string

 

Here you can find some information about new material creation: https://developer.unigine.com/en/docs/1.0/content/materials/#brand_new

 

If you want to set diffuse color/texture for your material:

- find the material you inherit from in docs (for example mesh_base: https://developer.unigine.com/en/docs/1.0/content/materials/mesh_base/)

- check the settings which affect the diffuse color. For mesh_base basic settings are Textures->Base->Diffuse and Parameters->Shading->Diffuse.

- find id of each setting using int findParameter(string name), int findState(string name), int findTexture(string name) from your code (in docs: https://developer.unigine.com/en/docs/1.0/scripting/library/class.material)

- set appropriate values and textures using id

[istvan.szennai]

Thanks, that worked out (took some time though to figure out, but seems to work). I think I'm making progress.

 

(A side note: changing the specular_color parameter (inherited from mesh_base) had a weird effect, so I ignored it for now. And I couldn't find a parameter for making windows transparent.)

Now I would like to ask about streaming (and this is the reason while I'm evaluating btw. :)) I see there is this world cluster thing, which does exactly what I need (keeps only relevant objects in memory). I would like to ask if you know a strategy to create these clusters automatically (assuming a building).

I thought of collecting building elements which are close to eachother (lets say a room). Okay, but thats not exactly a general solution, because smaller buildings (like the one on the picture) can fit into memory anytime. So the implementation should somehow determine whether the model will fit into memory (perhaps I can come up with something), then subdivide the set of elements into subsets which will be the clusters. Do you know any game development pattern/heuristic to this problem? Perhaps the engine has an automatic way to do this?

Still related to this: does this world cluster do static/dynamic batching too? If not how can I use these two techniques together?

 

 

edit: this WorldCluster is hard to use too... how do I create a node reference from script?

edit: whatever...saved them out by hand; the world cluster does weird things though (perhaps I have to set visible distance for surfaces?)

 

[istvan.szennai]

Alright, I got it to work in some form by setting surface min/max visible distances (in the .node files). Now the objects at least disappear when I move far, but its not deterministic at all. Also the world cluster doesn't add anything (visible) to it, so how do I know it works?

Visible distances: 0, 20
Fade distances: 0, 1

[sebastianbah]

Hi,

To fix visibility problems with single mesh may be using LODs could help:

https://developer.unigine.com/en/docs/1.0/tutorials/lods/

https://developer.unigine.com/en/docs/1.0/principles/world_management/#lods

As for cluster, please check cluster parameters and make sure that you've assigned the material to cluster elements. Parameters in docs: https://developer.unigine.com/en/docs/1.0/principles/objects/world_cluster/#params
You can check SDK samples as an example: <SDK>/data/samples/objects/cluster_00.cpp , cluster_01.cpp (From SDK browser: Samples & Demos -> Graphics Samples -> Objects demo -> cluster_00 , cluster_01)

[istvan.szennai]

LODs are something I can't do, because our models don't have them. I'm gonna check the docs you linked.

(edit: ok, this was the problem, min distances should be default)

World cluster: these parameters are set. But I don't really know what I should see, since the engine fades out objects anyway. So what does the world cluster add to this? Unloads invisible objects? Code:

// init()
cluster = new WorldCluster();
cluster.setVisibleDistance(20);
cluster.setFadeDistance(1);

// ...

// when loading model
NodeReference refs[0];
string fname;

for (int i = 0; i < numnodes; i++)
{
    fname = format("../nodes/element%d.node", i);
    NodeReference ref = new NodeReference(fname);

    ref.setParent(cluster);
    refs.append(ref);
}

cluster.createNodes(refs);

(btw. this createNodes call is absolutly not clear, I mean it says "int" as parameter yet it expects an array.

 

As I see those samples you mentioned use the ObjectMeshCluster. If I understand correctly it is used for meshes with the same material (so it might be ok for batching). However it is not mentioned whether these things get streamed too. Do they? If yes, then perhaps its a better solution than the world cluster. If not then theres another problem: the distribution of these "identical" meshes (as they can be anywhere in the model). Do these clusters take this into account (based on the documentation I suppose yes, but what about if they work together?)?

[sebastianbah]

All your objects behaves as if they have only one LOD if the LODs are not set.

 

If I understand you correctly the problem with visibility is in objects disappearing at the close range.
If it is so, you can set negative min visible distance value for such big objects as the stairs.

As for mesh cluster and world cluster, ObjectMeshCluster is optimized for rendering a big numbers of objects. But if you use ObjectMeshCluster you can generate or add to a cluster only identical objects (same mesh and same material).
To check that the cluster is added successfully you can set the visible distance to inf(infinity), so all objects will be rendered anyway.

 

Some additional information about clusters in docs here:

https://developer.unigine.com/en/docs/1.0/principles/objects/mesh_cluster/

https://developer.unigine.com/en/docs/1.0/principles/objects/world_cluster/

[istvan.szennai]

The LOD problem is solved now (since they had only 1 LOD, a -inf min visible distance was needed). Now they don't disappear at close range.

ObjectMeshCluster is not good then (as all of my objects are different).

I changed the visible distance in the above code to 200, but it had no effect. So I suppose I did something wrong with the world cluster.

[sebastianbah]

You can check samples with WorldCluster:

<SDK>/data/samples/worlds/cluster_00.cpp

(from SDK browser: Samples & Demos -> Misc Samples -> Worlds demo -> cluster_00)

[istvan.szennai]

The documentation says:

 

"The real range of Cluster visibility directly depends on the object (surface) maximum visible distance and also its fade out distance. In case node references disappear at a closer distance, further increase of the visible distance does not any affect the performance in any way."

 

Depends on it how? Lets say the object has (-inf, 20) visibility distance, and the world cluster has 200. What should happen? I checked the sample, did everything like its written there, so theoretically its good.

 

edit: setting object max distance to inf and world cluster to 20 seems to work, so its good after all

[istvan.szennai]

I checked it with a slightly bigger model, and the task manager. Well it doesn't seem like it's unloading anything. At the program start memory usage (private working set) is 80 MB, despite that nothing is visible. As I walk through the building it increases, but if I go back to my starting point it doesn't go back down (edit: perhaps it's not the engine's fault, because TM acts the same with any other application).

 

edit: same with Process Explorer and GPU memory

Also, how is this "visibility distance" measured? Your posts suggests that its calculated from the center of objects, so it doesn't work good for big ones (and now I experienced this with some big walls). I think its not really a solution that I set the visdistance to infinity for large objects, because what if they consume a lot of memory?


 

[sebastianbah]

Hello,

Sorry for the late reply, some of our specialists are on vacations now, so some of your questions will be answered a little bit later.

For the present moment, you can find some additional information about memory and world management here:
https://developer.unigine.com/en/docs/1.0/principles/world_management/#bsp
https://developer.unigine.com/en/docs/1.0/code/engine_architecture/

Also build-in profiler can be helpful:
https://developer.unigine.com/en/docs/1.0/tools/profiler/

As for visibility distance, it is calculated from the nearest border of bounding box.

[binstream]

I checked it with a slightly bigger model, and the task manager. Well it doesn't seem like it's unloading anything. At the program start memory usage (private working set) is 80 MB, despite that nothing is visible. As I walk through the building it increases, but if I go back to my starting point it doesn't go back down (edit: perhaps it's not the engine's fault, because TM acts the same with any other application).

 

edit: same with Process Explorer and GPU memory

 

Also, how is this "visibility distance" measured? Your posts suggests that its calculated from the center of objects, so it doesn't work good for big ones (and now I experienced this with some big walls). I think its not really a solution that I set the visdistance to infinity for large objects, because what if they consume a lot of memory?

 

Actually, the engine caches loaded resources and tries to keep them in memory for some time - just in case if they will be required again. This is a normal behavior, since it increases overall speed and reduces disk load operations.

 

Memory consumption limits can be adjusted for all data streaming subsystems: https://developer.unigine.com/en/docs/1.0/start/console/?words=cache%20memory%20limit#file_system

[sebastianbah]

Hi, istvan.szennai

Did you got all your questions answered or did we miss something?

[istvan.szennai]

Hi,

sorry I had a break with evaluation, I'm catching up :)

 

edit: I made a video about my problem/question (regarding this visibility distance):

https://www.dropbox.com/s/ycs3ubg3ggmts3i/unistreaming.avi

The column (and other objects) disappear when they shouldn't.
They are all set up like:
 

<surface name="elem2::mesh0" material="material9" property="surface_base" max_visible_distance="20" max_fade_distance="1"/>

Note that these objects are "pre-transformed" (and I can't help it). So their object space position defines their actual position (world = identity). Perhaps that is the issue?

[sebastianbah]

Hello,

Could you please send us a minimal test scene? For example, the column separately from the other components.

It would be very helpful.


Thanks!