Material Libraries / Assignment in 2.7

[f.reichl]

Hi all,

we're running a project on Unigine 2.4 and we are now evaluating upgrading to the latest version, but it seems that the way material handling was changed breaks our current pipeline...

Background: We have an external Pipeline running to convert scenes from a different format. Our output are node, mesh, mat, and property files that are then imported into Unigine. We create the XMLs by Python-Scripts during that process. With the later versions, there's two issues that came up with this:

• Mat-Files only store a single material now, and as far as we see, there is no way to store multiple materials in a single file, is that correct? This is a bit unfortunate because we were building reusable material-libraries in single files that we'd need to change to parsing complete folders now
• Material assignment to surfaces can only be performed by GUID, not by name any more. This is a bigger issue since we were generating missing materials in our Pipeline and then assigned them by name to the new nodes (the input data already defines material names in a library, so we were "only" left with shading them approprietly for new materials). I've tried creating our own guid and storing it into the material file, but loading the files into Unigine generates a new GUID and stores it in the .meta file, making the assignment invalid

My questions would be:

• Is there any way to still store multiple materials in a single XML that we are just missing?
• Is there a way to assign materials to mesh surfaces in the XML by name?
• If not, why is the GUID we create in our mat files ignored?
• How "stable" is the current approach in your future plans? We're seeing a lot of (in our current pipeline) breaking changes in minor version releases and we're a bit worried about the future of our approach in general.

[silent]

Hi Florian,

Long time no see :) How are you?  

1. There is no option to store multiple materials in a single XML. Right now the closest analogue to the material library is the regular directory on a hard drive. Putting all the materials that somehow connected to a single folder should do the trick. You can pack them into .UNG if it's required to have a single file instead of multiple and then load it on engine startup.
2. It should be possible, basically you don't need to change anything in your process (maybe the file paths, no more partial paths). XML can store GUIDS or names. GUIDS will be 100% used when you will create node via Editor. Otherwise you will have to manage materials manually (and keep links always valid). With GUIDs there a little bit more of freedom.
3. It looks like a bug to me, need to check a small test scene to find out why it's happening.
4. New filesystem approach will remain mostly unchanged. We will think how to return missing functionality (like modifiers ) in the future releases, no more groundbreaking changes are planned (at least for 2.x versions). 
   
    

Quote

Material assignment to surfaces can only be performed by GUID, not by name any more. This is a bigger issue since we were generating missing materials in our Pipeline and then assigned them by name to the new nodes (the input data already defines material names in a library, so we were "only" left with shading them approprietly for new materials). I've tried creating our own guid and storing it into the material file, but loading the files into Unigine generates a new GUID and stores it in the .meta file, making the assignment invalid

That's sounds like an issue and needs further investigation. Is there any chance to take a look at a small test scene with this behavior?

Thanks!

[f.reichl]

4 hours ago, silent said:

Hi Florian,

Long time no see :) How are you? 

Great, thanks :-)

4 hours ago, silent said:

1. It should be possible, basically you don't need to change anything in your process (maybe the file paths, no more partial paths). XML can store GUIDS or names. GUIDS will be 100% used when you will create node via Editor. Otherwise you will have to manage materials manually (and keep links always valid). With GUIDs there a little bit more of freedom.

How would I reference a material by name? I tried creating a material in the path "data/materials/test_material.mat". In the XML, I can only assign it to a surface using "material=<guid>". If I try it by name, the material shows up as red / unknown in the editor.

I tried:

• name="test_material"
• name="materials/test_material"
• name="materials/test_material.mat"

Or did I misunderstand?

About the overwriting of the GUID we created, I'll try to reproduce the issue in a minimal test scene.

[f.reichl]

Update: I think I found the issue why our GUIDs were ignored: They do not conform to what Unigine expects. I've been using a 64bit hex UUID generated from python:

uuid.uuid4().hex

With this, the material is parsed without issues, the GUID in the mat file stays unchanged, but the .meta file contains a new GUID. Interestingly, if I do not provide a GUID at all in the mat file, the file cannot be read - this looks like a bug to me?

[silent]

Hi Florian,

We use SHA1 algorithm to generate GUIDs. Sometimes GUID is generated from file path (that's can be the reason of different values).

Could you please send us a small test scene where after Editor launch GUID in .meta file is changing? That will surely help to find the root cause faster.

Thanks!

[f.reichl]

Small testscene for the behavior: data.zip

In a new project, copy the contents into the data folder and load DemoProject.world

In this case: The cube references materials/testmat by GUID, but the GUID does not conform to the Unigine standard. Unigine does not complain about this, but can't find the material, and after scanning the assets once, a testmat.mat.meta file is created that contains a different GUID than the one specified in the mat file.

I'd expect to at least get a warning about the invalid GUID on loading the material?

[silent]

Hi Florian,

Thanks for the test scene.

There is no warning in console due to performance (loading speed) reasons. We do support only SHA1 hash in GUID, so other strings are simply skipped.

We will think how we can improve this without losing much speed.

Also,  material tag in XML can only have GUIDs (not names), sorry for misleading.

[f.reichl]

Thanks for clarifying. It's a bit confusing though that the GUID tag in the XML is kept without complaint, but if I do not supply a GUID at all, the material fails to load.

About the tags in XML: The same applies to properties, correct? So I also can't link properties by name any more?

This will unfortunatly bring a bunch of changes in our processing pipeline... but we'll manage, and from what I've seen from the last versions, it's definitely worth the upgrade, quite an improvement :-) Voxel GI is quite a perfect solution for our use case, we were actually thinking of implementing this ourselfs over a year a go.

Thanks for the fast support again..

Edited October 11, 2018 by f.reichl

[silent]

Hi Florian,

Yeah, with properties pipeline exactly the same.

We do recommend to use more  (if possible) engine API - Materials / Properties classes that will allow you to easily inherit and save materials and properties with valid GUIDs :)

Thanks!