[SOLVED] Properties: Memory leak or misuse of array-of-structure property parameters

[helmut.bressler]

Hello,

if I walk through the following property, I get a significant memory leak (around 1 MB / second at 700 fps).

<?xml version="1.0" encoding="utf-8"?>
<property version="2.13.0.0" name="SomeProperties" manual="1" editable="0">
  <struct name="EmitterStruct">
		<parameter name="emitterProperty" type="property"/>
		<parameter name="emitterNode" type="node"/>
  </struct>
  <parameter name="emitterProperties" type="array" array_type="EmitterStruct"/>
</property>

For reading the properties, I use the following code:

int AppWorldLogic::init()
{
	// Write here code to be called on world initialization: initialize resources for your world scene during the world start.

	PropertyPtr someProp = Properties::findProperty("SomeProperties");
	somePropChild = someProp->getChild(0);
	paramEmitterPropertiesArray = somePropChild->getParameterPtr("emitterProperties");

	return 1;
}

int AppWorldLogic::update()
{
	bool doTextOutput = (Game::getFrame() % 5000) == 0;
	if (doTextOutput)
	{
		Log::message("Frame %d:\n", Game::getFrame());
	}

	for (int i = 0; i < paramEmitterPropertiesArray->getNumChildren(); i++)
	{
		PropertyParameterPtr paramStruct = paramEmitterPropertiesArray->getChild(i);
		PropertyParameterPtr paramEmitterProperty = paramStruct->getChild("emitterProperty");
		PropertyPtr propsEmitter = paramEmitterProperty->getValueProperty();
		PropertyParameterPtr paramEmitterNode = paramStruct->getChild("emitterNode");
		NodePtr node_emitterNode = paramEmitterNode->getValueNode();

		if (doTextOutput)
		{
			Log::message("\t%s is assigned to prop %s\n", node_emitterNode->getName(), propsEmitter->getName());
		}
	}

	return 1;
}

I suspect, that running

PropertyParameterPtr paramStruct = paramEmitterPropertiesArray->getChild(i);

actually causes the leak.

 

My question is now, if this leak is caused by a bug or am I misusing the property parameter API?

 

Thanks

Helmut

[silent]

Hi Helmut,

Try to add somePropChild->setOwner(true) after you calling getParameterPtr:

paramEmitterPropertiesArray = somePropChild->getParameterPtr("emitterProperties");
somePropChild->setOwner(true); // add this line

There was a bug in 2.13.x releases with properties API interface (it was fixed in 2.14.x).

Thanks!

[helmut.bressler]

Hello Silent,

unfortunately it doesn't fix the memory leak. I should have mentioned that I am using the 2.14.1 (Windows Engineering) SDK for this repro case. But the issue came up while we were porting one of our projects to 2.15. alpha, so it is most likely also showing up there.

I have extended my repro case a little bit, and it seems that iterating through arrays as well as iterating through struct members causes a memory leak:

The prop file looks now like this:

<?xml version="1.0" encoding="utf-8"?>
<property version="2.13.0.0" name="SomeProperties" manual="1" editable="0">
  <struct name="EmitterStruct">
		<parameter name="emitterProperty" type="property"/>
		<parameter name="emitterNode" type="node"/>
  </struct>
  <parameter name="emitterProperties" type="array" array_type="EmitterStruct"/>
  <parameter name="someFloat" type="float" max="200000000">0.1</parameter>
  <parameter name="emitterProperties" type="array" array_type="EmitterStruct"/>
  <parameter name="floatArray" type="array" array_type="float"/>
  <parameter name="someEmitter" type="EmitterStruct"/>
  <parameter name="someNode" type="node"/>
  <parameter name="someProperty" type="property"/>
</property>

 

On the C++ side I am doing:

int AppWorldLogic::init()
{
	// Write here code to be called on world initialization: initialize resources for your world scene during the world start.

	PropertyPtr someProp = Properties::findProperty("SomeProperties");
	somePropChild = someProp->getChild(0);
	paramEmitterPropertiesArray = somePropChild->getParameterPtr("emitterProperties");
	//somePropChild->setOwner(true);

#ifdef READ_FLOAT_ARRAY
	paramReadFloatArray = somePropChild->getParameterPtr("floatArray");
#endif

	return 1;
}

int AppWorldLogic::update()
{
	bool doTextOutput = (Game::getFrame() % 5000) == 0;
	if (doTextOutput)
	{
		Log::message("Frame %d:\n", Game::getFrame());
	}

#ifdef READ_FLOAT
	PropertyParameterPtr paramSomeFloat = somePropChild->getParameterPtr("someFloat");
	if (doTextOutput)
	{
		Log::message("\t%s has value %f\n", paramSomeFloat->getName(), paramSomeFloat->getValueFloat());
	}
#endif

#ifdef READ_STRUCT_ARRAY
	for (int i = 0; i < paramEmitterPropertiesArray->getNumChildren(); i++)
	{
		PropertyParameterPtr paramStruct = paramEmitterPropertiesArray->getChild(i);
		PropertyParameterPtr paramEmitterProperty = paramStruct->getChild("emitterProperty");
		PropertyPtr propsEmitter = paramEmitterProperty->getValueProperty();
		PropertyParameterPtr paramEmitterNode = paramStruct->getChild("emitterNode");
		NodePtr node_emitterNode = paramEmitterNode->getValueNode();

		if (doTextOutput)
		{
			Log::message("\t%s is assigned to prop %s\n", node_emitterNode->getName(), propsEmitter->getName());
		}
	}
#endif

#ifdef READ_FLOAT_ARRAY
	for (int i = 0; i < paramReadFloatArray->getNumChildren(); i++)
	{
		PropertyParameterPtr paramFloat = paramEmitterPropertiesArray->getChild(i);
		if (doTextOutput)
		{
			Log::message("\t%s has value %f\n", paramFloat->getName(), paramFloat->getValueFloat());
		}
	}
#endif

#ifdef READ_STRUCT
	PropertyParameterPtr paramSomeEmitter = somePropChild->getParameterPtr("someEmitter");
	PropertyParameterPtr paramEmitterProperty = paramSomeEmitter->getChild("emitterProperty");
	PropertyPtr propsEmitter = paramEmitterProperty->getValueProperty();
	PropertyParameterPtr paramEmitterNode = paramSomeEmitter->getChild("emitterNode");
	NodePtr node_emitterNode = paramEmitterNode->getValueNode();
	if (doTextOutput)
	{
		Log::message("\t%s is assigned to prop %s\n", node_emitterNode->getName(), propsEmitter->getName());
	}
#endif

#ifdef READ_NODE
	PropertyParameterPtr paramSomeNode = somePropChild->getParameterPtr("someNode");
	if (doTextOutput)
	{
		Log::message("\t%s has is referencing node %s\n", paramSomeNode->getName(), paramSomeNode->getValueNode()->getName());
	}
#endif

#ifdef READ_PROPERTY
	PropertyParameterPtr paramSomeProp = somePropChild->getParameterPtr("someProperty");
	if (doTextOutput)
	{
		Log::message("\t%s has is referencing property %s\n", paramSomeProp->getName(), paramSomeProp->getValueProperty()->getName());
	}
#endif

	// Write here code to be called before updating each render frame: specify all graphics-related functions you want to be called every frame while your application executes.
	return 1;
}

More precisely my observations are the following:

Defining any combination of READ_NODE, READ_PROPERTY or READ_FLOAT works as expected, nothing is leaking.

Once I enable any of READ_STRUCT_ARRAY, READ_FLOAT_ARRAY or READ_STRUCT, I get a memory leak, irrespective of if I invoke SetOwner() on somePropChild as you suggested. If I only define READ_STRUCT, the heap is growing slower than with READ_STRUCT_ARRAY defined. But at 700fps it is easy to observe all of them, after a couple of seconds.

Thanks

Helmut

 

[silent]

Hm, I was unable to reproduce.

Is there any chance that you could prepare a minimal test project (based on an a newly create project via SDK Browser) and send us couple of sources and additional files (such as properties and world)?

Thanks!

[helmut.bressler]

Hello Silent,

yes, please create a Visual Studio 2015+ C++ project with the Engineering SDK using the Float/development version.

Please copy the attached property files +  the .meta files into the data folder (There is no need for updating the world file, I'm referencing only nodes from the default scene - which seem to have a consistent GUID)

Please use the attached AppWorldLogic files for building the executable.

Once the application is running, you should be able to observe a growing heap and memory consumption in the profiler. The memory leak should show up in both, the debug and the release build (You may have to increase the render_max_fps to 1000 or something). Originally I have been using the 2.14.1 SDK, but I tested now the setup steps I'm describing here with a fresh 2.15 alpha project and the result is the same.

Thank you for looking at that, Silent

Helmut

some.prop some_other.prop some_other.prop.meta SomeOtherProp_0.prop SomeOtherProp_0.prop.meta SomeOtherProp_1.prop SomeOtherProp_1.prop.meta SomeProperties_0.prop SomeProperties_0.prop.meta AppWorldLogic.cpp AppWorldLogic.h some.prop.meta

[silent]

Thanks for the test scene. I've successfully reproduced the memory leak that you mentioned.

Our devs will try to look at this issue as soon as possible.

[cash-metall]

yes indeed it is a memory leak!
Default behavior is incorrectly configured for the PropertyParameterPtr. 

you need to add the setOwner(1) manually when you take any PropertyParameterPtr. its temprary object, Engine does not control its lifetime...

#ifdef READ_STRUCT_ARRAY
	for (int i = 0; i < paramEmitterPropertiesArray->getNumChildren(); i++)
	{
		PropertyParameterPtr paramStruct = paramEmitterPropertiesArray->getChild(i);
paramStruct->setOwner(1); // setOwner(1) for each PropertyParameterPtr to awoid memory leak
		PropertyParameterPtr paramEmitterProperty = paramStruct->getChild("emitterProperty");
paramEmitterProperty->setOwner(1);
		PropertyPtr propsEmitter = paramEmitterProperty->getValueProperty();
		PropertyParameterPtr paramEmitterNode = paramStruct->getChild("emitterNode");
paramEmitterNode->setOwner(1);
		NodePtr node_emitterNode = paramEmitterNode->getValueNode();
	}
#endif

#ifdef READ_FLOAT_ARRAY
	for (int i = 0; i < paramReadFloatArray->getNumChildren(); i++)
	{
		PropertyParameterPtr paramFloat = paramEmitterPropertiesArray->getChild(i);
  paramFloat->setOwner(1);
	}
#endif

#ifdef READ_STRUCT
	PropertyParameterPtr paramSomeEmitter = somePropChild->getParameterPtr("someEmitter");
paramSomeEmitter->setOwner(1);
	PropertyParameterPtr paramEmitterProperty = paramSomeEmitter->getChild("emitterProperty");
paramEmitterProperty->setOwner(1);
	PropertyPtr propsEmitter = paramEmitterProperty->getValueProperty();
	PropertyParameterPtr paramEmitterNode = paramSomeEmitter->getChild("emitterNode");
paramEmitterNode->setOwner(1);
#endif

we will fix this in the next release.

[helmut.bressler]

I can confirm that the setOwner() calls are indeed fixing the leak.

Once the fix is implemented (I would presume it will be part of the 2.15 release version), do I need to remove the setOwner() calls?

Many thanks

Helmut

[silent]

Quote

Once the fix is implemented (I would presume it will be part of the 2.15 release version), do I need to remove the setOwner() calls?

You can leave them untouched or remove, there should be no big difference. But if you will remove them and memory leak will appear again - that would be a bad sign :)

[cash-metall]

2 hours ago, helmut.bressler said:

setOwner() calls are indeed fixing the leak.

I checked the code more carefully (and fixed it in engine).
the memory leak was only in the PropertyParameter::getChild and PropertyParameter::getParent methods. You need add setOwner only in these cases.
after migration on 2.15 setOwner can be removed. if it remains somewhere, nothing terrible will happen.