Best way to run custom async tasks

[ExBemined]

We have a piece of code that takes too long to be executed within one frame that cannot be optimised (it basically calls an external SDK to generate an image). It's okay though for this to happen at a lower framerate, the image doesn't need to be updated at 60fps, so the code could safely run in a worker thread and in the main update we check whether the task is finished and set the image to the widget when it is.

What would be the best way to do this? We can of course create a thread of our own and give it it's own main loop, but when looking at the microprofile I noticed Unigine had 'async task' threads in there from a thread pool, which might be more efficient to use then our own managed thread. Can you create custom tasks in here from the C++ API? I know about CPUShader, but it says it's not suitable for tasks as these shaders need to finish before the next swap.

[honya]

Hi,

I using https://developer.unigine.com/en/docs/2.11/api/library/common/class.thread?rlang=cpp that i extend with work logic.

I using it for async work and also for sync work and works fine for me.

Honya

[christian.wolf2]

Hi Bemined,

 

would second honyas approach. UNIGINE Thread wrapper class is pretty straightforward and easy to use. For our logic we created on top of it an customized ThreadPool-System that handles UNIGINE Thread-Wrapper pretty decent without loosing advantages of them.

You can contact me directly and I can share with you my implementation.

 

Best

Christian

Edited May 14, 2020 by christian.wolf2

[rohit.gonsalves]

Hello,

Third to the same opinion.

I am using Unigine::Thread for many of async tasks, Network commands send/ Receive, Security implementation in different worker thread, Reading Media files from hard disks and reading Video textures from Video I/O cards like AJA and Black magic. All are in separate thread and when they are ready, I just roll them out with main rendering thread. Other synchronization you need to handle.

But pretty much easier and a working one.

Rohit 

[ExBemined]

I don't doubt he Unigine::Thread works, I was just wondering whether Unigine offered an even better solution. When working with custom threads there is always a (small) risk of running too many tasks at the same time. An engine managed async task pool can manage to not run more tasks at the same time as safe. We had problems in the past where additional threads caused framerate to become unstable as they started to fight over the same CPU core. Getting a smooth 60fps output is an important goal for us.

[christian.wolf2]

57 minutes ago, Bemined said:

When working with custom threads there is always a (small) risk of running too many tasks at the same time. An engine managed async task pool can manage to not run more tasks at the same time as safe.

 

That is (in my usecases) still the same. I have tested the thread system in various cases and come to conclusion that is totally safe to run X number of threads, where X is number of CPU cores - 1. If I executed the same amount of threads than cores I have, the system get incredible slow.

Same with thread managing (not running more threads that your CPU can handle), thats why I created an custom thread pool system. Also (and that is maybe something the UNIGINE team have to answer) the question is that the engine is running various async jobs on their own and didn't know if your created thread jobs are handeled concurrently or leads to bottlenecks in some use cases. To be fair, I never run into any issue with engine specific jobs where framerate were dropped.

[leon.dong]

The unigine thread system is just a simplest wrapper for each platform .

So regarding multiple threading issue , I recommend just design by yourself and use rest thread solution.

 

[andrea.padovani]

It's a really old topic, but I still need to implement something similar.
Actually I'm building a C++ project starting from VR template, I've done a basic function to read a value from an external backend, so I just need to update a variable value with the result of this function.
If I put the call to my function in the int AppWorldLogic::update() of course I'll crash all the performance since it's trying to hit the backend as many times he can, and delay the rendering until response.
So the best way I think is to start a separate thread and call the backend just 1 time per second for example, and then update the value when it's ready, the problem is that I've no idea on how to do this!

If someone has some example code it will be really appreciated! I'm looking at thread sample in the browser but is not really clear to me how to merge in my AppWorldLogic

Thanks!

[christian.wolf2]

Hi Andrea,

do you have some overall experience with Multi-Threading before or is that a new topic for you? So basically use the already implemented Thread-class from UNIGINE and extend it for your need. For every job, you want to to asynchronously, create a new child class from the Thread-class and override the process()-function, which will be called in an seperate thread. Here we have an example for an asynchronous job:

Spoiler

#include "DataFetcher"   //<- another class that will get data over the network

class MyHeavyJobThread : public Thread
{
private:

	Entity* entity;

public:

	MyHeavyJobThread(Entity* ent) : entity(ent)
	{
		//maybe to some basic stuff for async preparation

	}


    //this is your actual function that have some heavy weight.
	void process() override
	{
        DataFetcher* fetcher = new DataFetcher();
      
		//1. Call some external data from the backend
        DataChunk* dataToGet = dataFetcher->RetreiveDataForEntity(entity->getID())
        
        //2. Update the retreived data in our entity
        entity->AsyncUpdateFetchedData(dataToGet);
      
        delete fetcher;
	}
  
}

 

In the above mentioned example we have a simple ThreadJob, that is derived from the UNIGINE-Thread class. We provide this class a pointer to our Entity-class, which represents an object in our world (can be a player or some AI-controlled enemy). He might also have an DataFetcher-class which will setup an connection to your backend, retreive some data (here the DataChunk-class) from there and closes the network connection afterwards. After doing all this work, we will update our entity by giving him the retreived data so he can do some internal stuff as well (by calling AsyncUpdateFetchedData()).

The entity might be something like this:

Spoiler

class Entity
{
private:

	MyHeavyJobThread* jobThread;		//a pointer to our async thread
	int id;								//an unique ID for the entity

	ObjectMeshDynamicPtr mesh;			//a mesh, that will represent this entity in our world

	/* all the other variables */


	Mutex dataFetchingMutex;			//a mutex for save data access

	float curTimeUntilNewDataFetch;		//a timer value that will create a new job when it runs out

public:



	//constructor
	Entity()
	{
		id = 0;

		jobThread = new MyHeavyJobThread(this);		//we create an async job during object creation. But we can do this also somewhere else
		curTimeUntilNewDataFetch = 0.f;				//time until we fetch new data

		//initialize your other variables here

	}


	//will be called from our AppWorldLogic every frame
	void update(float iFPS)
	{
		curTimeUntilNewDataFetch -= iFPS;

		//when we run out of time we start a new fetch
		if (curTimeUntilNewDataFetch <= 0.f)
		{
			if (jobThread->isRunning())
			{
				//fetching data from previous call is not done.

				//we can do
				//jobThread->terminate();
				//to kill it

			}
			else
				jobThread->run();


			curTimeUntilNewDataFetch = 1.f;		//we will do another update in 1 sec

		}


		//we block writing for other threads. Thus prevents e.g. AsyncUpdateFetchedData to overwrite our data
		//the lock is valid until the end of the function
		ScopedReaderLock(dataFetchingMutex);

		//use other variables from your entity (like the mesh her)

	}



	void AsyncUpdateFetchedData(DataChunk* dataChunk)
	{
		//prevents the main-function from reading data in the meantime
		//the lock is valid until the end of this function
		ScopedWriterLock(dataFetchingMutex);


		//update your data

	}

}

 

 

During our object creation we can also creat a new async job, that we can start with run() whenever we want. So in the entities update() function we let a timer run and after a second, we try to run our job, when it isn't still running.

Notice our Mutex dataFetchingMutex. This variable prevents other threads (also the main thread) for executing a peace of code while another thread is already busy doing some stuff. In the above example, the update()-function has a ScopedReaderLock(). After this line of code you can savely reading data because we forbid other threads to writing (still, other threads can also read data in the meantime). In the AsyncUpdateFetchedData(), which will be called from our thread-function, we have a ScopedWriterLock. This prevents other threads (also the main thread) from reading and writing the data. So we are save to manipulate our data.

The AppWorldLogic-class is simple: create in your init function a new entity, call his update function in the AppWorldLogic::update and destroy it in the shutdown()-function. In your destructor, make sure, that you will call stop() for your thread-job, so it will be safely exited when running.

 

A short note to writing multi-threaded code: While it is not hard to create and call new thread-jobs the most difficult part is either save access your data (never reading or writing some data without locking, because we might get inappropiate behavior) or execution time. In the above mentioned example, calling some heavy code in a thread job is useless, when AsynUpdateFetchedData() takes a lot of time as well, when writing or processing the data internally. Because if this one is also very time-consuming, your main-thread will wait in the Entity:update() function, until the Mutex can be blocked for reading. So always take care, that blocking some code from execution should take as less time as possible.

 

Best

Christian