Thoughts on Testing

[jaaaaa]

Hello Folks,

I would be interested in experiences with testing and Unigine, especially for larger projects.

What kind of toolchain do you use for continuous integration and how do you unit test? 
Interacting with the Unigine API often needs Unigine to be initialized which makes tests tedious.
Do you use some kind of mocking?
Do you test your rendered output? How do you compare test results and expected outputs? I would guess pixelwise comparison can be achieved seldomly.

I'm new to both testing in C++ and Unigine and would appreciate any input on this topic.

Kind regards

[leon.dong]

Wating for solution .....

[jaaaaa]

Maybe some insights from Unigine itself?
Do you have some kind of Integration testing set up with Unigine?

Could you share insights on how you test, what you test and what you don't test (automatically)?

I'd appreciate any input.

Kind regards

[silent]

ipg_jallmenroeder

We have automated test benches for the performance (checking the profiler values (CPU / GPU) and compare them each and every day) and critical CPU-side tasks (like filesystem, assets, bounds and so on).

We don't use any automated graphics testing due to major rendering changes (that we can do very often) that forces us to regenerate the references screenshots very often. Also due to the difference in the GPU models / GPU drivers / OS / API absolutely the same scene can be rendered slightly differently (for example, TAA can jitter different pixels). So, it's pretty much hard to find the right metrics that will say that this generated picture has artifacts (compared to the reference one). Right now all the graphics are tested manually via samples or internal projects.

The best approach (at least in theory) is to check the RAW G-Buffers instead, but there can be also some discrepancy across the different PC configurations. We didn't tried it yet, so I don't know if it will work or will produce the same results that final image comparison.

[Lales.Charles]

Good, got partially an answer ;-)
Any update on this maybe?

Kind regards,
Charles

[silent]

What kind of information are you looking for? As I already mentioned before, there is no simply way to automate the final rendering tests, but you can try to it on your side.

You have full access to every rendering stage and GBuffer textures, so you can write a separate logic that would try to produce as stable screenshots as possible (with fixed cameras position, fixed object's positions and so on) and compare them after each update.

If you will eventuall find a better way to do it - please let us know :)

[Lales.Charles]

Hi,

Were just wondering if anything could have pop-up since summer.
Thanks for feedback, and suggestion for focus on GBuffer textures.

Kind regards,
Charles

[silent]

Oh, I see. Nothing new, I'm afraid :)