shader cache corruption with multiple simultaneous launches


photo

Recommended Posts

Hello,

The shader cache (shader_d3d11_debug.cache) will always get corrupted in the following condition:

  1. Have a basic world with some materials, and precompile all shaders in the editor.
  2. Launch ONE instance of your runtime (default launch_debug.bat for example). All will go well, the debug cache will be written correctly (can take some time)
  3. Close your launched instance
  4. Create a new fancy material in your editor (I tried with a material with emission, multiplicative mode), and apply it to some node. Precompile and save.
  5. Launch AT LEAST TWO instances simultaneously of your runtime (eg. with a bat with two "start" inside).

You'll get shaders compilation error and a corrupted debug cache.

Understandably, the two instances tried at the same time to update the same file and write over each other, I guess.

As this process is quite common here (creating/modifying materials and launching a bunch of master/slave on the same machine), I run quite frequently in this "bug".

Maybe in some future version, some kind of file lock mechanism could be put in place so only one instance can write while the others wait for access. Probably not an easy task, and I already can imagine some pitfalls, but this could help and prevent some headache for our "less tech savvy" users :-)

Thanks!

Side question: why isn't the debug file generated by the editor? this would solve the issue too :) 

Link to post

Any feedback?

Did the debug cache generation changed since 2.9? I have one case where the generation takes up to 26 minutes (!!!) (our airport vdb). I don't remember having such a long wait time before. Does that mean the upgrade had an issue? (we use a few mount points, each having been upgraded individually).

 

Link to post

Hi Stephane,

DX11 shaders compilation time is 5-10 times slower than OpenGL. For the fast iterations you can use OpenGL, for the final build you can use DX11. 

Editor shader compilation algorithm can skip some shaders combinations and when you launch two instances simultaneously you get simultaneously file write from multiple engine instances that lead to file corruption.

To avoid this you can either run instances with some delay so the first launched application will generate missing shaders cache or specify for the second instance different directory for storing the shader cache (-project_name parameter).

Thanks!

How to submit a good bug report
---
FTP server for test scenes and user uploads:

Link to post