Depth buffer flexibility

[Greg.Mildenhall]

We're somewhere through the process of migrating from 2.3.1 to 2.6.1 and I thought I would try to use the new rendering callback system to achieve an affect we've been wanting for a while. (side-note: is there sample/tutorial code out there that uses these callbacks? Edit: GBufferRead looks a likely candidate but it's not yet clear to me from the source exactly what it does)

As you may remember, we've modified the terrain shaders to support translucency. There are other translucent objects that intersect the terrain that we would also like to visualise translucently. Our plan was to pre-render the terrain depth and then render the translucent objects in two halves before and after the terrain.

There are two obstacles I see here, and hopefully you can explain how best to get around them.

The first problem is how to pre-render the depth. It seems like there's still no way to add a new pass that does that? And no way to switch on depth_mask during passes that don't currently support it? For some weird reason the callback for each pass happens _before_ all the state setup, so anything we override will get immediately lost!

We don't currently use the standard depth_pre_pass for anything, so we could use that and then afterwards swap out the texture using TextureRender, right? Except the pre-pass doesn't appear to have a callback and instead happens within the OPACITY_GBUFFER callbacks. :( Are there any other suitable passes we could hijack? Or will we need to render it in a separate viewport? How easy is it to render the same object in two different viewports with two different materials?

The second problem is with making the first of the two halves only draw where its depth is _beneath_ the terrain. There doesn't seem to be a way to set DEPTH_LEQUAL? I suppose I could get around that by writing to a stencil in the pre-pass, but I can't work out how to apply setStencilFunc to a particular material instead of the whole pass? Otherwise maybe I could use the depth to write a mask into some channel during the auxiliary pass, then discard ambient pass fragments based on the auxiliary texture. But that sounds really overcomplicated.

What information, advice, suggestions do you have for us?

Edited February 2, 2018 by Greg.Mildenhall

[Greg.Mildenhall]

Another idea (if it's not possible to set the depth function) would be to read the depth buffer in as a colour texture and somehow convert that into depth values which could be compared against for a conditional discard. Are there any good examples around of such a calculation?

[morbid]

Hello, Greg!

Sorry for late response. Your question requires a little research. It could take some time for our engine team.

How critical is this for now?

[Greg.Mildenhall]

It's not time-critical for us, we'd rather find a good way to do it than the quickest hack. But if there are any existing examples out there that relate to any of it, we could make use of them immediately. There will likely be other places where we'd like to make use of the new flexibility in rendering. In particular: if we're pre-rendering terrain depth, we'll try find a way to use it to render (opaque, unlit) decals onto our transparent terrain.

[Greg.Mildenhall]

Bump.

Any examples out there of using the depth buffer for something other than a GEQUAL depth_test?

[silent]

Hi Greg,

Sorry, we didn't have chance to look into this yet. Devs are preparing the release, so we have very limited resources now.

But we didn't forget about your request :)

[Greg.Mildenhall]

Since we're still waiting for guidance on any of these things, I've started playing around with it myself. First stumbling block: Unigine::Renderer::get()->getViewport() and Unigine::Render::get()->getViewport() both return nullptr! It's going to be hard to addCallback() without a Viewport. Could I be calling them too early? I don't think I am.

At this point the only reason I wanted to add a callback was because Unigine::Renderer::get()->getTextureOpacityDepth() was also returning nullptr, and it looked like maybe I could only get a result from that during a render. (it's OK if the texture gets destroyed after the render, I just need to read its parameters so I can clone it with my own Texture->create2D()) So alternate ways of getting those parameters will suffice for the moment, but in the long term I'm going to want to add callbacks.

[Greg.Mildenhall]

Now I can see the 2.7 API Migration page, it looks like maybe addCallback() never worked on the main viewport, and that's why a duplicate mechanism had to be added to Render. Am I guessing right?

I'm also hopeful that if I render to an identically-sized viewport in a callback from the main render, it _won't_ end up sharing its GBuffer so I can use the viewport's depth texture later in the frame without needlessly copying it aside as a post-process? It would be nice to have that confirmed.

[silent]

Hi Greg,

There was indeed number of callback improvements. Could you please share some test code that you are using to identify this issue?

Thanks!

[Greg.Mildenhall]

If I add 

		ObjectPtr screen = Object::cast(Editor::get()->getNodeByName("depth_screen"));
		MaterialPtr screen_material = screen->getMaterialInherit(0);
		screen_material->setTexture(screen_material->findTexture("emission"), renderer->getTextureOpacityDepth());

at the end of gbuffers_ready_callback() in GBufferRead.cpp (2.6.1) I get the behaviour I'm looking for - directly using the depth buffer as a texture for a later draw in the main viewport. If I comment out the setViewport() call, it no longer works. I'm suspect this must be because the two viewports share rendertargets when they're the same size.

Unfortunately my pre-rendered depth buffer needs to be the same size as the main one to prevent artifacts in the effect I'm asking for here. (But I'm still just as keen for suggestions of other approaches as I was when I asked for them three months ago. Hopefully we can get some support now the release is out.)

[silent]

Hi Greg,

After some internal discussion with colleagues we find out that requirements for the scene:

1. Rendering transparent objects below the terrain
2. Rendering transparent terrain
3. Rendering transparent objects on terrain
4. Correct sorting between all of that objects

will require a different approach on engine level (implemetings some order-independent transparency algorithm). These alrogithms all pretty much performance unfriendly and may reduce the overall frame time of the scene down to 50ms on current high-end GPUs like 1080TI. As far as I remember, you have Intel HDs mostly, so these algorithms will not work at all or produce stable 1 frame per second on almost any scene.

Changing the depth to LEQUAL will result on all objects to changes their depth functions and they will have other rendering artifacts. And somehow change the Gbuffer and modify it later will also give you no results that you expect.

The easiest solution that came up to our minds (that will at least work on Intel HD) is just to abandon usage of ObjectTerrain and generate a simple mesh from the heightmap. This mesh can use the default mesh_base material with transparency. We do also have an convenient rendering order setting (per material) that will help you to render some objects before the terrain mesh and after. 

However, If you can send us your minimal working project or a test scene and at least a couple of pictures of what you really want to achieve - we would see what else we can do to improve this situation. 

Thanks!

[Greg.Mildenhall]

Yeah, given our min spec we're not considering full OIT. Just 3 layers. Our transparent objects don't typically intersect each other and we're not very interested in their relative depths, so for now the existing sorting is fine between those. The only significant intersections we need to visualise are between objects and the terrain. Currently neither draw order (for transparent objects vs transparent terrain) is giving us all the visual information we need to depict. Drawing after the terrain gives us just what we want for the above-ground pixels, so it seems we need a special method to draw the _only_ the below-ground part of the object _before_ the terrain. Almost like a single iteration of dual depth peeling.

Hopefully that's easy enough to follow and I don't need to make a picture?

If we can achieve the LEQUAL method for the below-ground pass, the artifacts won't be a problem because we just won't write the objects' depth. It's not a big deal if a further object draws after a nearer one - as I said it isn't very important for us to visualise that correctly.

I can't see a reason why it would be easier with a mesh than with ObjectTerrain? We are already able to use the group/order setting for our transparent terrain material.

[silent]

Hi Greg,

Sorry for the late reply. Even 2 layer can become an issues with a relatively simple gometry :)

We will try to add option to customize depth testing to the engine in the upcoming 2.7.1 update. However, we didn't decide yet how to do it better (per material setting or only for base material). Moreover, we have plan to expose polygon culling modes (maybe changing it will also help in some cases to fix the artifacts).

Right now you can also try to use Multiple environment probes checkbox in the States tab that will enable correct sorting for the alpha blended object (but with two or more overlapping separate objects it will produce artifacts).

Quote

I can't see a reason why it would be easier with a mesh than with ObjectTerrain? We are already able to use the group/order setting for our transparent terrain material.

In terms of the future updates (ObjectTerrain will be removed completely in 2.8), but mesh will continue to render. Mesh can be easily modified in any DCC tool and simplfied as you need, some polygons can be manually changed to avoid artifacts. Less shaders to maintain - only mesh_base :) But if ObjectTerrain is OK to you, you can use it in 2.7 and 2.7.1 as well.

Thanks!

[Greg.Mildenhall]

A belated update: in the end, I got the technique working the way I described above (rendering transparencies in two halves, either side of the transparent terrain) without too much of a performance hit.

An answer to any of the questions I raised would still help me improve it, but that's not why I'm reviving the thread. It's because locking the terrain into the transparency pass has painted me into a corner somewhat with another feature we'd been planning to implement.

Presently, we have to give up on transparency and switch back to deferred terrain in order to apply decals. To get decals and transparency at the same time, I'd like to pre-render the terrain and decals into their own scene (I already pre-render the terrain separately to get my depth pre-pass so this part is easy enough), then overlay that image as an object in the transparencies pass. (it has to stay part of this pass because we use setOrder() to interleave it correctly with the two halves of the other transparent objects)

I think this should be straightforward enough, but I'm writing here because I'm hoping there might be a sample out there somewhere of how to fake a fullscreen quad into the transparencies pass? I'll need to make my own shader and material obviously, so it's just generating/updating the geometry that I'm concerned about.

I'm also slightly concerned that faking the depth in the pixel shader won't be early enough to make everything work, but my guess is that'll probably be fine?

[Greg.Mildenhall]

Bump.

TLDR: "I'm hoping there might be a sample out there somewhere of how to fake a fullscreen quad into the transparencies pass?"

[Greg.Mildenhall]

OK, I've muddled my way through this one on my own. Any suggestions you have will still be useful for me to make sure I'm doing things the Right Way(tm) and for others who might face the same task in the future.

[silent]

Hi Greg,

Could you please share your results with us as well? We need to check a small test scene to see what else we can suggest. How do you think will be this possible? If you also can add some comments to the sources where to look at - it would be just perfect :)

I would start with adding a callback when transparent bass begins or ends (depending on what  you want to achieve) (CALLBACK_BEGIN_TRANSPARENT / CALLBACK_END_TRANSPARENT) and perform a render to color texture with renderPostMaterial.

Thanks!

[Greg.Mildenhall]

There's a partial implementation of this in the newer thread I called "renderNodesTexture2D()" so I'll try to remember to put a fuller implementation into the sample when we've got it all working. But I just noticed a misconception in your post above that I should set straight: we can't use CALLBACK_BEGIN_TRANSPARENT or CALLBACK_END_TRANSPARENT because that would break the "order" mechanism. In our case it needs to be in the middle of the transparencies pass because of the depth-peeling implementation I mentioned earlier in this thread. (that's why I asked about this here rather than create a new thread for it)