Rendering Sequence

Unigine uses a multi-pass rendering technique to produce various visual effects, including dynamic lighting, post-processing, and so on. Some of the passes re-render scene geometry, some do not, therefore, some polygons are rendered multiple times per frame. The actual number of polygons to be rendered per frame by the GPU depends on polygon count of the static geometry, used effects, culling efficiency and a lot of additional factors.

In the editor, rendering options for each of the pass are set independently on per-material basis (Materials -> States -> Passes).

Regarding lighting opacity geometry, the rendering pipeline can combine the forward rendering (light passes) and deferred lighting for selected light sources (deferred light passes).

The very first to come is the deferred pass which is written into four RGBA 8 buffers. The data from deferred buffers is used by shaders in next rendering passes.

• Depth buffer. Scene objects in the current field of view (found between the near and far clipping planes) are rendered as pure geometric models and stored into this buffer:
  • RGB channels for depth values. They allow to sort objects relative to the camera in the right order and cull invisible surfaces.
  • Alpha channel to flag the geometry to be rendered with volumetric shadow shafts.

  Initially depth buffer is filled with opaque geometry. Transparent objects are not written into the depth buffer by default, though invisible ones are still successfully culled by the video card. However, sometimes to avoid visual artifacts (like water or particles systems) transparent objects need to be written into the buffer later. For such transparent objects the Post deferred flag in Materials settings is enabled. See below for details.

  

  Deferred depth buffer
• Diffuse colors buffer stores pure diffuse colors of all material textures:
  • RGB channels for diffuse colors
  • Alpha channel for glow (if Ambient emission option is enabled for the material)

  

  Deferred diffuse color buffer
• Normals buffer stores normal vectors for each vertex of geometry which will be necessary to calculate the proper lighting:
  • RGB three components of the normal vectors.
  • Alpha channel for specular power that would define intensity of specular highlights.

  

  Deferred normals buffer
• Parallax mapping buffer stores parallax displacement values for materials with Parallax mapping option giving them the illusion of having depth. This means the flat surface will appear to have a bulgy relief. If there are no materials with parallax mapping, the buffer is not used.
  • RG the X component.
  • BA the Y component.

  

  Deferred parallax buffer

Unlike the forward light passes, deferred light pre-calculation passes allow simplified, moderate-quality but faster lighting of opaque objects. Deferred lights can be enabled with the distance on per light source basis and thus be used as light LODs.

Deferred lights are pre-calculated and stored in the RGBA 8 buffer (or RGBA 16F if HDR is used). Basing on the information from Depth and Normals buffers, each light source is rendered in a separate pass in the following way:

• The light's bounding sphere is rendered as geometry.
• Basing on the sampled deferred data, pixel shader computes colour contribution of the light to each pixel inside its radius in screen space. Diffuse component is stored in RGB channels and specular component is monochrome, because for it only the alpha channel is left.
• All lights are additively blended into the buffer in their own passes.
• Deferred lights are now calculated, but they are not rendered at once. They will rendered on the screen only with the next-following ambient light pass. Depending on how further rendering will be performed, deferred lighting can be of two types:
  • Deferred lighting itself. It does not change the ambient pass (see details below).
  • Deferred shading for all lights. Not only the lights passes are simplified, but in addition the ambient lighting pass is also simplified. The pay-off for speed is quality. See details below.
  Notice

  To use deferred lighting, set console command render_deferred to 1 (default). To use deferred shading for all lights, set it to 2. (To take effect, these commands should be followed by render_restart).

As you can see, deferred lights do not redraw the geometry of the lit objects (while normal lights do), hence the performance increases. However, flexibility and accuracy of lighting suffers:

1. All the lit objects are rendered using uniform Phong shading with default settings.
2. Specular component, being monochrome, is not exactly precise.
3. Anti-aliasing is also unavailable.

Ambient light pass is the first one to store its color data in the screen buffer: RGBA 16F (16-bit floating point precision per each channel) required if HDR used, or otherwise RGB10 A2.

Taking into account deferred buffers data, only opaque objects are rendered (though they still may use alpha testing). During this pass geometry in the view frustum is rendered lit by the nondirectional ambient lighting, which is calculated in the following way:

ambient color * material diffuse color     Plus:

• If any, prebaked Ambient texture for the material is also rendered in this pass. It simulates self-shadowing in corners and cracks of objects.
• If environment cubemap is specified, it also contributes to this pass: according to the colors of cube sides, all objects in the scene are lit and shaded. The global environment texture is modulated (multiplied) by the material's ambient texture.
• Light maps are rendered during this pass, if they were defined in material settings.

Deferred Lighting and Deferred Shading

Next goes a series of passes for each of the light sources that re-render opaque objects being lit:

• World lights
• Omni lights
• Spot lights
• Projected lights
• Prob lights

But before actually rendering the lights, shadows should be calculated. Shadowing technique directly depends on the light source type (for details see Shadows section):

• Parallel-split shadow mapping is used for world lights. Depending on the number of set splits (from 1 to 4), PSSM requires the same number of passes to redraw scene geometry. To avoid the worst-case scenario when all the geometry is rendered four times, overlapping areas are optimized to be rendered only in one split.
• Shadow mapping is used for all the other light sources.

Shading

Decals are written to Deferred Depth buffer along with other objects, so as to store glow or parallax mapping information. They are rendered projected onto the surfaces in separate passes after opaque geometry is lit and rendered into screen buffer.

1. First comes the ambient pass, in which the decals are rendered with ambient lighting, just like in the ambient pass for the opaque objects.
2. Lighting from light sources in the scene is calculated during the next several light and shadows passes (one pass per light source). Again, those passes are similar to the light passes for the opaque objects.

Impostors are also rendered separately from the other objects, and have their own update method. They are also written into the Deferred Depth buffer to be correctly affected by light scattering.

1. Each frame, before the very start of the rendering sequence, impostors are baked into three textures (RGB10 A2), depending on the distance to the camera and hence, their size. The particular number of slots for impostors in these textures depends on the render_impostor console variable. The largest impostors most close the camera are rendered into the texture with the least slots and are the first to be updated.
2. During the rendering sequence, baked impostors are rendered in the separate ambient pass.

After the opacity geometry is rendered lit, the image still seems somewhat flat without self-shadowing. Screen-space Directional Occlusion allow to approximate the effect of global illumination in real time. It combines the following:

• Ambient occlusion computes how much ambient light, which is equally incident from all directions, has actually reached the surface and how much of it was occluded. As a result, crevices are realistically darkened, while salient parts remain exposed to light.
• Indirect illumination specifies how the indirect light bounces off objects and brings its color to the neighboring ones. Indirect illumination allows to have red-colored shadows from the red objects, while the ordinary occlusion renders only the colorless grey shadows..

SSDO is computed for all the objects which are stored in the Deferred Depth buffer. It is implemented in the following stages:

1. Auxiliary pass (RGBA 32F buffer is allocated). Depth and Normals texture are sampled and stored in a new buffer downscaled by half. This pass allows to reduce cache misses.
2. Directional occlusion pass (another RGBA 16F buffer is allocated). It calculates the amount of incident and indirect bounce light at each point. In the screen space a certain radius around each point is sampled. It means, that in this radius a number of random sample points are tested (the exact number depends on the quality of shaders). All these sampled emitter points can occlude or bounce off the light on the receiver point (and vice versa, the receiver point can receive it) no further than the set distance in world space.
   Directivity of normals is also taken into account. Emitter points can shade and color the receiver point along their normals or in all directions. In the same way, receiver can receive shade and bounced-off light that came only along its normal or from anywhere.
3. The resulting image is written to the screen buffer using alpha blending technique: the screen image will be multiplied by occlusion texture. In spite of differences in resolutions, there are no stretch-induced artifacts along the edges due to the depth threshold. It allows to render sharp and clear edges, because in areas of depth change only such pixel is sampled from the occlusion texture which depth value is most close to the pixel of the ambient light texture.

When shading of the opaque objects is over, transparent objects can be rendered. They are rendered separately as their material should be blended with underneath layers depending on their alpha blending settings.

The problem arises when simultaneously rendering transparent objects and light scattering that attenuates objects with the distance. Deferred Depth buffer can store only one layer of depth data. This means two scenarios are possible:

• Transparent objects overwrite depth value in the depth buffer (by enabling Post deferred flag in Materials settings). In this case, after being correctly blended into the current scene, using the existing depth values, they overwrite it to compute light scattering. As the result, transparent geometry is correctly attenuated, but all the objects behind it are not affected by light scattering at all.
• In the other case, transparent objects are rendered before light scattering pass and are not written into depth buffer. As they use the depth value of underneath geometry, they can turn out greatly attenuated (of a very blue color), though being just three feet away.

To avoid these visual artifacts, rendering sequence for transparent objects is flexibly divided into the following stages:

1. Objects in pre-scattering passes
2. Light scattering
3. Objects in post-scattering passes

Ambient Pass

Lights & Shadows Passes

Screen-space light scattering is calculated in this pass. Using information from the Depth buffer, it approximates the physical model (Rayleigh and Mie scattering by atmospheric particles) with a set of parameters. Light scattering determines the color of sky and sun, simulates atmospheric effects, attenuates objects with the distance, and on the whole makes outdoor scenes look even more realistic. Light scattering is rendered as a full-screen quad.

Transparent objects drawn after the scattering pass are not distance-attenuated. These can be windows, objects in enclosed spaces etc - everything that will be seen from close range. If choosing wisely, disregarding light scattering trades off with no visible artifacts, that originate from deferred depth buffer limitation of storing only one depth value. Post-scattering objects are rendered in absolutely the same way as pre-scattering ones.

Ambient Pass

Lights & Shadows Passes

Refraction can be applied only to transparent materials (with Refraction pass set to default). It is rendered in the following way:

1. The refractive surfaces are rendered in the separate buffer (RGBA 8), using the Deferred Depth buffer.
2. The color values are displaced (according to the default refraction texture):
   • Red represents displacement based on the surface normal along the X axis.
   • Green represents displacement based on the surface normal along the Y axis.
   • The displacement is scaled by the refraction multiplier, that controls the amount of distortion.

Transparent materials are affected by the Refraction pass even if they are closer to the camera than the refraction surface. If a transparent material should not be affected, it can be rendered after the refraction pass (by setting Post refraction flag). Other than that, rendering of such objects requires the same passes:

• Ambient pass for ambient lighting.
• Lights and Shadows passes for lighting with light sources.

Volumetric shadows that are spread in the air forming crepuscular rays require two passes:

1. Sample geometry pass. A new buffer is allocated that stores geometry casting volumetric shadows (Shadow shafts box should be checked for material). Compared to deferred buffer, it is downsized to half and holds only depth data in RGBA 8 format.
2. Calculating shadows. In the second buffer (which is similar to the previously allocated one) shadows are drawn depending on the direction of world light. No other lights are considered in calculations. The intensity of shadowing and length of the shadowed regions are defined in Render settings.
3. After these passes are over, volumetric shadows are rendered into the screen buffer using alpha blending addition.

Post-processing completes the rendering sequence. During this stage various effects are rendered and the final image is put together with the screen texture in the composite shader.

HDR

HDR is rendered in the following passes (all using RGBA 16F buffers).

• During the first pass, screen buffer is downsized to half to be faster processed during the next passes.
• In order to correctly transform the HDR image in the screen buffer to an LDR image and preserve details both in bright and dark areas in the scene, average value of luminance is calculated using the chosen algorithm (logarithmic or quadratic one). This process is called tone mapping. The average of four neighboring pixels is estimated and form a new downsized texture. It takes five successive passes to get one pixel texture indicating the average luminance in the scene that will be a reference for tone mapping. This value, multiplied by custom set exposure and clipped between minimum and maximum luminance, allows to map scene radiance values to a displayable output range.
  It should be noted though, that scene luminance is calculated with three frames delay. That, plus the set adaptation time determines the time the camera adapts to the lighting conditions change.
• To create HDR illumination effect, in the next pass the bright areas that exceed the brightness threshold are extracted from the screen buffer. This pass is also downsampling one.
• Then the extracted areas are blurred using smart two-pass filter. It uses vertical and horizontal sampling, which increases sampling rate.
• The resulting HDR image is passed to the composite shader to be additively blended with screen buffer image.

Notice

Due to usage of a logarithmic function, the HDR image should not contain areas filled with a pure black color, meaning ambient color should not equal 0.
Usage of HDR and anti-aliasing at the same time is supported only by DirectX10-compatible graphics cards and higher, as well as cards based on ATI R500 (only in Direct3D9).

Here is how tone mapping for HDR looks like:

Source downsized image used to compute average value of luminance

An image with HDR corrected according to the average value of luminance

To create special HDR effects, additional filters can be enabled. All of them operate on the buffer with already extracted bright areas.

• Cross flares are rendered in several passes: fading out copies of glaring objects are added with small offsets in the direction of the flare. The process continues until the defined length is reached, and is repeated for each flare shaft.
• Lens flares require only one pass, during which several copies of glaring objects are added to create a flare coming from the center of the screen.
• Light shafts also require one pass: in the direction of world light copies of the glaring objects are drawn until the set length is reached.

A buffer with extracted bright areas and HDR effects

An image with HDR effects

Motion Blur

Depth of field

Glow

Color correction

Auxiliary pass allows you to write the contour of object with the specified material into an auxiliary color buffer (if enabled). After that, you can apply a postprocess material to it. Postprocess materials are part of the Unigine standard material library helping to render some special effect for a surface (for example, blurring, subsurface scattering, etc.)

You can also write your own shaders and apply your custom postprocess using this buffer.

Auxiliary color can also be used in an overlay color mode (used for flat-coloring of meshes in the main viewport). Depending on the alpha channel of the auxiliary color, objects are rendered fully or semi-colored: