Projections with AppProjection Plugin

WarningThe scope of applications for UnigineScript is limited to implementing materials-related logic (material expressions, scriptable materials, brush materials). Do not use UnigineScript as a language for application logic, please consider C#/C++ instead, as these APIs are the preferred ones. Availability of new Engine features in UnigineScript (beyond its scope of applications) is not guaranteed, as the current level of support assumes only fixing critical issues.

Warning

The functionality described in this article is not available in the Community SDK edition.
You should upgrade to Sim SDK edition to use it.

AppProjection allows you to create multi-projector setups. Edge blending, non-linear image mapping, and color correction for each projection are supported. Therefore, AppProjection can project images onto large curved screen displays covering 360 degrees, if necessary. Moreover, each projection can be fully customized to use a required view frustum.

Notice

This plugin cannot be used in a Qt-based application.

Projection setup

AppProjection

Customizable projection with AppProjection

Launching AppProjection#

To launch the application and render projections:

Specify an engine plugin library (lib/AppProjection_*) together with the rest of required start-up arguments (such as rendering API, window size, etc.). For example:
Shell commands
```
main_x64.exe -extern_plugin AppProjection
```
You can use the 64-bit debug or release version of the library. (The engine automatically loads the appropriate version of the library depending on the specified main application.)
Notice
It is impossible to use AppProjection with:
- Multi-monitor apps (AppWall, AppSurround)
- Panoramic rendering mode
- Stereo 3D mode
To change the number of displays in a row, specify -width CLI option. For example:
Shell commands
```
main_x64.exe -extern_plugin AppProjection -width 2
```
By default, up to 8 projections are supported to be output from one rendering node.

AppProjection with two projections

AppProjection with two projections

Setting Up Projections#

To set up projections, open the system menu by pressing Esc, go to the Plugins tab and click the Projection button.

The Projection Configurator window will open. This window makes it possible to:

Rotate and flip projections.
Control warping for each projection.
Set up edge blending and masks for each projection.
Correct color intensity for each projection.

You should first choose a Projection to be adjusted at the top-right corner.

Warping#

The Warp tab allows you to set up geometry correction for your projections. Areas where geometry correction is required include:

Advanced off-axis correction where projector placement is awkward and needs an advanced mapping over the keystone function in a projector
Non-planar screens such a curved screens and hemispherical domes
Projecting one image from one projector onto more than one surface
Unusual projection applications onto custom designed screens

Interface

Warping is performed by modifying the displayed grid. The following options are available:

Projection	Select a projection in the drop-down list to set up its parameters. You can also use Page Up and Page Down buttons to switch between projections.
Multiplier	The displacement step in pixels to be used when moving points with arrow keys. Use + to increase the value, or - to decrease it.
Resolution	The number of control points along the grid horizontal and vertical axes. A 2 x 2 grid is used by default. Notice The maximum grid size is 32 x 32.
Points	Visual editor enables you to set up warping for the selected projection using control points and yellow handles. These handles are displayed for each selected control point depending on its type. You can select control points by clicking the mouse, and also switch between them using arrow keys. Multi-selection using the mouse is performed as a common selection of multiple objects; to add or remove points from the selected group, click on them holding the Ctrl key. Multi-selection using the keyboard is possible in a chain-like way using arrow keys with the Shift key pressed. The Ctrl + Z / Ctrl + Y combinations are used for the Undo / Redo actions. Shift + Ctrl + Z cancels all transformations. Notice Points are always moved in the screen-space coordinates regardless of image orientation (whether it is flipped or not), this means if you press left arrow, the point always goes left. Drag control points and their yellow handles to warp the projection and adjust it for the required configuration. You can also move points using arrow keys with the Ctrl key pressed. You can see an example of a warped grid and a corresponding projection image below. Warped grid Warped image You can fine-tune projection warping by adjusting the parameters of each control point by selecting it and changing parameters via the following panel next to grid resolution settings: The following parameters are available: Point — horizontal and vertical indices of the selected control point. Enumeration starts from the bottom left corner. Position — coordinates of the selected control point along the X and Y axes. Coordinates for control points Type — warping type for the selected control point: — automatic linear interpolation (no control handles available) — symmetric control handles for smooth curves — asymmetric control handles (you can control each handle independently) — automatic smooth curve (no control handles available) The grid itself can also be moved inside the view space: Middle Mouse Button for panning (dragging the grid vertically or horizontally) Mouse Scroll for zooming in and out button to undo all panning and zooming actions and make the grid fit the window
Canvas	Configure the canvas for the selected projection: set rotation angle and enable vertical and horizontal flipping.

Color and Brightness Correction#

The Color tab allows you to correct color intensity, as well as to adjust color balance for the selected projection.

Brightness	Per-channel color multiplier to control the intensity of: Red Green Blue RGB at once Brightness for the red channel = 2
Contrast	Settings that allow you to adjust the color balance per channel (as well as for all channels at once): Positive values are added to the color values on the screen. Negative values are subtracted from color values on the screen. Contrast for the red channel = -0.1
Brightness (corners)	Settings to adjust brightness compensation for each of the projection's corners individually.

Brightness

Per-channel color multiplier to control the intensity of:

Red
Green
Blue
RGB at once

Red channel scale

Brightness for the red channel = 2

Contrast

Settings that allow you to adjust the color balance per channel (as well as for all channels at once):

Positive values are added to the color values on the screen.
Negative values are subtracted from color values on the screen.

Red channel bias

Contrast for the red channel = -0.1

Brightness (corners)

Settings to adjust brightness compensation for each of the projection's corners individually.

NoticeAll color parameters can be reset to defaults by clicking the corresponding parameter name (by line or by column for the Color Balance set of parameters, and individually for each corner).

Blending#

The Blend tab allows you to configure blending in the selected area and along the edges.

Screen-Space Blending	Manages blending in the selected area. The screen-space blend area is not affected by warping. To select the area, click Add to start creating the blending area, and in the grid, click to define one side of the area, then click to define the other side — the green line will appear. The third click shows the blending direction. The area of the screen-space blend area is divided into two parts: the part between the blue and green lines is a solid-black area, and the part between the green and red lines is a gradient area. By default, each of these lines has two points marking the line limits. Using Resolution, you can increase the number of points along each line to adjust it as required by moving the points. Multi-selection of points is available. Every blend area is adjusted using the following settings: Enabled — if toggled on, enables the use of the blend area selected from the list. Link to Projection — if toggled on, allows adjusting Alpha, Contrast, and Gamma for two blends simultaneously for better neighboring of two projections. Select the projection and the blend area from the drop-down lists and adjust the necessary parameters. Name — the window to modify the blend area name. Alpha — transparency of the whole blended area. Contrast — contrast of the gradient part of the blended area. Gamma — gamma correction value.

Screen-Space Blending

Manages blending in the selected area. The screen-space blend area is not affected by warping.

To select the area, click Add to start creating the blending area, and in the grid, click to define one side of the area, then click to define the other side — the green line will appear. The third click shows the blending direction.

The area of the screen-space blend area is divided into two parts: the part between the blue and green lines is a solid-black area, and the part between the green and red lines is a gradient area.

By default, each of these lines has two points marking the line limits. Using Resolution, you can increase the number of points along each line to adjust it as required by moving the points. Multi-selection of points is available.

Every blend area is adjusted using the following settings:

Enabled — if toggled on, enables the use of the blend area selected from the list.
Link to Projection — if toggled on, allows adjusting Alpha, Contrast, and Gamma for two blends simultaneously for better neighboring of two projections. Select the projection and the blend area from the drop-down lists and adjust the necessary parameters.
Name — the window to modify the blend area name.
Alpha — transparency of the whole blended area.
Contrast — contrast of the gradient part of the blended area.
Gamma — gamma correction value.

Defining a screen-space blending area

The blended area

Warped Edge Blending

Controls soft edge blending for each side of the projection (left, right, top, and bottom). If you warp the image, the border blending becomes warped, too.

Size controls the size of the blended area:
- 0 — there is no blending.
- 0.5 — a half of the screen is blended.
Alpha controls transparency of the blended area:
- 0 — the blended area is not transparent.
- 1 — the blended area is completely transparent.
Contrast defines contrast of the blended area and makes transition from the darkest to the lightest area non-linear.
Gamma — gamma correction value.

Gamma = 0.5

Gamma = 5

NoticeBorder values can be reset by clicking the corresponding column or line name.

Masking#

The Mask tab allows using masks to cut out certain areas of the selected projection.

To create a mask, click Add and ensure that the Creation Mode button is enabled. Then create at least three points on the grid by clicking in the required place. The corresponding area of the projection will be cut out. The list of masks for the current projection is displayed under the grid. You can add new masks and remove existing ones using the corresponding buttons.

To add new points to the mask selected in the list, ensure that the Creation Mode button is enabled. Then you can add new points by clicking within the grid area. To insert a new point into an existing mask, select the point next to which a new point is to be added, and click to insert a new one. You can drag an existing point to move it. To remove a point, right-click it.

You can also adjust your masks precisely by specifying the coordinates of each point. Select a point on the grid (its color will turn to green) and enter the coordinates in the Position fields below.

You can change the type of a selected control point using the following icons:

— automatic linear interpolation (no control handles available)
— symmetric control handles for smooth curves
— asymmetric control handles (you can control each handle independently)
— automatic smooth curve (no control handles available)

As you select a mask in the list, its parameters are displayed on the panel next to it:

Enabled — enables or disables display of the selected mask. Can be used for convenience when configuring overlapping regions, when there are a lot of masks.
Name — mask name, can be edited.
Smooth — roundness of the mask edges.

Render#

The Render block of settings is located at the bottom of the Projection Configurator window. These settings are designed to facilitate fine-tuning of the projection by displaying the auxiliary elements on the actual projection.

The Debug Color option allows you to temporarily set individual colors for different projections. This helps to visualize overlapping regions.
Show Grid — display the grid on the projection.
Warp Points — display points and auxiliary red lines going from a selected point or points to the center of each side.
Blend Lines — display screen-space blend lines and auxiliary purple lines going from a selected point or points to the center of each side.

Multiple Projections' Viewing Settings#

If more than two projections are created with AppProjection, the following viewing settings can be tweaked in the System menu -> Plugins.

FOV	The field of view for all projections (in this case, the Camera's Field of View option is not applicable). FOV = 60 FOV = 80
Angle	Allows adjusting the viewing angle between projections: when projection surfaces are turned at some angle to each other, this angle can be entered as the Angle value and the projection is automatically recalculated to show the right image. For example, if two surfaces are turned to each other at 40 degrees: Angle = 40 degrees
Bezel X	Compensates for the horizontal bezel of monitors. Positive values decrease the viewport space; negative ones increase it (for overlapping).
Bezel Y	Compensates for the vertical bezel of monitors. Positive values decrease the viewport space; negative ones increase it (for overlapping).

Saving and Loading Presets#

After the projection configuration is set up, it can be saved to a preset file (*.proj) to be loaded at any time (e.g. to set up another projection):

Save button to save a preset file
Load button to load a preset file

Notice
			A *.proj file that you save contains settings only for the currently selected projection.
Settings loaded from a *.proj file are applied to the currently selected projection.

		

Customizing AppProjection#

AppProjection can be customized to support custom viewing frustums (symmetric or asymmetric ones) for each projection.

AppProjection Viewports#

AppProjection has one primary viewport, while all other viewports are rendered as auxiliary ones. By default, the primary display is the UNIGINE engine viewport used for one-monitor configuration. It uses matrices of the Player used by the engine to view the scene. Other displays are arbitrary cameras with any perspective and facing whatever direction needed. Each display has its own model-view and projection matrices. Both the primary projection and auxiliary ones can be enabled or disabled, if necessary.

The primary viewport can be set to any projection (for supported configurations it is already set).
Each viewport has its own model-view and projection matrices.
By default, only a primary one has a graphical user interface (render system GUI, editor widgets, wireframe, or the profiler). However, separate GUIs can be drawn on all projections.
All viewports have their own Viewport and Reflection mask to selectively render nodes and reflections from them.

Default Configurations#

For default configurations, the primary viewport is set to the following projection:

For 2 projections, the 1st viewport.
For 3 projections, the 2nd viewport.
For 4 projections, the 2nd viewport.
For 5 projections, the 3rd viewport.

How to Customize Projections#

Rendering of AppProjection viewports is controlled by the wall.h script (found in the <UnigineSDK>/data/core/scripts/system/ folder). There are two possible setups depending on how the primary projection is rendered. It can be drawn by:

The default engine renderer (the same as when a usual one-monitor application is rendered)
The AppProjection renderer itself (which is safer if you are going to use asymmetric frustum for the center projection and modify its model-view matrix)

Notice

Only one of two renderers should be enabled at the same time.

The following example demonstrates how to tweak cameras configuration and choose the renderer for the main projection.

1. Using default engine renderer#

The first variant is to render the main (primary) projection by the default engine renderer.

In case the created configuration is not already supported, set the primary projection via engine.projection.setPrimary():
Source code (UnigineScript)
```
// Set the first projection as a primary one
engine.projection.setPrimary(1);
```
Enable all non-primary projections via engine.projection.setEnabled(). The main one should be left disabled, as it is drawn by the default engine renderer.
Source code (UnigineScript)
```
// Enable the 2nd projection in a row
// The second argument of the function sets 'enabled' flag
engine.projection.setEnabled(1,1);
```

Set projection and model-view matrices for an auxiliary projection.

Source code (UnigineScript)

// Settings for the 2nd projection
Camera camera1 = engine.projection.getCamera(1, 0);
camera1.setProjection(projection_1);
camera1.setModelview(modelview_1);

2. Using AppProjection renderer#

Another variant is to render the main projection by the AppProjection renderer. This variant can be used, for example, if you want to set up symmetric frustums for all projections.

Set the projection to be used as a primary one and enable its rendering by AppProjection.
Source code (UnigineScript)
```
// Set the first projection as a primary one
engine.projection.setPrimary(1);
```
Disable rendering into the default UNIGINE viewport via engine.render.setEnabled():
Source code (UnigineScript)
```
engine.render.setEnabled(0);
```
Enable all AppProjection projections including the primary one. As a result, all viewports will be rendered by AppProjection renderer itself:
Source code (UnigineScript)
```
// Enable all projections
engine.projection.setEnabled(0,1);
engine.projection.setEnabled(1,1);
```

Set model-view and projection matrices for all projections.