Experimental SDK build: new Window Manager (with sample contents)

[fox]

We have released an experimental SDK build to demonstrate work in progress on the new Window Manager (as a first step for the new in-app GUI system) along with Input system and Engine integration improvements (see the description of changes below). 

All samples demonstrating the changes are implemented as C++/C# components and included in the default project template for your convenience.They will give you a quick overview and demonstrate new features and capabilities.

We appreciate any feedback regarding the functionality, feature-completeness, and ease of use - together we can make the future production version better!

Major improvements for all widgets are on the way to bring you more versatility and convenience, so stay tuned for the upcoming releases.

Installing and Working with Samples

To try out the new experimental build:

1. Open the Unigine Developers Portal (https://developer.unigine.com/)
2. Go to the Downloads section
3. Choose Experimental (Not Stable) in the left menu
4. Select the right build for your SDK edition (2.15_210913_window-manager) and download it.
   
5. Once the SDK ZIP archive is downloaded, unpack its contents to any location on your computer.
6. Open the SDK Browser. Go to the Products tab, click Add Installed, then browse to the location of the unpacked SDK and click Select Folder. The SDK will be added.
7. Make this SDK default by clicking the Make Default button.

 

Now you're ready to start exploring the new functionality. Depending on the desired programming language (С#/C++), create a new project from the default template.

1. Press Create New Project

   C#	C++

2. Press Run in SDK Browser

3. Check out the new samples

 

Note: Samples were made using the component system, which means you can modify and use them independently.

C# Components

 

C++ Components

Main Changes

System Proxy for Engine Integration

The CustomSystemProxy class is a new way of Engine integration. The old approach, using CustomApp, had several restrictions when it came to working with windows, thus disrupting the workflow. The new class covers the following functionalities:

1. Defining the set of functions, to be available via proxy. These are functions for graphics, functions for creating the main and external windows, functions for mouse, keyboard, touch sensor interactions, functions for working with multiple screens.

2. Initiating event callbacks for window, keyboard, mouse, text input, etc.

3. Overriding of functions working with graphics context (currently DirectX 11 and OpenGL 4.5). This set includes initialization and removing context, functions for creating buffers for windows (initialization, resizing, and removing), functions for rendering.

4. Overriding of functions responsible for:

   • creating the main window and setting up its parameters.
   • creating additional windows.
   • mouse interactions.
   • keyboard interactions.
   • touch screen interactions.
   • working with multiple screens. Functions include getting several screens, order of screens, specifications, changing settings, etc.

5. Overriding of additional functions, like working with dialog windows, clipboard, etc.

The functionality of some Engine subsystems is to be defined depending on the set of functions made available by the user. This class forms the basis for the operation of WindowManager, Input, GUI, Displays, and App classes. WPF and Qt integration samples are to be added in the upcoming releases.

 

Input System

To unify processing of the mouse and keyboard inputs we have extended the Input class. New functionality covers methods, that were used for input in the App class. Instead of App::getKeyState(), App::setKeyState(), and App::clearKeyState(), user should use Input::getRawKey(), Input::getAndClearRawKey(), and Input::forceSetRawKey() respectively. Similar functions are used for mouse input with an ability to control masks for every pressed mouse key getMouseButtonMask() / setMouseButtonMask(). All keyboard keys, mouse buttons and axes have been moved to this class.

 

Extended Set of Callbacks

From now on, access to the following input event callbacks is available:

1. MOUSE_DOWN
2. MOUSE_UP
3. MOUSE_AXES
4. KEY_DOWN
5. KEY_PRESS
6. KEY_UP
7. CODE_DOWN
8. CODE_PRESS
9. CODE_UP
10. TEXT_PRESS

 

Text input callback allows getting a symbol in Unicode format. The user is getting an unsigned char, then it is translated into a proper code (for example: If Shift+q is pressed, then we get “Q”, and not just a key symbol).

New Keys and Physical Codes 

Apart from keys dependent on the keyboard layout (QWERTY, QWERTZ, AZERTY), physical codes for keys have also been added. You can work with these codes via isCodeDown(), isCodePressed(), and isCodeUp(). This ensures standardization of controls to the same layout for every keyboard type, with the same button opening the console everywhere. Those codes have standard QWERTY layout names.

Some new keys have also been added:

1. ESZETT
2. KEY_U_UMLAUT
3. KEY_O_UMLAUT
4. KEY_A_UMLAUT
5. KEY_ACUTE_ACCENT
6. KEY_U_WITH_GRAVE
7. KEY_SUPERSCRIPT_TWO

 

Moreover, paired keys (Ctrl, Alt, Shift, etc) are now divided (LEFT_SHIFT, RIGHT_SHIFT, LEFT_ALT, etc) making it easy to determine any pressed key on QWERTY, QWERTZ, and AZERTY keyboards. Numpad keys can also be obtained depending on the NumLock state.

 

The following keys have been added to the Input class, but are yet to be functional:

1. KEY_ANY_SHIFT
2. KEY_ANY_CTRL
3. KEY_ANY_ALT
4. KEY_ANY_CMD

You can find examples for working with new callbacks in the Input sample.

 

Working with Displays

The New Display class allows getting information about connected displays. You can get the number of screen displays and the index of the main one. With those indexes, you can identify their position and size in pixels. Also, there is access to the name and current dpi. Apart from that, there are functions for obtaining available modes for displays. You can get the number of modes and by their index get resolution and FPS.

For now, this class allows getting all that information, although later there will be an ability to apply the modes during fullscreen rendering.

You can find examples of obtaining that information in the Display sample.

 

Window Manager

For the creation of external windows, we have added a new GuiWindow class. All operations for creating, removing, and merging windows are performed via this manager. With its help, you can access the main window of the app, create and remove additional windows, and stack them. Stacking with the main window is currently unavailable.

 

The following callbacks are available:

1. WINDOW_CREATED
2. WINDOW_REMOVED
3. WINDOWS_STACKED
4. WINDOW_UNSTACKED

 

Additional external windows can be combined into the following three groups:

1. Horizontal
2. Vertical
3. Tabs

 

When you merge two windows, a third window is created, it includes the first two. The new window is called a group. You can add an unrestricted number of windows to the group without creating additional ones. In the case of merging two groups, a new group is created, which includes the contents of the two groups. A single placement mode (vertical, horizontal, or tabbed) is used for all windows in a group. The hierarchy of a sample looks the following way:

 

There are two main functions for grouping:

1. GuiWindow stack(first, second, index, stacking_type, decompose_second)
2. GuiWindow stackGroups(first, second, stacking_type, flag decompose_second)

first and second - windows or groups for merging

index - defines a place, where a window or a group should be placed in a group.

stacking type - defines a type of a group under creation if the first argument is a window and not a group.

decompose_second - a flag, if the second argument of the merge is a group, then it should be broken into its contents (windows), and afterward, combined with the first window or a group (this flag is not used currently).

 

The first function always creates a new group if its first argument is a window. Also, it simply adds a second element, if the first argument is a group. If we need to combine two groups into a new one, then we should use stackGroups(). Both functions return a newly created group or the first argument (when it is a group).

 

There are the following wrappers, for the stack function for your convenience:

1. GuiWindow stackToParentGroup(destination, window, index, decompose_second)
2. GuiWindow stackWindows(first, second, stacking_type)
3. GuiWindow stackToWindow(destination_window, window, stacking_type,  decompose_second)
4. GuiWindow stackToGroup(destination_group, group, index, decompose_second)

 

These functions perform additional checkups of the arguments according to function names, then the stack is called. It allows to reduce the number of arguments during a call and to logically divide all merger types.

 

The unstack() function is used to remove a window or a group from a parent group. If there is only one window left, the group is automatically deleted after removing the window from it.

 

When stacking with a mouse, the window is logically divided into 9 zones (the process is not yet visualized). Releasing the mouse during the hover over one of the zones will result in stacking the window.

 

 

To add a window or a group into an existing group with a mouse, release it in the following zones:

Insert position is determined by mouse position in accordance with those areas.

 

The following callbacks are available for windows:

1. RESIZED
2. SHOWN
3. HIDDEN
4. FOCUS_GAINED
5. FOCUS_LOST
6. ENTER
7. LEAVE
8. MOVING
9. MOVED
10. CHANGING
11. SIZE_CHANGED
12. MINIMIZED
13. MAXIMIZED
14. RESTORED
15. CLOSE
16. UNSTACK_MOVE

 

All callbacks, as well as some functions, are currently functional for external windows only. However, the events are called for all windows. This will be fixed after migration to new widgets (in the upcoming releases) that are designed to work properly with new behavior.

 

The Windows samples set demonstrates how to work with windows.

The majority of windows-related functions are available in the Sandbox. It also shows the current hierarchy of created windows along with their types.

 

Check out the following video with a sample of a Simple Editor:

 

[fox]

Known Issues

• All Engine plugins are not currently available.
• Some of the Engine samples may behave unexpectedly.
• Only a minimal integration of the new proxy with UnigineEditor is implemented currently.
• Cursor capture does not work properly when exiting from a maximized window on Windows 10.
• Support for joysticks is not available.
• Hotkeys are not recognized by UnigineEditor.
• The Engine Viewport does not work in UnigineEditor for C++ projects.
• A preview (on Alt+Tab) is not rendered properly on Windows 10/11 in case some parts of windows are beyond the screen edges.

[Amerio.Stephane]

Hi, these are good news!

1/ Am I correct to assume this will fix the SDL limitation reported here? 

 

A quick test shows that now the Component::update() and WorldLogic::update() are called even when moving the app window around :) Still need to check the  IG+syncker works too.

 

2/ About the keyboard new callbacks, the CODE_* callbacks are a great addition! Just a few small remarks: The Enter key is usually named ENTER, not RETURN though maybe you could have the two names as synonyme in the api; You use the ANSI layout as a base for the hardware code layout (104 keys), but Europe uses the ISO layout (105 keys) and miss the "<" key beside Left-Shift (it is reported as an Unknown key).

In the sample, the Unicode Text does not work correctly when entering a complexe character such as é or à on a French keyboard. This is probably just an issue in the sample code because you cast a unicode code into a 8 bits 'char'.

 

3/ What is the intended difference between code_DOWN and code_PRESSED callbacks? In my tests, they behave the same (triggered as soon as the key is fully pressed).

 

4/ About display, it would be great to have the capability on the command line to select which display to start on, and the window coordinates (I know we can use 3rd party tools for that, but it would be easier on new users and for quick test)

 

Thanks!

 

[silent]

Hi Stephane,

Thanks for the feedback!

Quote

1/ Am I correct to assume this will fix the SDL limitation reported here? 

The issue is not on our side, but rather the SDL2 implementation. We didn't change it, so I guess there should be any difference in behavior. Patches that were supposed to fix this are not working at all :(

 

Quote

2/ About the keyboard new callbacks, the CODE_* callbacks are a great addition! Just a few small remarks: The Enter key is usually named ENTER, not RETURN though maybe you could have the two names as synonyme in the api; You use the ANSI layout as a base for the hardware code layout (104 keys), but Europe uses the ISO layout (105 keys) and miss the "<" key beside Left-Shift (it is reported as an Unknown key).

Yeah, we will check and add proper keycodes for that use-cases.

 

Quote

3/ What is the intended difference between code_DOWN and code_PRESSED callbacks? In my tests, they behave the same (triggered as soon as the key is fully pressed).

Will return back to you as soon as I will get a proper answer from the dev team.

 

Quote

4/ About display, it would be great to have the capability on the command line to select which display to start on, and the window coordinates (I know we can use 3rd party tools for that, but it would be easier on new users and for quick test)

I think that's should be also easy to implement with a new API.

Thanks!

[silent]

Quote

3/ What is the intended difference between code_DOWN and code_PRESSED callbacks? In my tests, they behave the same (triggered as soon as the key is fully pressed).

_DOWN callback is called on the very first key down
_PRESSED callback is called while key is being pressed (down)
_UP callback is called in very moment of key release

Hope that makes sense :)