UnigineEditor
Interface Overview
Assets Workflow
Settings and Preferences
Working With Projects
Adjusting Node Parameters
Setting Up Materials
Setting Up Properties
Lighting
Landscape Tool
Using Editor Tools for Specific Tasks
Extending Editor Functionality
Programming
Fundamentals
Setting Up Development Environment
Usage Examples
UnigineScript
C++
C#
UUSL (Unified UNIGINE Shader Language)
File Formats
Rebuilding the Engine Tools
GUI
Double Precision Coordinates
API
Containers
Common Functionality
Controls-Related Classes
Engine-Related Classes
Filesystem Functionality
GUI-Related Classes
Math Functionality
Node-Related Classes
Objects-Related Classes
Networking Functionality
Pathfinding-Related Classes
Physics-Related Classes
Plugins-Related Classes
IG Plugin
CIGIConnector Plugin
Rendering-Related Classes

Unigine::Plugins::Kinect Class

Header: #include <plugins/UnigineKinect.h>

This set of functions is available when the Kinect2 plugin is loaded. This plugin is used for receiving already detected data from a Kinect2 sensor.

Notice
Kinect SDK 2.0+ must be installed on your computer.

If the plugin is loaded together with the engine, the HAS_KINECT definition is set. This definition can be used, for example, to avoid errors if the plugin is not loaded: the code in which the plugin functions are executed can be wrapped around as follows:

Source code (UnigineScript)
#ifdef HAS_KINECT
	// engine.kinect functions
#endif

The Unigine Kinect plugin works in the foolowing way: when engine.kinect.isBodyTracked(body) returns 0, engine.kinect.getBonePosition(body, bone) will return vec3_zero. Kinect has its own framerate, so the body tracking information is not available each engine frame. Wrist tracking can be achieved via:

Source code (UnigineScript)
vec3 wrist;

int update() {
    if(engine.kinect.isBodyTracked(0)) {
        wrist = engine.kinect.getBonePosition(0,KINECT_BONE_WRIST_LEFT);
    }
    engine.visualizer.renderSphere(0.05f,translate(wrist),vec4_one);
    return 1;
}

See Also#

  • Article on Kinect2 Plugin
  • A set of UnigineScript API samples located in the <UnigineSDK>/data/samples/plugins/ folder:
    • kinect_00
    • kinect_01
    • kinect_02

Kinect Class

Enums

FACE_PROPERTY#

NameDescription
FACE_PROPERTY_HAPPY = 0The user's face is happy (for example, the user is smiling).
FACE_PROPERTY_ENGAGED = 1The user's face is engaged.
FACE_PROPERTY_GLASSES = 2There are glasses on the face.
FACE_PROPERTY_EYE_LEFT_CLOSED = 3The user's left eye is closed.
FACE_PROPERTY_EYE_RIGHT_CLOSED = 4The user's right eye is closed.
FACE_PROPERTY_MOUTH_OPEN = 5The user's mouth is open.
FACE_PROPERTY_MOUTH_MOVED = 6The user's mouth moved.
FACE_PROPERTY_LOOKING_AWAY = 7The user is looking away.

FACE_POINT#

NameDescription
FACE_POINT_EYE_LEFT = 0The left eye.
FACE_POINT_EYE_RIGHT = 1The right eye.
FACE_POINT_NOSE = 2The nose.
FACE_POINT_MOUTH_CORNER_LEFT = 3The left corner of the mouse.
FACE_POINT_MOUTH_CORNER_RIGHT = 4The right corner of the mouse.

FACE_DETECTION_RESULT#

NameDescription
FACE_DETECTION_RESULT_UNKNOWN = 0A flag indicating that the face is in the unknown state.
FACE_DETECTION_RESULT_NO = 1A flag indicating that the face property (state) is not tracked.
FACE_DETECTION_RESULT_MAYBE = 2A flag indicating that the face property (state) is partially tracked.
FACE_DETECTION_RESULT_YES = 3A flag indicating that the face property (state) is tracked.

BONE#

NameDescription
BONE_HEAD = 3The head.
BONE_NECK = 2The neck.
BONE_SPINE_SHOULDER = 20The spine at the shoulder.
BONE_SPINE_MID = 1Middle of the spine.
BONE_SPINE_BASE = 0Base of the spine.
BONE_SHOULDER_LEFT = 4The left shoulder.
BONE_ELBOW_LEFT = 5The left elbow.
BONE_WRIST_LEFT = 6The left wrist.
BONE_HAND_LEFT = 7The left hand.
BONE_HAND_TIP_LEFT = 21The tip of the left hand.
BONE_THUMB_LEFT = 22The left thumb.
BONE_SHOULDER_RIGHT = 8The right shoulder.
BONE_ELBOW_RIGHT = 9The right elbow.
BONE_WRIST_RIGHT = 10The right wrist.
BONE_HAND_RIGHT = 11The right hand.
BONE_HAND_TIP_RIGHT = 23The tip of the right hand.
BONE_THUMB_RIGHT = 24The right thumb.
BONE_HIP_LEFT = 12The left hip (except the thumb).
BONE_KNEE_LEFT = 13The left knee.
BONE_ANKLE_LEFT = 14The left ankle.
BONE_FOOT_LEFT = 15The left foot.
BONE_HIP_RIGHT = 16The right hip (except the thumb).
BONE_KNEE_RIGHT = 17The right knee.
BONE_ANKLE_RIGHT = 18The right ankle.
BONE_FOOT_RIGHT = 19The right foot.

HAND_STATE#

NameDescription
HAND_STATE_UNKNOWN = 0A flag indicating that the state of the hand is unknown.
HAND_STATE_NOT_TRACKED = 1A flag indicating that the state of the hand is not tracked.
HAND_STATE_OPEN = 2A flag indicating that the hand is open.
HAND_STATE_CLOSED = 3A flag indicating that the hand is closed (clenched in a fist).
HAND_STATE_LASSO = 4A flag indicating that the hand is in the lasso state (a closed hand with the middle and index fingers both up).

TRACKING_CONFIDENCE#

NameDescription
TRACKING_CONFIDENCE_LOW = 0A flag indicating that a hand is tracked with the low level of confidence (perhaps a hand is tracked correctly).
TRACKING_CONFIDENCE_HIGH = 1A flag indicating that a hand is tracked with the high level of confidence (a hand is fully tracked).

TRACKING_STATE#

NameDescription
TRACKING_STATE_NOT_TRACKED = 0A flag indicating that a body and bones aren't tracked.
TRACKING_STATE_INFERRED = 1A flag indicating that a body and bones are inferred (Kinect inferring their positions).
TRACKING_STATE_TRACKED = 2A flag indicating that a body and bones are being tracked.

NUM#

NameDescription
NUM_BODIES = 6Number of tracked bodies. The maximum value is 6.
NUM_BONES = 25Number of tracked bones. The maximum value is 25.
NUM_FACE_POINTS = 5Number of tracked face points (left and right eyes, nose, left and right mouth corners). The maximum value is 5.
NUM_FACE_PROPERTIES = 8Number of tracked face states (for example, happy, engaged, wearing glasses and so on). The maximum value is 8.

STREAM#

NameDescription
STREAM_COLOR = 1A color stream. If this constant isn't specified on sensor initialization, there will be no access to the color stream.
STREAM_INFRARED = 2An infrared stream. If this constant isn't specified on sensor initialization, there will be no access to the infrared stream.
STREAM_DEPTH = 8A depth stream. If this constant isn't specified on sensor initialization, there will be no access to the depth stream.
STREAM_BODY = 32A body. If this constant isn't specified on sensor initialization, bodies won't be tracked.
STREAM_ALL = 43The STREAM_COLOR, STREAM_INFRARED, STREAM_DEPTH, STREAM_BODY constants combined by using logical OR.

Members


Kinect::TRACKING_STATE getBodyLeanState ( int body ) const#

Returns a value indicating whether the body with the given number was tracked as lean or not.

Arguments

  • int body - Body index in range [0;NUM_BODIES - 1 ].

Return value

One of the TRACKING_STATE_* constants specifying the tracking state. If the STREAM_BODY is not specified on sensor initialization, TRACKING_STATE_NOT_TRACKED will be returned.

Math::quat getBoneOrientation ( int body, Kinect::BONE bone ) const#

Returns orientation of the given bone of the given body relative to the parent bone.

Arguments

  • int body - Body number in range [0;NUM_BODIES - 1 ].
  • Kinect::BONE bone - Bone number in range [0;NUM_BONES - 1 ].

Return value

Bone orientation relative to the parent bone. If the STREAM_BODY is not specified on sensor initialization, zero quat will be returned.

Math::vec3 getBonePosition ( int body, Kinect::BONE bone ) const#

Returns position of the given bone of the given body relative to the sensor.

Arguments

  • int body - Body number in range [0;NUM_BODIES - 1 ].
  • Kinect::BONE bone - Bone number in range [0;NUM_BONES - 1 ].

Return value

Bone position relative to the sensor. If the STREAM_BODY is not specified on sensor initialization, zero vector will be returned.

Kinect::TRACKING_STATE getBoneState ( int body, Kinect::BONE bone ) const#

Returns the current tracking state of the given bone of the given body.

Arguments

  • int body - Body number in range [0;NUM_BODIES - 1 ].
  • Kinect::BONE bone - Bone number in range [0;NUM_BONES - 1 ].

Return value

One of the TRACKING_STATE_* constants specifying the tracking state. If the STREAM_BODY is not specified on sensor initialization, TRACKING_STATE_NOT_TRACKED will be returned.

Ptr<Image> getColorBuffer ( ) const#

Returns an image representing the current color buffer in the RGBA8 format.
Notice
The image in the color buffer is deleted each frame. However, the data isn't sent to the buffer each frame, so the function may return NULL.

Return value

An image representing the current color buffer. If the STREAM_COLOR is not specified on sensor initialization, NULL will be returned.

Ptr<Image> getDepthBuffer ( ) const#

Returns an image representing the current depth buffer in the R16 format.
Notice
The image in the depth buffer is deleted each frame. However, the data isn't sent to the buffer each frame, so the function may return NULL.

Return value

An image representing the current depth buffer. If the STREAM_DEPTH is not specified on sensor initialization, NULL will be returned.

Math::ivec4 getFaceBoundsInColorSpace ( int face ) const#

Returns bounds of the given face relative to the size of the color buffer.

Arguments

  • int face - Face number in range [0;NUM_BODIES - 1 ].

Return value

Face bounds in the format (Left,Top,Right,Bottom). If the STREAM_BODY is not specified on sensor initialization, zero vector will be returned.

Math::ivec4 getFaceBoundsInInfraredSpace ( int face ) const#

Returns bounds of the given face relative to the size of the infrared buffer.

Arguments

  • int face - Face number in range [0;NUM_BODIES - 1 ].

Return value

Face bounds in the format (Left,Top,Right,Bottom). If the STREAM_BODY is not specified on sensor initialization, zero vector will be returned.

Math::quat getFaceOrientation ( int face ) const#

Returns orientation of the given face relative to the sensor.

Arguments

  • int face - Face number in range [0;NUM_BODIES - 1 ].

Return value

Face orientation. If the STREAM_BODY is not specified on sensor initialization, zero quat will be returned.

Math::vec3 getFacePointInColorSpace ( int face, Kinect::FACE_POINT point ) const#

Returns coordinates of the given point on the given face relative to the size of the color buffer.
Notice
Only X and Y components of the returned vector are used, Z component should be ignored.

Arguments

  • int face - Face number in range [0;NUM_BODIES - 1 ].
  • Kinect::FACE_POINT point - Face point number in range [0;NUM_FACE_POINTS - 1 ].

Return value

Face point coordinates. If the STREAM_BODY is not specified on sensor initialization, zero vector will be returned.

Math::vec3 getFacePointInInfraredSpace ( int face, Kinect::FACE_POINT point ) const#

Returns coordinates of the given point on the given face relative to the size of the infrared buffer.
Notice
Only X and Y components of the returned vector are used, Z component should be ignored.

Arguments

  • int face - Face number in range [0;NUM_BODIES - 1 ].
  • Kinect::FACE_POINT point - Face point number in range [0;NUM_FACE_POINTS - 1 ].

Return value

Face point coordinates. If the STREAM_BODY is not specified on sensor initialization, zero vector will be returned.

Kinect::FACE_DETECTION_RESULT getFaceProperty ( int face, Kinect::FACE_PROPERTY property ) const#

Returns a value indicating how accurate the property of the given face was tracked.

Arguments

Return value

One of the KINECT_FACE_DETECTION_RESULT_* constants. If The STREAM_BODY is not specified on sensor initialization, KINECT_FACE_DETECTION_RESULT_UNKNOWN will be returned.

Ptr<Image> getInfraredBuffer ( ) const#

Returns an image representing the current infrared buffer in the R16 format.
Notice
The image in the infrared buffer is deleted each frame. However, the data isn't sent to the buffer each frame, so the function may return NULL.

Return value

An image representing the current infrared buffer. If the STREAM_INFRARED is not specified on sensor initialization, NULL will be returned.

Kinect::TRACKING_CONFIDENCE getLeftHandConfidence ( int body ) const#

Returns the confidence level for the tracked left hand of the given body.

Arguments

  • int body - Body number in range [0;NUM_BODIES - 1 ].

Return value

One of the TRACKING_CONFIDENCE_*. If The STREAM_BODY is not specified on sensor initialization, TRACKING_CONFIDENCE_LOW will be returned.

Kinect::HAND_STATE getLeftHandState ( int body ) const#

Returns the current state of the left hand of the given body.

Arguments

  • int body - Body number in range [0;NUM_BODIES - 1 ].

Return value

One of the HAND_STATE_*. If The STREAM_BODY is not specified on sensor initialization, HAND_STATE_UNKNOWN will be returned.

Kinect::TRACKING_CONFIDENCE getRightHandConfidence ( int body ) const#

Returns the confidence level for the tracked right hand of the given body.

Arguments

  • int body - Body number in range [0;NUM_BODIES - 1 ].

Return value

One of the TRACKING_CONFIDENCE_*. If The STREAM_BODY is not specified on sensor initialization, TRACKING_CONFIDENCE_LOW will be returned.

Kinect::HAND_STATE getRightHandState ( int body ) const#

Returns the current state of the right hand of the given body.

Arguments

  • int body - Body number in range [0;NUM_BODIES - 1 ].

Return value

One of the HAND_STATE_*. If The STREAM_BODY is not specified on sensor initialization, HAND_STATE_UNKNOWN will be returned.

int init ( unsigned int stream_flags ) #

Kinect2 sensor initialization. The STREAM_* constants specify which data streams should be initialized.

Arguments

  • unsigned int stream_flags - A bit mask represented by one of or combination of the STREAM_* constants.

Return value

1 if the sensor was initialized successfully; otherwise, 0.

Examples

For example, there will be access to the color and depth buffers if you initialize the sensor as follows:

Source code (UnigineScript)
engine.kinect.init(KINECT_STREAM_COLOR | KINECT_STREAM_DEPTH);

bool isBodyTracked ( int body ) const#

Returns a value indicating if the body with the given number was tracked by the sensor.

Arguments

  • int body - Body number in range [0;NUM_BODIES - 1 ].

Return value

true if the body was tracked; otherwise, false. If the STREAM_BODY is not specified on sensor initialization, fals will be returned.

bool isFaceTracked ( int face ) const#

Returns a value indicating whether the face with the given number was tracked or not.

Arguments

  • int face - Face number in range [0;NUM_BODIES - 1 ].

Return value

true if the given face is tracked; otherwise, false. If the STREAM_BODY is not specified on sensor initialization, false will be returned.

void shutdown ( ) #

Kinect2 sensor shutdown.
Last update: 2020-10-10