Programming
Fundamentals
Setting Up Development Environment
Usage Examples
UnigineScript
High-Level Systems
C++
C#
UUSL (Unified UNIGINE Shader Language)
File Formats
Rebuilding the Engine and Tools
GUI
Double Precision Coordinates
API
Containers
Common Functionality
Controls-Related Classes
Engine-Related Classes
Filesystem Functionality
GUI-Related Classes
Math Functionality
Node-Related Classes
Networking Functionality
Pathfinding-Related Classes
Physics-Related Classes
Rendering-Related Classes

engine.kinect Functions

This set of functions is available when the Kinect2 plugin add-on is loaded. This plugin is used for receiving already detected data from a Kinect2 sensor.

If the plugin is loaded together with the engine, the HAS_KINECT definition is set. This definition can be used, for example, to avoid errors if the plugin is not loaded: the code in which the plugin functions are executed can be wrapped around as follows:

Source code (UnigineScript)
#ifdef HAS_KINECT
	// engine.kinect functions
#endif

The Unigine Kinect plugin works in the foolowing way: when engine.kinect.isBodyTracked(body) returns 0, engine.kinect.getBonePosition(body, bone) will return vec3_zero. Kinect has its own framerate, so the body tracking information is not available each engine frame. The wrist tracking can be achieved via:

Source code (UnigineScript)
vec3 wrist;

int update() {
    if(engine.kinect.isBodyTracked(0)) {
        wrist = engine.kinect.getBonePosition(0,KINECT_BONE_WRIST_LEFT);
    }
    engine.visualizer.renderSphere(0.05f,translate(wrist),vec4_one);
    return 1;
}

See Also

int KINECT_BONE_ANKLE_LEFT

Description

The left ankle.

int KINECT_BONE_ANKLE_RIGHT

Description

The right ankle.

int KINECT_BONE_ELBOW_LEFT

Description

The left elbow.

int KINECT_BONE_FOOT_LEFT

Description

The left foot.

int KINECT_BONE_FOOT_RIGHT

Description

The right foot.

int KINECT_BONE_HAND_LEFT

Description

The left hand.

int KINECT_BONE_HAND_RIGHT

Description

The right hand.

int KINECT_BONE_HAND_TIP_LEFT

Description

The tip of the left hand.

int KINECT_BONE_HAND_TIP_RIGHT

Description

The tip of the right hand.

int KINECT_BONE_HEAD

Description

A head.

int KINECT_BONE_HIP_LEFT

Description

The left hip (except the thumb).

int KINECT_BONE_HIP_RIGHT

Description

The right hip (except the thumb).

int KINECT_BONE_KNEE_LEFT

Description

The left knee.

int KINECT_BONE_KNEE_RIGHT

Description

The right knee.

int KINECT_BONE_NECK

Description

A neck.

int KINECT_BONE_SHOULDER_LEFT

Description

The left shoulder.

int KINECT_BONE_SHOULDER_RIGHT

Description

The right shoulder.

int KINECT_BONE_SPINE_BASE

Description

Base of the spine.

int KINECT_BONE_SPINE_MID

Description

Middle of the spine.

int KINECT_BONE_SPINE_SHOULDER

Description

A spine at the shoulder.

int KINECT_BONE_THUMB_LEFT

Description

The left thumb.

int KINECT_BONE_THUMB_RIGHT

Description

The right thumb.

int KINECT_BONE_WRIST_LEFT

Description

The left wrist.

int KINECT_BONE_WRIST_RIGHT

Description

The right wrist.

int KINECT_FACE_DETECTION_RESULT_MAYBE

Description

A flag indicating that the face property (state) is partially tracked.

int KINECT_FACE_DETECTION_RESULT_NO

Description

A flag indicating that the face property (state) is not tracked.

int KINECT_FACE_DETECTION_RESULT_UNKNOWN

Description

A flag indicating that the face is in the unknown state.

int KINECT_FACE_DETECTION_RESULT_YES

Description

A flag indicating that the face property (state) is tracked.

int KINECT_FACE_POINT_EYE_LEFT

Description

The left eye.

int KINECT_FACE_POINT_EYE_RIGHT

Description

The right eye.

int KINECT_FACE_POINT_MOUTH_CORNER_LEFT

Description

The left corner of the mouse.

int KINECT_FACE_POINT_MOUTH_CORNER_RIGHT

Description

The right corner of the mouse.

int KINECT_FACE_POINT_NOSE

Description

The nose.

int KINECT_FACE_PROPERTY_ENGAGED

Description

The user's face is engaged.

int KINECT_FACE_PROPERTY_EYE_LEFT_CLOSED

Description

The user's left eye is closed.

int KINECT_FACE_PROPERTY_EYE_RIGHT_CLOSED

Description

The user's right eye is closed.

int KINECT_FACE_PROPERTY_GLASSES

Description

There are glasses on the face.

int KINECT_FACE_PROPERTY_HAPPY

Description

The user's face is happy (for example, the user is smiling).

int KINECT_FACE_PROPERTY_LOOKING_AWAY

Description

The user is looking away.

int KINECT_FACE_PROPERTY_MOUTH_MOVED

Description

The user's mouth moved.

int KINECT_FACE_PROPERTY_MOUTH_OPEN

Description

The user's mouth is open.

int KINECT_HAND_STATE_CLOSED

Description

A flag indicating that the hand is closed (clenched in a fist).

int KINECT_HAND_STATE_LASSO

Description

A flag indicating that the hand is in the lasso state (a closed hand with the middle and index fingers both up).

int KINECT_HAND_STATE_NOT_TRACKED

Description

A flag indicating that the state of the hand is not tracked.

int KINECT_HAND_STATE_OPEN

Description

A flag indicating that the hand is open.

int KINECT_HAND_STATE_UNKNOWN

Description

A flag indicating that the state of the hand is unknown.

int KINECT_NUM_BODIES

Description

Number of tracked bodies. The maximum value is 6.

int KINECT_NUM_BONES

Description

Number of tracked bones. The maximum value is 25.

int KINECT_NUM_FACE_POINTS

Description

Number of tracked face points (left and right eyes, nose, left and right mouth corners). The maximum value is 5.

int KINECT_NUM_FACE_PROPERTIES

Description

Number of tracked face states (for example, happy, engaged, wearing glasses and so on). The maximum value is 8.

int KINECT_STREAM_ALL

Description

The KINECT_STREAM_COLOR, KINECT_STREAM_INFRARED, KINECT_STREAM_DEPTH, KINECT_STREAM_BODY constants combined by usinglogical OR.

int KINECT_STREAM_BODY

Description

A body. If this constant isn't specified on sensor initialization, bodies won't be tracked.

int KINECT_STREAM_COLOR

Description

A color stream. If this constant isn't specified on sensor initialization, there will be no access to the color stream.

int KINECT_STREAM_DEPTH

Description

A depth stream. If this constant isn't specified on sensor initialization, there will be no access to the depth stream.

int KINECT_STREAM_INFRARED

Description

An infrared stream. If this constant isn't specified on sensor initialization, there will be no access to the infrared stream.

int KINECT_TRACKING_CONFIDENCE_HIGH

Description

A flag indicating that a hand is tracked with the high level of confidence (a hand is fully tracked).

int KINECT_TRACKING_CONFIDENCE_LOW

Description

A flag indicating that a hand is tracked with the low level of confidence (perhaps a hand is tracked correctly).

int KINECT_TRACKING_STATE_INFERRED

Description

A flag indicating that a body and bones are inferred (Kinect inferring their positions).

int KINECT_TRACKING_STATE_NOT_TRACKED

Description

A flag indicating that a body and bones aren't tracked.

int KINECT_TRACKING_STATE_TRACKED

Description

A flag indicating that a body and bones are being tracked.

int engine.kinect.getBodyLeanState(int body)

Returns a value indicating whether the body with the given number was tracked as lean or not.

Arguments

Return value

One of the KINECT_TRACKING_STATE_* constants specifying the tracking state. If the KINECT_STREAM_BODY is not specified on sensor initialization, KINECT_TRACKING_STATE_NOT_TRACKED will be returned.

quat engine.kinect.getBoneOrientation(int body, int bone)

Returns orientation of the given bone of the given body relative to the parent bone.

Arguments

Return value

Bone orientation relative to the parent bone. If the KINECT_STREAM_BODY is not specified on sensor initialization, zero quat will be returned.

vec3 engine.kinect.getBonePosition(int body, int bone)

Returns position of the given bone of the given body relative to the sensor.

Arguments

Return value

Bone position relative to the sensor. If the KINECT_STREAM_BODY is not specified on sensor initialization, zero vector will be returned.

int engine.kinect.getBoneState(int body, int bone)

Returns the current tracking state of the given bone of the given body.

Arguments

Return value

One of the KINECT_TRACKING_STATE_* constants specifying the tracking state. If the KINECT_STREAM_BODY is not specified on sensor initialization, KINECT_TRACKING_STATE_NOT_TRACKED will be returned.

Image engine.kinect.getColorBuffer()

Returns an image representing the current color buffer in the RGBA8 format.
Notice
The image in the color buffer is deleted each frame. However, the data isn't sent to the buffer each frame, so the function lang="usc" may return NULL.

Return value

An image representing the current color buffer. If the KINECT_STREAM_COLOR is not specified on sensor initialization, NULL will be returned.

Image engine.kinect.getDepthBuffer()

Returns an image representing the current depth buffer in the R16 format.
Notice
The image in the depth buffer is deleted each frame. However, the data isn't sent to the buffer each frame, so the function lang="usc" may return NULL.

Return value

An image representing the current depth buffer. If the KINECT_STREAM_DEPTH is not specified on sensor initialization, NULL will be returned.

ivec4 engine.kinect.getFaceBoundsInColorSpace(int face)

Returns bounds of the given face relative to the size of the color buffer.

Arguments

Return value

Face bounds in the format (Left,Top,Right,Bottom). If the KINECT_STREAM_BODY is not specified on sensor initialization, zero vector will be returned.

ivec4 engine.kinect.getFaceBoundsInInfraredSpace(int face)

Returns bounds of the given face relative to the size of the infrared buffer.

Arguments

Return value

Face bounds in the format (Left,Top,Right,Bottom). If the KINECT_STREAM_BODY is not specified on sensor initialization, zero vector will be returned.

quat engine.kinect.getFaceOrientation(int face)

Returns orientation of the given face relative to the sensor.

Arguments

Return value

Face orientation. If the KINECT_STREAM_BODY is not specified on sensor initialization, zero quat will be returned.

vec3 engine.kinect.getFacePointInColorSpace(int face, int point)

Returns coordinates of the given point on the given face relative to the size of the color buffer.
Notice
Only X and Y components of the returned vector are used, Z component should be ignored.

Arguments

Return value

Face point coordinates. If the KINECT_STREAM_BODY is not specified on sensor initialization, zero vector will be returned.

vec3 engine.kinect.getFacePointInInfraredSpace(int face, int point)

Returns coordinates of the given point on the given face relative to the size of the infrared buffer.
Notice
Only X and Y components of the returned vector are used, Z component should be ignored.

Arguments

Return value

Face point coordinates. If the KINECT_STREAM_BODY is not specified on sensor initialization, zero vector will be returned.

int engine.kinect.getFaceProperty(int face, int property)

Returns a value indicating how accurate the property of the given face was tracked.

Arguments

Return value

One of the KINECT_FACE_DETECTION_RESULT_* constants. If The KINECT_STREAM_BODY is not specified on sensor initialization, KINECT_FACE_DETECTION_RESULT_UNKNOWN will be returned.

Image engine.kinect.getInfraredBuffer()

Returns an image representing the current infrared buffer in the R16 format.
Notice
The image in the infrared buffer is deleted each frame. However, the data isn't sent to the buffer each frame, so the function lang="usc" may return NULL.

Return value

An image representing the current infrared buffer. If the KINECT_STREAM_INFRARED is not specified on sensor initialization, NULL will be returned.

int engine.kinect.getLeftHandConfidence(int body)

Returns the confidence level for the tracked left hand of the given body.

Arguments

Return value

One of the KINECT_TRACKING_CONFIDENCE_*. If The KINECT_STREAM_BODY is not specified on sensor initialization, KINECT_TRACKING_CONFIDENCE_LOW will be returned.

int engine.kinect.getLeftHandState(int body)

Returns the current state of the left hand of the given body.

Arguments

Return value

One of the KINECT_HAND_STATE_*. If The KINECT_STREAM_BODY is not specified on sensor initialization, KINECT_HAND_STATE_UNKNOWN will be returned.

int engine.kinect.getRightHandConfidence(int body)

Returns the confidence level for the tracked right hand of the given body.

Arguments

Return value

One of the KINECT_TRACKING_CONFIDENCE_*. If The KINECT_STREAM_BODY is not specified on sensor initialization, KINECT_TRACKING_CONFIDENCE_LOW will be returned.

int engine.kinect.getRightHandState(int body)

Returns the current state of the right hand of the given body.

Arguments

Return value

One of the KINECT_HAND_STATE_*. If The KINECT_STREAM_BODY is not specified on sensor initialization, KINECT_HAND_STATE_UNKNOWN will be returned.

int engine.kinect.init(int flags)

Kinect2 sensor initialization. The KINECT_STREAM_* constants specify which data streams should be initialized.

Arguments

  • int flags - A bit mask represented by one of or combination of the KINECT_STREAM_* constants.

Return value

1 if the sensor was initialized successfully; otherwise, 0.

Examples

For example, there will be access to the color and depth buffers if you initialize the sensor as follows:

Source code (UnigineScript)
engine.kinect.init(KINECT_STREAM_COLOR | KINECT_STREAM_DEPTH);

int engine.kinect.isBodyTracked(int body)

Returns a value indicating if the body with the given number was tracked by the sensor.

Arguments

Return value

1 if the body was tracked; otherwise, 0. If the KINECT_STREAM_BODY is not specified on sensor initialization, 0 will be returned.

int engine.kinect.isFaceTracked(int face)

Returns a value indicating whether the face with the given number was tracked or not.

Arguments

Return value

1 if the given face is tracked; otherwise, 0. If the KINECT_STREAM_BODY is not specified on sensor initialization, 0 will be returned.

void engine.kinect.shutdown()

Kinect2 sensor shutdown.
Last update: 2017-07-03