Tracking Hands and Fingers With LeapMotion Plugin

The Leap Motion system uses a right-handed Cartesian coordinate system. The origin is centered at the top of the Leap Motion Controller. The x- and z-axes lie in the horizontal plane, with the x-axis running parallel to the long edge of the device. The y-axis is vertical, with positive values increasing upwards (in contrast to the downward orientation of most computer graphics coordinate systems). The z-axis has positive values increasing toward the user.

The hand model provides information about the identity, position, and other characteristics of a detected hand, the arm to which the hand is attached, and lists of the fingers associated with the hand.

Hands are represented by the Hand class.

An arm is a bone-like object that provides the orientation, length, width, and end points of an arm. When the elbow is not in view, the Leap Motion controller estimates its position based on past observations as well as typical human proportion.

Arms are represented by the Arm class.

The Leap Motion controller provides information about each finger on a hand. If all or part of a finger is not visible, the finger characteristics are estimated based on recent observations and the anatomical model of the hand. Fingers are identified by type name, i.e. thumb, index, middle, ring, and pinky.

Fingers are represented by the Finger class.

Each finger has a set of bones describing the position and orientation of corresponding anatomical finger bones. All fingers contain four bones ordered from base to tip.

Bones are represented by the Bone class.

The bones are identified as:

• Metacarpal – the bone inside the hand connecting the finger to the wrist (except the thumb).
• Proximal Phalanx – the bone at the base of the finger, connected to the palm.
• Intermediate Phalanx – the middle bone of the finger, between the tip and the base.
• Distal Phalanx – the terminal bone at the end of the finger.

Along with the computed tracking data, you can get the raw sensor images from the Leap Motion cameras.

The image data contains the measured IR brightness values and the calibration data required to correct for the complex lens distortion. You can use the sensor images for augmented reality applications, especially when the Leap Motion hardware is mounted to a VR headset.

• C++ sample illustrating the basic aspects of working with the plugin:
  source/samples/3rdparty/LeapMotionVisualizer

LeapMotion API:

• The LeapMotion interface article for more details on managing LeapMotion via code (C++, UnigineScript).
• The LeapMotion Arm class article for more details on managing arms via code (C++, UnigineScript).
• The LeapMotion Bone class article for more details on managing finger bones via code (C++, UnigineScript).
• The LeapMotion Finger class article for more details on managing fingers via code (C++, UnigineScript).
• The LeapMotion Hand class article for more details on managing hands via code (C++, UnigineScript).