The Samples contain a project which shows how to use the Kinect sensor to control a 3D model. Here is a video of this sample:

• Video: Real-Time Motion Capture with Kinect and XNA

The sample project demonstrates 2 different methods to animate 3D models using Kinect in real-time: The first approach uses Skeleton Mapping to animate the XNA Dude model and a Space Marine model. There are three different skeletons involved: The Kinect player skeleton, the Dude skeleton and the Space Marine skeleton. All three skeletons are different, i.e. have different bone names and a different number of bones. DigitalRune Animation contains a SkeletonMapper which can be used to transfer an animation from one skeleton to another skeleton with a different structure.

Setting up the bone mapping for a model is not very difficult. This is the setup for the Dude model for (Kinect v2):

_skeletonMapperA = new SkeletonMapper(_kinectWrapper.SkeletonPoseA, _meshNodeA.SkeletonPose);
var ks = _kinectWrapper.SkeletonPoseA.Skeleton;
var ms = _meshNodeA.SkeletonPose.Skeleton;

_skeletonMapperA.BoneMappers.Add(new DirectBoneMapper(ks.GetIndex("HipCenter"), ms.GetIndex("Root"))
{
  MapTranslations = true,
  ScaleAToB = 1f,           // TODO: Make this scale factor configurable.
});

_skeletonMapperA.BoneMappers.Add(new UpperBackBoneMapper(
  ks.GetIndex("Spine"), ks.GetIndex("ShoulderCenter"), ks.GetIndex("ShoulderLeft"), ks.GetIndex("ShoulderRight"),
  ms.GetIndex("Spine"), ms.GetIndex("Neck"), ms.GetIndex("R_UpperArm"), ms.GetIndex("L_UpperArm")));

_skeletonMapperA.BoneMappers.Add(new ChainBoneMapper(ks.GetIndex("ShoulderLeft"), ks.GetIndex("ElbowLeft"), ms.GetIndex("R_UpperArm"), ms.GetIndex("R_Forearm")));
_skeletonMapperA.BoneMappers.Add(new ChainBoneMapper(ks.GetIndex("ShoulderRight"), ks.GetIndex("ElbowRight"), ms.GetIndex("L_UpperArm"), ms.GetIndex("L_Forearm")));
_skeletonMapperA.BoneMappers.Add(new ChainBoneMapper(ks.GetIndex("ElbowLeft"), ks.GetIndex("WristLeft"), ms.GetIndex("R_Forearm"), ms.GetIndex("R_Hand")));
_skeletonMapperA.BoneMappers.Add(new ChainBoneMapper(ks.GetIndex("ElbowRight"), ks.GetIndex("WristRight"), ms.GetIndex("L_Forearm"), ms.GetIndex("L_Hand")));
_skeletonMapperA.BoneMappers.Add(new ChainBoneMapper(ks.GetIndex("HipLeft"), ks.GetIndex("KneeLeft"), ms.GetIndex("R_Thigh"), ms.GetIndex("R_Knee")));
_skeletonMapperA.BoneMappers.Add(new ChainBoneMapper(ks.GetIndex("HipRight"), ks.GetIndex("KneeRight"), ms.GetIndex("L_Thigh1"), ms.GetIndex("L_Knee2")));
_skeletonMapperA.BoneMappers.Add(new ChainBoneMapper(ks.GetIndex("KneeLeft"), ks.GetIndex("AnkleLeft"), ms.GetIndex("R_Knee"), ms.GetIndex("R_Ankle")));
_skeletonMapperA.BoneMappers.Add(new ChainBoneMapper(ks.GetIndex("KneeRight"), ks.GetIndex("AnkleRight"), ms.GetIndex("L_Knee2"), ms.GetIndex("L_Ankle1")));
_skeletonMapperA.BoneMappers.Add(new ChainBoneMapper(ks.GetIndex("ShoulderCenter"), ks.GetIndex("Head"), ms.GetIndex("Neck"), ms.GetIndex("Head")));

The setup for the Space Marine model looks similar – only the bone names are different. The sample source code, contains additional comments and explanations.

In each frame the SkeletonMapper is called to transfer the current pose of the Kinect player skeleton to the skeleton of the 3D model:

_skeletonMapperA.MapAToB();

Kinect sensor input is not perfect and might cause jittering. Jittering can be reduced by applying a low-pass filter to the bone orientations.

The second method uses a completely different approach: A physics ragdoll is created for the Dude, which is used to animate the model. Certain joints of the Kinect player skeleton are used as target positions (e.g. hands, elbows, head, knees, etc.). The ragdoll is attached to the target positions using weak ball joint constraints. – This is like a marionette: The ragdoll is the puppet and the ball joints are strings that pull on the marionette.

This approach is more complex: A suitable ragdoll needs to be created for each 3D model. It can be difficult to make this approach stable. A lot of parameter tuning is required.

The skeleton mapping approach is a lot simpler and in most cases yields better results. But the direct mapping does not prevent unrealistic poses, for example, when the Kinect reports wrong data and the 3D model ends up in an unrealistic, twisted pose.

The marionette approach is difficult to use, but has the advantage of configurable joint limits: The ragdoll can be set up with proper limits to prevent unrealistic rotations. In addition, the ragdoll can interact with other objects. It can push other bodies – and it can itself be pushed by other objects.

In practice the skeleton mapping approach will satisfy most needs.