Kinect Application Project Template

People often have difficulty getting started with Kinect programming. This VS 2012 project template for WPF and C# stubs out many standard patterns developed over the months since the Kinect SDK was first released including setting up stream event handlers, deallocating streams and closing the sensor, gracefully handling situations like accidentally unplugging the Kinect sensor.

This project template will work with both the Kinect 4 Windows sensor as well as the standard Xbox Kinect sensor for development mode.

The template additionally includes extension methods for the Kinect Sensor object that makes it easy to create a green screen (background subtraction) image, a color blob image, or draw a player skeleton.

These are provided to make it easier to replicate the most common visualization scenarios for presentations or quick prototypes. The extension methods are based on the ones I originally wrote for the Apress bookBeginning Kinect Programming with the Microsoft Kinect SDK.

A deeper discussion of the capabilities of this project template can be found here.

Following are some key features:

1. Initialization Code

All the initialization code and Kinect stream event handlers are stubbed out in theInitSensor method. All you need to do is uncomment the streams you want to use. Additionally, the event handler code is also stubbed out with the proper pattern for opening and disposing of frame objects. Whatever you need to do with the image, depth and skeleton frames can be done inside those using statements. This code also uses the latest agreed upon best practices for efficiently managing streamed data as of the 1.7 SDK.

void sensor_ColorFrameReady(object sender    , ColorImageFrameReadyEventArgs e){    using (ColorImageFrame frame = e.OpenColorImageFrame())    {        if (frame == null)            return;        if (_colorBits == null) _colorBits =             new byte[frame.PixelDataLength];        frame.CopyPixelDataTo(_colorBits);        throw new NotImplementedException();    }}

2. Disposal Code

Whatever you enable in the InitSensor method you will need to disable and dispose of in theDeInitSensor method. Again, this just requires uncommenting the appropriate lines. TheDeInitSensor also implements a disposal pattern that is somewhat popular now. The sensor is actually shut down on a background thread rather than on the main thread. I’m not sure if this is a best practice as such, but it resolves a problem many C# developers were running into in shutting down their Kinect-enabled applications.

3. Status Changed Code

The Kinect can actually be disconnected in mid-process or simply not be on when you first run an application. It is also surprisingly common to forget to plug the Kinect’s power supply in. Generally, your application will just crash in such situations. If you properly handle the KinectSensors.StatusChanged event, however, your application will just start up again when you get the sensor plugged back in. A pattern for doing this was first introduced in the KinectChooser component in the Developer Toolkit. A lightweight version of this pattern is included in the Kinect Application Project Template.

void KinectSensors_StatusChanged(object sender    , StatusChangedEventArgs e){    if (e.Status == KinectStatus.Disconnected)    {        if (_sensor != null)        {            DeInitSensor(_sensor);        }    }    if (e.Status == KinectStatus.Connected)    {        _sensor = e.Sensor;        InitSensor(_sensor);    }}

4. Extension Methods

While most people were working on controls for the Kinect 4 Windows SDK, Clint Rutkas and the Coding4Fun guys brilliantly came up with the idea of developing extension methods for handling the various Kinect streams.

The extension methods included with this template provide lots of conversions from bitmaps byte arrays toBitmapSource types (useful for WPF image controls) and vice-versa. This allows you to do something easy like display a color stream which otherwise can be rather hairy. The snippet below assumes there is an image control in theMainWindow named canvas.

using (ColorImageFrame frame = e.OpenColorImageFrame()){    if (frame == null)        return;    if (_colorBits == null) _colorBits =         new byte[frame.PixelDataLength];    frame.CopyPixelDataTo(_colorBits);    // new line    this.canvas.Source =         _colorBits.ToBitmapSource(PixelFormats.Bgr32, 640, 480);}

More in line with the original Coding4Fun Toolkit, the extension methods also make some very difficult scenarios trivial – for instance background subtraction (also known as green screening), skeleton drawing, player masking. These methods should make it easier for quickly mock up a demo or even show off the power of the Kinect in the middle of a presentation using just a few lines of code.

private void InitSensor(KinectSensor sensor){    if (sensor == null)        return;    sensor.ColorStream.Enable();    sensor.DepthStream.Enable();    sensor.SkeletonStream.Enable();    sensor.Start();    this.canvas.Source = sensor.RenderActivePlayer();}

Again, this code assumes there is an image control in MainWindow named canvas.  You’ll want to put the following code in the InitSensor method to ensure that the code is called again if your Kinect sensor accidentally gets dislodged.  To create a simple background subtraction image, enable the color, depth and skeleton streams and then call the RenderActivePlayer extension method.  By stacking another image beneath the canvas image, I create an effect like this:
Here are some overloads of the RenderActivePlayer method and the effects they create.  I’ve removed Tatooine from the background in the following samples.
 
canvas.Source = sensor.RenderActivePlayer(System.Drawing.Color.Blue);
 
canvas.Source = sensor.RenderActivePlayer(System.Drawing.Color.Blue                , System.Drawing.Color.Fuchsia);
 
canvas.Source = sensor.RenderActivePlayer(System.Drawing.Color.Transparent                , System.Drawing.Color.Fuchsia);
 
And so on.  There’s also this one:
canvas.Source = sensor.RenderPredatorView();
 
… as well as this oldie but goodie:
canvas.Source = sensor.RenderPlayerSkeleton();
The base method uses the colors (and quite honestly most of the code) from the Kinect Toolkit that goes with the SDK.  As with the RenderActivePlayer extension method, however, there are lots of overrides so you can change all the colors if you wish to.
canvas.Source = sensor.RenderPlayerSkeleton(System.Drawing.Color.Turquoise    , System.Drawing.Color.Indigo    , System.Drawing.Color.IndianRed    , trackedBoneThickness: 1    , jointThickness: 10);
 
Finally, you can also layer all these different effects:
canvas.Source = sensor.RenderActivePlayer();canvas2.Source = sensor.RenderPlayerSkeleton(System.Drawing.Color.Transparent);