Making Microsoft Lync be able to see and hear Part 2/3

In part one of this series I discussed how you can use the Kinect V2 for Windows to make programs listen without having to enable the terrible Windows.Speech library. As we continue to try to move Lync closer to a place where it can function with only sight and sound input, we come to the sight part. In our app we decided there were 2 distrint visual things we wanted to accomplish with the Kinect integration.

1. We want to be able to use the PHIZ Kinect offers to be able to use our hand like a mouse cursor, ala Mr. Miyagi doing a wax on, wax off.

2. We wanted to be able to use some machine learning gestures.

This article will deal with #1.

So to begin we wanted a couple of things. First we want our program to be able to pop out with an appropriate sized menu bar consisting of buttons at least the 208x208 Kinect recommended minimum size. This obviously would need to appear on top of everything else. The other consideration would be to make the menu hide again once you put your hand down. This seemed like a simple idea, but it turns out there is not any set of events that can be triggered when you raise or put your hand down to interact in the PHIZ. We submitted a question on the forum;
http://social.msdn.microsoft.com/Forums/en-US/1abee91e-ddd7-4592-8647-b5ae56c3062a/event-notification-when-kinect-hand-is-engaged-disengaged?forum=kinectv2sdk


Fortunately Rob at the Kinect team came to the rescue with a work around. Using something like this in a dispatcher timer we were then able to detect if you were in hand engaged or disengaged mode and act accordingly. The great thing about Kinect v2 for Windows is it works with standard WPF controls, so you don't need to use custom menu or button objects any more, you can just make existing WPF controls the appropriate size. Make sure you add the WPF controls as a reference, and make a <KinectRegion><Other WPF Controls></KinectRegion> to house the area your hand cursor will be allowed to operate in.

Here is our result of our simple 4 button call control menu.
http://www.bridgeoc.com/images/kinect.mp4