Making Microsoft Lync be able to see and hear Part 1/3

Having grown up in a house with a physically handicapped brother, it seems the idea of how to make software more accessible is always stuck in the back of my mind. Despite his handicap, my brother has had a job his entire adult life. In today's world, not being able to proficiently use a keyboard and mouse quickly eliminates a lot of jobs from your potential list. Considering most of the jobs that don't involve technology are traditionally of a more physical nature, you really aren't left with a lot of options. 

Last week I received my Kinect for Windows v2 and my team and I set off on a journey to make it work with our Bridge Lync Operator Console, an attendant console for Microsoft Lync, as well as our Bridge Operator Console for Cisco phone systems.

Step 1 - Using your voice.

Today's post will cover the first step, being able to automate call control functions in BLOC3 and therefore Microsoft Lync. One of the best things about the Kinect is it uses the speech recognition technology that Lync uses, not the stuff build into Windows that has been plagued with errors for years. What our team did is enable communication between the attendant console and kinect apps so they could exchange information, and our kinect app could "automate" Bridge Lync Operator Console, which intern "automates" Lync.

What to listen for?

We decided to make our kinect app listen for the basic call control commands like answer, hang up, hold, and resume. We also decided it was a good idea to require a confirmation before executing all of the commands, with the exception of hold. That way when an operator says "Please hold" we actually hold immediately without any confirmation, making the call process smoother. So now that we had a call, how on earth would we get kinect to know the difference between Derek Riser and Dave Mathews? Again since the kinect is build on the Microsoft.Speech library we were able to feed it the names of all 600 of our Lync endpoints in a few milliseconds, and even in my noisy office full of servers, it's ability to hear and differentiate between them was nothing short of amazing. We would say "Transfer to Derek Riser" then "Confirm" when we saw the confirmation window.



The end result was our program would look up Derek and transfer the call to him, then return itself to our default screen. With no hands, an operator is able to take on the operator / receptionist job, as well as any number of call center positions. Check back next week when we add the ability to optionally use your hands, and the following week for gestures as we close the communications gap between Lync and Kinect.


Doug Routledge, C# Lync, SQL, Exchange, UC Developer