Using Cortana in Windows 10 to Automate Skype For Business Part I

The goal of this blog series is to show some of the capability that can be harnessed in Cortana for Windows 10.  In this example, I am going to use it to listen and respond to commands that can automate status updates and call control in Skype For Business.

Please watch the video above before continuing with this post.

Step 1.  Windows Universal.

The Universal Windows Platform (UWP) is the name for this new technology that let's developers write apps that can run on desktops, tablets, xbox, and Windows phone.  Details Here.

To build a universal app you are going to want to Visual Studio 2015, and Windows 10.  Once you have your environment in place, make sure you setup and configure Cortana so she can understand your voice.

Step 2.  Create a UWP app.

In Visual Studio create a new project, navigate to the Windows Universal section, and give it a project name.

The 2nd set is to add a 2nd project to our solution called CortanaComponent or something like that, make sure you specify the type "Windows Runtime Component"

Now in your first project you need to configure a declaration to this App Service like I have in this below screenshot.

This will give us the ability for us to process voice commands from our application outside of cortana if we desire to do so down the line.

Step 3.  What to have Cortana listen for.

In this example I am going to use both the XML file method for voice commands, as well as a hybrid where I am going to populate a Phraselist with user names from our company to make the voice search exponentially more accurate.  So here is the XML.  Create your own similar and add it to your project.

<?xml version="1.0" encoding="utf-8" ?>

<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.2">

  <CommandSet xml:lang="en-us" Name="S4BCommandSet_en-us">

    <CommandPrefix>Gideon</CommandPrefix>

    <Example>Transfer call to, Hold call, Answer call</Example>

    <Command Name="Transfer">

      <Example> Transfer call to Doug Routledge </Example>

      <ListenFor> Transfer call to {destination} </ListenFor>

      <Feedback> Ok I am transfering this call to {destination} </Feedback>

      <Navigate/>

    </Command>

    <Command Name="Park">

      <Example> Park call for Doug Routledge </Example>

      <ListenFor> Park call for {destination} </ListenFor>

      <Feedback> Ok I am parking this call for {destination} </Feedback>

      <Navigate/>

    </Command>

    <Command Name="SetStatus">

      <Example> Set my status to Busy </Example>

      <ListenFor> Set my status to {status} </ListenFor>

      <Feedback> Ok I set your status to {status} </Feedback>

      <Navigate/>

    </Command>

    <PhraseList Label="destination">

      <!--<Item> I will fill in from code </Item>-->

    </PhraseList>

    <PhraseList Label="status">

      <Item> Busy </Item>

      <Item> Do No Disturb </Item>

      <Item> Available </Item>

      <Item> Be Right Back </Item>

      <Item> Appear Away </Item>

      <Item> Off Work </Item>

    </PhraseList>

  </CommandSet>

  <!-- Other CommandSets for other languages  RequireAppName="BeforeOrAfterPhrase">-->

</VoiceCommands>

The first thing to note is the <CommandPrefix> section.  Here I am choosing the name Gideon for my teenage daughter who loves The Flash on the CW Network.  That will give us something to say at the start of our commands, that tells Cortana which app to send the command to.  Take a minute a examine the rest of the file, you will notice I have 3 command type in my file.  The first is to transfer a call, the 2nd is to park a call for a user, and the 3rd is change my Skype for Business status.

The PhraseList destination will be populated by talking to our software which can return all the user names, making our voice searches deadly accurate.

Step 4.  App.xaml.cs

Open your Appl.xaml.cs file and make these modifications.

In  protected async override void OnLaunched(LaunchActivatedEventArgs e)

add the following code block, remembering to change the filename to match your xml voice commands file.

try

            {

                System.Diagnostics.Debug.WriteLine("Load Voice Commands");

                var storageFile = await Windows.Storage.StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///VoiceCommands.xml"));

                await Windows.ApplicationModel.VoiceCommands.VoiceCommandDefinitionManager.InstallCommandDefinitionsFromStorageFileAsync(storageFile);

            }

            catch

            { }

You will want to add this at the top as well.

using Windows.Media.SpeechRecognition;

Once you have those in place your app will now load the voice commands file and you should be able to do some basic debug testing to see if Cortana is recognizing your commands.

Next week we'll look at how we populate or destination Phraselist from code, and how to handle the voice commands when they are sent to us from Cortana.

Doug Routledge, C# Lync, Skype for Business, SQL, Exchange, UC Developer  BridgeOC
Twitter - @droutledge @ndbridge