Bots sample with speech recognition and synthesis

Some posts ago I wrote about the Microsoft Bot Framework, giving some useful links. Among others, there were the following, related to bots examples with Cortana, speech recognition and synthesis … there are not so many references online:

In this post I will give some more details related the TriviaBotSpeechSample in order to make it run and see most of the features related to speech recognition and synthesis. It is a very complex examples that make use of many features, like embed the bot in a custom UWP app, showing how to add the speech support in an app using the DirectLine channel. This channels allows to interact with the bot at a low level with specific APIs:  the library supports both UWP and C# applications in XAMARIN, to allow developers to include speech enabled conversations with bots across different platforms (native iOS, Android, Windows).

A good tutorial of that sample can be found in the bot framework portal (Speech Support in Bot Framework – Webchat to Directline, to Cortana) and the more relevant instructions to make it be deployed and see it work can be found in the instructions available in the GitHub repository of that solution. As said there, this sample contains two projects, a trivia bot (built on top of the Microsoft Bot Framework C# Builder SDK) and a UWP app (that talks to the bot using the Microsoft.Bot.Client NuGet package).

As this is a combined client/sample demo, there is a bit of setup required and in the following I will improve the available instructions available there, with some screenshots I did during the test I did together with a friend, … that possibly can help you! 😉

At the end , you can talk to the sample bot in multiple ways, all of which support speech input and output … I succeeded to make it run with all of them:

  1. Using the TriviaApp (UWP app) included in this sample and  simply hitting F5 in Visual Studio to run it.
  2. Using the bot framework emulator https://docs.microsoft.com/en-us/bot-framework/debug-bots-emulator.
  3. Host an instance of the Bot Framework WebChat client: https://aka.ms/BfWebChat
  4. Enabled the bot as a Cortana Skill. Simply enable the Cortana channel and provide an invocation phrase. Then make sure you are logged in to Cortana using the same Microsoft account, and say “Ask to start a game of trivia“. Cortana should trigger that bot!

========================================================================

First of all, be sure to update all libraries used in the solution, going in Visual Studio Tools -> NuGet Packet Manager -> Manage Nuget Packeges for Solution: otherwise, especially if you will use the latest emulator version, you will have the error shown in the following screenshot-.
Note that only one library (System.IdentityModel.Tokens.Jwt) cannot not be updated to the latest version because of some dependences that require a lower version of that package.

 

In order to  link the bot of the sample, it is convenient to choose the seccond options, otherwise it will be created and registered in Azure a new “basic” bot (and then you have to download the solution and substitute it with the one of the sample … so more wotk!).

If you are trying to create a  ID/password with your MSDN subscription done by your company, you will possibly receive the following error due to insufficient privilege: because I needed that codes only for a trial and not to publish a company app, I solved the problem generating them with my personal Microsoft account.

You need to set the bot handle in order it is unique, because the URL will be realoted to that name:

  • Enable the Direct Line channel in the bot settings page, Add a new site and paste a Direct Line secret in TriviaApp\BotConnection.cs file.

  • Reference to Bing recognition API, as requested by the code and related generated error.

  • Publish the Bot as an Azure App Service, and add the public endpoint (yourhosturl/api/messages) to the Bot Framework portal settings page. Make sure to use https instead of http in the url.

  • [Optional] To improve speech recognition for your bot: on http://dev.botframework.com/ go to the bot’s settings. In the “Speech recognition priming with LUIS” section you should see a list of LUIS apps associated with the account you are logged in with. Check the new LUIS app you created for this bot and hit save. This information is used to improve speech recognition when you speak to this bot and uses the Cognitive Speech apis for speech recognition. Speech recognition priming improves the recognition accuracy for the utterances and entities defined in your LUIS app for this bot.

 

 

 

 

 

 

 

 

 

 

Informazioni su Enzo Contini

Electronic engineer
Questa voce è stata pubblicata in Review e test, Windows, Windows Phone/Mobile. Contrassegna il permalink.

Lascia un Commento/Leave a comment

Inserisci i tuoi dati qui sotto o clicca su un'icona per effettuare l'accesso:

Logo WordPress.com

Stai commentando usando il tuo account WordPress.com. Chiudi sessione /  Modifica )

Google+ photo

Stai commentando usando il tuo account Google+. Chiudi sessione /  Modifica )

Foto Twitter

Stai commentando usando il tuo account Twitter. Chiudi sessione /  Modifica )

Foto di Facebook

Stai commentando usando il tuo account Facebook. Chiudi sessione /  Modifica )

Connessione a %s...

This site uses Akismet to reduce spam. Learn how your comment data is processed.