If you have a bot deployed on the WhatsApp channel, then I am sure you would know all the capabilities that your WhatsApp bot has, be it media exchange or text conversations. But apart from these, there is also one more feature that your users can use while conversing with your bot, and that is the WhatsApp Voice Notes.

What are WhatsApp Voice Notes?

WhatsApp Voice Notes is a way to send messages in audio format. Rather than typing it out, users tap on the mic icon on the composer area of your conversation screen and record your message. Once you release the mic icon, your message will be sent. 

Who can use WhatsApp Voice Notes?

Once you have made your WhatsApp bot live for your users, they will be able to use this feature. With this feature, they will be able to send a voice note to the bot asking queries, in place of sending a text message. This will definitely save their time, and that will have a positive effect on the feedback you receive from them on your bot. 

How does this work on the bot?

In this scenario, when the user sends the voice note to the bot, the bot will understand the intent of the query and will send an appropriate response to the user. The bot will respond back in its usual manner as if the user had typed out the same message. Voice Notes thus are just a more convenient way for users to send their queries to the bot. They do not influence the bot’s behavior or response in any way. 

We use ASR(Automatic Speech Recognition) technology, wherein the voice note when received from the user is sent to ASR which converts the voice note into a text format.

Voice Notes are just an input mechanism, wherein the users can send their queries in the form of a voice note rather than typing it out in the composer. For the bot, it makes no difference, whether the input was a voice note or a text. It will understand the intent, and respond back in a similar fashion. 

Also, this does not change bot building at all. When our pipeline receives a voice note from the user, it simply converts the voice note to text before sending it for further processing. Thus, from the bot's perspective, it simply receives a text message as if the user had typed it out.

Which languages will the bot be able to understand?

The users can send a voice note in English, Spanish, and many other languages. Our bots will understand these voice messages and will respond back to the users accordingly. 

However, do note that we always assume the language the user has spoken in (in the Voice Note) to be the same as the one in which they are currently chatting with the bot. What this means is, whenever the user sends a voice note, the bot will not detect the language automatically. Instead, it will assume that the user is speaking in the same language, as the one currently set for the bot. If the current language of the bot is set to Spanish, then the bot will assume that the user is speaking in Spanish, and it will convert speech to text for extracting the intent, and for providing a response. 

It might happen that the current language of the bot is set to Spanish, and the voice note sent by the user is in English, or in some language other than Spanish. In this case, the ASR will fail and it won't be able to convert the voice note into text. It will send a message to the user to send the voice note again. This is better than the ASR converting the voice note into text incorrectly which might lead to a Bot Break.

Why do we not support language auto-detection on Voice notes?

As we use ASR(Automatic Speech Recognition), it expects us to tell what language the voice note is spoken in before it can convert the voice note to text. So, when we send the user's voice note over to the ASR service, we will have to tell it that this is a Spanish voice note, or English voice note, and so on. 

How to enable this feature on your WhatsApp bot?

In order to enable WhatsApp Voice Notes for your bot, you will have to contact our Support Team at support@haptik.ai, and they will help you in this enablement.