Contact Us

If you still have questions or prefer to get help directly from an agent, please submit a request.
We’ll get back to you as soon as possible.

Please fill out the contact form below and we will reply as soon as possible.

  • Go to Haptik Website
  • Contact Us
  • Home
  • Bot Building
  • Testing

What is bot functional testing?

Written by Medha Anand

Updated on October 26th, 2021

Contact Us

If you still have questions or prefer to get help directly from an agent, please submit a request.
We’ll get back to you as soon as possible.

Please fill out the contact form below and we will reply as soon as possible.

  • Getting Started
    Build Deploy Analyse Manage Account Bot Deactivation
  • Bot Building
    Essentials Smart Skills Steps User Messages Bot Responses Entities Connections Customisations User feedback collection Testing Whatsapp Bots NLU Bot Maintenance
  • Smart Agent Chat
    Set up Admin Settings MyChats Section (Agent Inbox) Live Traffic Section Teams Section Archives Section Analytics Plans on Smart Agent Chat
  • Conversation Design
    Design Basics Design Guides Designing for Platforms Designing WhatsApp Bots
  • Developer Guides
    Code Step Integration Static Step Integration Shopify Integration SETU Integration Exotel Integration CIBIL integration Freshdesk KMS Integration PayU Integration Zendesk Guide Integration Twilio Integration Razorpay Integration LeadSquared Integration USU(Unymira) Integration Helo(VivaConnect) Integration Salesforce KMS Integration Stripe Integration PayPal Integration CleverTap Integration Fynd Integration HubSpot Integration Magento Integration WooCommerce Integration Microsoft Dynamics 365 Integration
  • Deployment
    Web SDK WhatsApp Facebook Instagram Sunshine Conversation LINE Google Business Messages Telegram MS Teams Bot as an API iOS SDK Android SDK
  • External Agent Tool Setup
    Zendesk Chat Salesforce Service Cloud Freshchat Zoho NICE CXOne Gorgias
  • Analytics & Reporting
    Intelligent Analytics
  • Notifications
    SMS Notifications Success Measurement
  • Commerce Plus
    Catalog Integration Bot Building Guide Channel Deployments Unified ML Pipeline Documentation
  • Troubleshooting Guides
    Error Messages FAQs
  • Release Notes
+ More

Table of Contents

All functional flows verification (as per IVA design document):API testing and its alignment with requirements in IVA:Small talk verification:FAQs verification (if FAQs are present):Error handling: API error, invalid inputs, Bot break, and reprompts:UI elements and Content VerificationOutro flows/follow up messages:Negative User Messages handling:For multilingual IVAs (if enabled):Chat agent functionality (if enabled):Preparing Test cases

After testing the Training data, you can now conduct full Functional testing of the bot. This phase will ensure if the bot is ready to handle different types of user queries, whether the bot is taking the user through the right conversational flow and if the bot is able to handle out-of-scope queries from the users.

The following points elaborate on different components which are required to be tested.

All functional flows verification (as per IVA design document):

Testing all the functional flows will ensure if the bot is giving correct responses to the user. It will also ensure if the users are going through the journey as designed during the Conversation Design phase.

  • Test all the positive and negative flows as per the design.
  • Identify and write different scenarios before test execution.
  • Execution of scenarios, and issues fixes with retesting.
  • Prepare a sheet for all the test cases you are testing. Find a sample sheet at the end of the document.

API testing and its alignment with requirements in IVA:

All the APIs should be tested to ensure users are getting responses and the APIs is giving timely responses.

  • Verify the API responses are as per requirements.
  • Verify API errors.
  • Verify API response time (should be no more than 20 seconds).
  • Verification of different APIs responses handling on the IVA.
  • Verify the responses for different types of test data.

Small talk verification:

Small Talk is a module present for all the bots. Small Talk responds to random questions asked by users such as "Are you a robot?"

  • IVA should reply correctly for small talk (if it is enabled).

  • Verify the negative intents for Small talk as well.

    Example: You are useless.

  • Ask the question such as ‘Who made you?’ It should not give Haptik branding. Small talk customized messages are recommended.

FAQs verification (if FAQs are present):

  • All the FAQs should be answered correctly by IVA.
  • Each step should have a specific intent, avoid similar intents on multiple steps.
  • Ask questions with 3-4 variations for the same intent.

Error handling: API error, invalid inputs, Bot break, and reprompts:

As the bot may break on different scenarios, the users should receive a prompt reply from the bot regarding the error.

  • Bot break message should be displayed for invalid user inputs.

  • Out of scope queries should be handled by giving proper guidance through bot break or outro messages, so as to bring the user back in the bot journey.

  • APIs error messages should be handled by giving an appropriate response to the user.

  • The entity reprompt message should be relevant, along with hint text; so as to guide the user about what is expected

    Eg. Entity bot response: What's your phone number?

    Entity retry message: That seems incorrect. Please enter a valid 10-digit phone number.

    Entity bot response: What's your date of birth?

    Entity retry message: That doesn't seem like a date. Please enter a valid date between 18/5/1990 and 26/11/2019.

Users should not get the same message every time the bot throws any error. Check the variations added in as bot responses.

UI elements and Content Verification

  • Verify the functioning of various UI elements such as Buttons, QRs, Carousel, Forms, Webviews, Location Picker, and so on.
  • Hyperlinks should be clickable and should point to the correct endpoints.
  • Verify if the bot persona is in line with the company logo and branding.
  • Verify the tone used in the bot is as per design i.e causal, formal, friendly, funny.
  • Check the grammar and spelling along with punctuations.
  • Use of correct emojis at correct places and bullet points
Carousel Text and Buttons Location
Text and Quick Replies Menu Chatform

Outro flows/follow up messages:

The journey should never end abruptly at any juncture unless the user abandons the conversation midway. If any Bot Response does not contain QRs, buttons, or a follow-up message, it is considered abrupt.

  • Check for follow-up messages based on the use case/journey.

    Example: Was I able to resolve your payment issue?

  • Check if there is a generic follow-up message to re-engage the user.

    Example: Wait at the end of a journey (20-30 Seconds).

    A normal follow-up message should appear.

    Example: Still looking for chairs? Tell me how many you need.

    Support email or contact number should be present in the bot whenever the user asks for further assistance from the support team.

Negative User Messages handling:

Test variations with negative tone within the bot flow.

  • The IVA should be able to handle negative cases and journeys that are not linear. The goal should be to channel the user back into the flow.

    Example: I don't remember account number I didn't receive OTP I did not get any email

For multilingual IVAs (if enabled):

  • Functional flows should work same as it works for the primary language.
  • Adding a different language should not impact English language flow.
  • Bot break message language should be checked.

Chat agent functionality (if enabled):

  • Check if the chat is transferred to the agent on bot break or upon user messages.
  • Check if the chat gets assigned to an agent and if you are getting responses on the bot.
  • Check if the agent is able to close the chat.

Preparing Test cases

Test cases preparation guidelines and sample template:

  • All the positive functional flows of IVA
    • Verify all possible flows in IVA
    • Verify all the bot’s invocation points
    • Verify UI elements like carousels, buttons, quick replies, chat elements, dropdowns, location pickers, attachments, web views, forms, message copies, emojis(WhatsApp IVA)
    • Verify user authentication e.g: OTP, PAN, etc
  • Negative cases and validations
    • Bot break scenarios (when out of scope query is asked or invalid inputs are given by the user)
    • Verify the outro messages are clear and meaningful to the user

The following is a file containing sample test cases, click here to download.

Was this article helpful?

Yes
No
Give feedback about this article

Related Articles

  • How to make use of Training Suggestions?
  • How to test the bot?
  • How to use bot QA tool?
  • How to debug the bot?

Platform

  • Conversation Studio
  • Smart Skills
  • Advanced NLU
  • Intelligent Analytics
  • Omnichannel
  • Smart Agent Chat
  • Enterprise Security
  • Integrations

Solutions

  • Conversational Commerce
  • Lead Generation
  • Customer Care
  • WhatsApp
  • Conversational IVR
  • Google Business Messages

Industries

  • Retail/ E-Commerce
  • Financial Services
  • Travel & Hospitality
  • Telecom

Knowledge

  • ROI Calculator
  • Reports & Research
  • Case Studies
  • Webinars
  • ISAT
  • Tech Blog
  • Business Blog
  • Resources
  • Haptik v/s Yellow
  • Haptik v/s Liveperson
  • Haptik v/s IBM Watson
  • Haptik v/s Verloop
  • Conversations on AI

Company

  • Why Haptik
  • About Us
  • Careers
  • News & Media
  • Awards & Recognition
  • Contact Us
  • Partnerships
  • Investor Relations

Subscribe

Sign up to recieve the latest updates

Find us on

  • Twitter-footer
  • Linkedin-footer
  • YT-footer
  • Insta-footer
  • G2-footer
  • Facebook-footer

Knowledge Base Software powered by Helpjuice

Copyright © jio Haptik Technology Limited 2021 | Data Security & Privacy Policy | GDPR

North America | Asia Pacific | Africa | enterprise@haptik.ai

Definition by Author

0
0