About experience and automation

The Experience & Automation page provides a summary of how your virtual agent is doing in terms of automation, overall customer experience, volume, and cost per automated chat across a variety of date ranges, channels, language, and devices.

Prerequisites

  • You have a Bot Analytics license.
  • You have the View Experience Automation permission.

Page location

Bot Analytics > Home > Experience & Automation

Bot Experience Score (BES)

The Bot Experience Score (BES) is a key performance indicator (KPI) used to measure the user experience of the bot (chat or voice). The score takes into account the following signals in a conversation:

  • Bot repetition - The bot repeats itself for any reason during a conversation

  • User paraphrase - The customer uses a similar query twice or more in a conversation

  • Abandonment - The customer leaves in the middle of the conversation without reaching a legitimate end response configured on the bot

  • Negative sentiment - AI-based sentiment of conversation

  • Negative feedback - Explicit negative feedback received in the conversation

  • Profanity - Profanity present in the conversation

  • Request to escalate multiple times - The customer used the word “agent” or similar more than once in a conversation

    NOTE   If the customer uses the word “agent” once and is directly escalated, this is generally not a bad experience for the customer.

There is also a dial that displays your current BES score with colors to indicate a weak (red), medium (yellow), and strong (green) score.

Bot Automation Score (BAS)

The Bot Automation Score (BAS) is a KPI used to measure how effective the bot is at handling user problems. The score takes into account the following signals in a conversation:

  • Escalate to an agent

  • Abandonment - The customer leaves in the middle of the conversation without reaching a legitimate end response configured on the bot

  • False positive - The customer receives an unrelated response to their question

  • Negative feedback - Explicit negative feedback received in the conversation

  • Escalation requested not connected

There is also a dial that displays your current BAS score with colors to indicate a weak (red), medium (yellow), and strong (green) score.

For both BES and BAS, a conversation is evaluated based on the presence of negative signals with the following scoring system:

  • If there are no negative signals, the conversation receives a perfect score of 100

  • If there is one negative signal, the score decreases to 75

  • If there are two negative signals, the score decreases to 50

  • If there are three or more negative signals, the conversation is assigned a score of 0

All conversations are individually scored and these scores are then averaged to provide an overall assessment. BES and BAS are also available at the conversation topic level.

A metric signal breakdown for overall BES and BAS is included to give the user an understanding of which negative signals occur most frequently.

Cost per Automated Chat ($/AC)

This metric measures how much each automated conversation costs to the business.

The following table shows the topic-level KPIs of each conversation, namely volume, BES, and BAS along with the targets that you can set up for each of these KPIs. You can use a topic level breakdown to monitor the experience and automation score for each topic. You can view the top 50 conversation topics.

NOTE   The conversation volume on the Experience & Automation page differs from the volume on the Conversation Topics page. The Conversation Topics page includes more conversations, including immediately escalated conversations.