About Bot Analytics KPIs

KPI Metric to Analyze Recommendation
BAS
  • Conversations handed off

  • Difference between BAS and containment by topic

  • Hand-off rate by response

  • Negative feedback by response

  • Negative sentiment

  • False positive rate

By reviewing these metrics, you can identify instances where the bot fails to handle user queries effectively, leading to hand-offs, abandonment, negative sentiment, and feedback. This information helps to optimize the bot's training data, error handling, containment strategies, and intent recognition to improve user satisfaction and overall bot performance.
BES
  • Virtual agent repetition

  • Customer paraphrase

  • Negative sentiment

  • Negative feedback

  • Request to escalate multiple times

Analyzing these metrics can help you identify areas where the bot is struggling to provide effective responses or where it encounters negative user experiences, allowing you to refine the bot's training data, error handling, response text, and escalation protocols to enhance user satisfaction and bot performance.
Containment
  • VA Conversations Contained

  • Conversations handed off

  • Customer requested hand-off

  • Automatic hand-off after intent-based trigger

By monitoring these metrics, you can identify patterns and trends that indicate successful containment or instances where conversations are escalated or handed off. You can use this information to optimize the bot's responses, train it to handle a wider range of user queries, and reduce the need for manual or automatic hand-offs, thereby improving the overall containment metric.
NLU
  • Customer messages understood

  • False positive rate

  • Customer messages with DYM

  • Customer messages not understood

  • Candidates for new intents

  • Candidates for existing intents

  • Out of Domain

  • Under-trained intents

  • Similar intents

  • Similar utterances

  • Intents with low quality utterances

Go to the Overview section in Bot Analytics to review these metrics. By analyzing the percentage of customer messages understood and the false-positive rate, you can refine the intent classification models for improved accuracy. Identifying customer messages with DYM assists in enhancing error handling and providing relevant suggestions. Metrics related to under-trained intents, similar intents, and intents with low-quality utterances guide you in optimizing training data, intent boundaries, and response quality, respectively, resulting in a more effective natural language understanding (NLU).