About Bot Analytics KPIs

KPI	Metric to Analyze	Recommendation
BAS	Conversations handed off Difference between BAS and containment by topic Hand-off rate by response Negative feedback by response Negative sentiment False positive rate	By reviewing these metrics, you can identify instances where the bot fails to handle user queries effectively, leading to hand-offs, abandonment, negative sentiment, and feedback. This information helps to optimize the bot's training data, error handling, containment strategies, and intent recognition to improve user satisfaction and overall bot performance.
BES	Virtual agent repetition Customer paraphrase Negative sentiment Negative feedback Request to escalate multiple times	Analyzing these metrics can help you identify areas where the bot is struggling to provide effective responses or where it encounters negative user experiences, allowing you to refine the bot's training data, error handling, response text, and escalation protocols to enhance user satisfaction and bot performance.
Containment	VA Conversations Contained Conversations handed off Customer requested hand-off Automatic hand-off after intent-based trigger	By monitoring these metrics, you can identify patterns and trends that indicate successful containment or instances where conversations are escalated or handed off. You can use this information to optimize the bot's responses, train it to handle a wider range of user queries, and reduce the need for manual or automatic hand-offs, thereby improving the overall containment metric.
NLU	Customer messages understood False positive rate Customer messages with DYM Customer messages not understood Candidates for new intents Candidates for existing intents Out of Domain Under-trained intents Similar intents Similar utterances Intents with low quality utterances	Go to the Overview section in Bot Analytics to review these metrics. By analyzing the percentage of customer messages understood and the false-positive rate, you can refine the intent classification models for improved accuracy. Identifying customer messages with DYM assists in enhancing error handling and providing relevant suggestions. Metrics related to under-trained intents, similar intents, and intents with low-quality utterances guide you in optimizing training data, intent boundaries, and response quality, respectively, resulting in a more effective natural language understanding (NLU).

BAS

Conversations handed off
Difference between BAS and containment by topic
Hand-off rate by response
Negative feedback by response
Negative sentiment
False positive rate

By reviewing these metrics, you can identify instances where the bot fails to handle user queries effectively, leading to hand-offs, abandonment, negative sentiment, and feedback. This information helps to optimize the bot's training data, error handling, containment strategies, and intent recognition to improve user satisfaction and overall bot performance.

BES

Virtual agent repetition
Customer paraphrase
Negative sentiment
Negative feedback
Request to escalate multiple times

Analyzing these metrics can help you identify areas where the bot is struggling to provide effective responses or where it encounters negative user experiences, allowing you to refine the bot's training data, error handling, response text, and escalation protocols to enhance user satisfaction and bot performance.

Containment

VA Conversations Contained
Conversations handed off
Customer requested hand-off
Automatic hand-off after intent-based trigger

By monitoring these metrics, you can identify patterns and trends that indicate successful containment or instances where conversations are escalated or handed off. You can use this information to optimize the bot's responses, train it to handle a wider range of user queries, and reduce the need for manual or automatic hand-offs, thereby improving the overall containment metric.

NLU

Customer messages understood
False positive rate
Customer messages with DYM
Customer messages not understood
Candidates for new intents
Candidates for existing intents
Out of Domain
Under-trained intents
Similar intents
Similar utterances
Intents with low quality utterances

Go to the Overview section in Bot Analytics to review these metrics. By analyzing the percentage of customer messages understood and the false-positive rate, you can refine the intent classification models for improved accuracy. Identifying customer messages with DYM assists in enhancing error handling and providing relevant suggestions. Metrics related to under-trained intents, similar intents, and intents with low-quality utterances guide you in optimizing training data, intent boundaries, and response quality, respectively, resulting in a more effective natural language understanding (NLU).