Optimize bot responses best practice

You have released a new bot response and you want to measure its success. There are some best practices for optimizing bot responses.

Prerequisites

  • You have a Bot Analytics license.

  • You have the View VAA Response Performance permission.

  • You have the View VAA Performance Comparison permission.

  • This feature relies on the responseID metadata value. You cannot use this feature if Calabrio ONE does not receive responseID metadata from your AI platform.

NOTE   Salesforce Einstein and Microsoft PVA integrations are not compatible with the Response Performance page.

Page location

Bot Analytics > Virtual Agent Analytics > Response Performance

Bot Analytics > Virtual Agent Analytics > Intent Performance

Procedure

  1. To track your bot responses, go to Response Performance, find the response, and view the number of times it is presented, priority, handed-off, bot repetitions, and more.

  2. View the number of issues (for example, repetition, escalation, or negative feedback) on the response. You can see that a priority is already applied based on the number of flagged issues.

  3. Depending on what type of release occurred (for example, updated bot response, new bot response, or enhanced experience changes), you can derive a recommendation based on the issues that are present.

  4. On the Intent Performance page, view the performance of the intent that leads to the response. The issues section monitors similar signals to those on the Response Performance page, but at the intent level. The two main issues monitored at the intent level are false positives and bot repetitions.

    NOTE   When you review the Response Performance page, you might observe a significant increase in user escalations and be unsure about the reason behind this trend, considering that you have updated the response to provide the necessary information for resolving user issues on the given topic. If this happens, Calabrio recommends navigating to the intent performance page. You might notice an unusually high false positive rate associated with the intent corresponding to the response. This indicates that, although the response content might be adequate, there is an issue with the intent training. Users who do not intend to see this response are inadvertently encountering it, leading to escalations.

  5. If the response you have updated or released has high volume and high expectations to move the needle on some overall KPIs, you must use the performance comparison tab to review key KPIs for the day before the release, along with the day after. You can also apply months, quarterly, or yearly time frame comparisons.

    NOTE   This image shows high level metrics. An experience must have a high enough volume to see an overall impact in this view.