Monitor automated evaluation scores

Auto QM uses AI to evaluate conversations and provide scores for standard agent performance questions. The scores are automatically generated and available at both individual and aggregated levels. For further information about the Auto QM evaluation criteria, go to About the Auto QM evaluation criteria

Prerequisites

  • Your organization has an Analytics Enterprise or Analytics Enterprise Plus license.
  • You have the View Auto QM Dashboard permission.
  • (To view usage alerts for Auto QM) You have the View Auto QM Notification permission.
  • (To set and manage usage alerts for Auto QM) You have the Edit Auto QM Notification permission.
  • An administrator has created an Auto QM task (see Create Analytics tasks).

Page location

Analytics > Auto QM

Procedures

Navigate the dashboard

  1. Click Home.
  2. Select a form and click Apply.
  3. Click Date range
  4. Select a filter check box or manually select a date range and click Apply.
  5. Click Organization.
  6. Click Direction, select Inbound or Outbound, and click Apply. Click Reset all to reset the direction.
  7. If required, select a group, team, and agent, and click Apply.

  8. View the following metrics:

    • Average auto evaluation score - The overall average score of all interactions evaluated by Auto QM.
    • Total auto evaluated contacts - The total number of interactions evaluated by Auto QM.
    • Total auto evaluated agents - The number of unique agents whose interactions are evaluated by Auto QM.
    • Auto evaluation score trend - A graphical representation showing how auto evaluation scores have changed over time, helping identify improvements or declines in performance.
    • Auto evaluation sections - Specific categories within the auto evaluation (for example, empathy, professionalism, greeting, and so on), each assessing different aspects of agent performance.
    • Performance score chart - A visual representation (for example, a graph or bar chart) comparing the performance scores of agents, teams, or overall auto evaluation scores over time, facilitating easy identification of top performers, areas for improvement, and progress tracking.
  9. To view detailed information, click a group, team, or agent name.

View the evaluation metrics for a group

  1. Click Evaluation.
  2. On the Groups tab, select a Group filter.
  3. View the Evaluation score trend. Select View by year, View by month, View by week, or View by day.
  4. View Auto evaluation sections.
  5. Click a team name to view detailed information about the team metrics.

Navigate the evaluation metrics for a team

  1. Click Evaluation.
  2. On the Teams tab, select a Team filter.
  3. View the Evaluation score trend. Select View by year, View by month, View by week, or View by day.
  4. View Auto evaluation sections.
  5. Click an agent name to view detailed information about the agent metrics.

Navigate the evaluation metrics for an agent

  1. Click Evaluation.
  2. Click the Teams tab.
  3. Click an agent's name.
  4. View the Evaluation score trend. Select View by year, View by month, View by week, or View by day.
  5. View Auto evaluation sections.

  6. To view more information about the agent's conversation with the customer, click a contact ID. For further information, go to Evaluate a contact.

    NOTE   In the Evaluation tab, you can view the Auto evaluation scores and the Manual evaluation scores. To override a score generated by AI in the Auto evaluation tab, click Edit score, select Yes 100%, No 0%, or N/A, add a comment, and click Save score. This will also update the overall score. You can only use this feature if you have edit permissions.

View your monthly usage rate for Auto QM

Your organization has purchased a specific amount of usage for Analytics features like Auto QM that run on AI. If your organization exceeds this level, Calabrio will charge extra for the additional usage. You need the View Auto QM Notification permission to view the usage rate.

  • Click Notifications (the alert icon in the upper-right corner of the page). The Alerts panel opens. If an administrator has set up usage notifications (see Get notified about usage level for AI-based features), this panel contains details about how many contacts Analytics AI has processed in the current billing cycle.

Get notified about usage level for AI-based features

Your organization has purchased a specific amount of usage for Analytics features like Auto QM that run on AI. If your organization exceeds this amount, Calabrio will charge extra for the additional usage. Follow these steps to stay on top of your usage. You need the Edit Auto QM Notification permission to do this.

NOTE   The settings you enter here also apply to other AI-based Analytics features that your organization uses, such as Trending Topics and AI Tags. See Monitor Trending Topics and Monitor the trends in AI Tags.

  1. Click Configuration. The Configuration window opens.
  2. In the Monthly billing cycle section, select your contract start date.
  3. In the Monthly usage limit section, enter the number of interactions included in your license.

  4. In the Alert threshold field, enter the percentage of usage at which you want to be notified.

    EXAMPLE   You want to be notified when your organization reaches 90% of your contracted usage rate. You enter 90 in the Alert threshold field.

  5. Click Update.

Related Topics