Create forms for AI evaluation

You can use the Auto QM Evaluation Form Manager to create your own AI evaluation forms. Auto QM uses these forms to score contacts. You can create as many forms as you wish, but only the latest three forms are active for each interaction, even if more than three forms are assigned to the selected agent, groups, or teams. In the forms, you can configure the threshold for each metadata-based question.

If your organization has an Analytics Premium license, you can also add your own custom questions to an evaluation form. This feature allows you to enhance the form in the following ways:

Create your own questions and group them by section.
Choose your own question names, descriptions, and answers.
Provide an example for each answer.

The customized questions are added to a question library. Then, when you create an evaluation form, you can choose custom questions from the library.

Prerequisites

Your organization has the Analytics Enterprise license.
To manage custom questions, your organization must have the Analytics Premium license.
You have the Edit Auto QM permission.

Page location

Application Management > Auto QM Configuration > Auto QM Evaluation Form Manager

Procedures

Step 1: Create a new form

Click New form.
Enter the form name in the Form name field.
Enter a description in the Form description field. The character limit is 3,000.
(Optional) Select the Enable target score check box.
1. Toggle the slider to set the target scoring range for the agent performance on the evaluation form.
2. Enter the names of the scoring ranges.
3. Select or clear the Show scoring range name on evaluation form.
(Optional) If you want to manage section and question percentage weighting manually, select the Manual weighting check box. If you want Calabrio ONE to calculate weighting automatically and evenly, skip this step.

NOTE The total weighting of the sections and questions must be 100%.

Step 2: Add sections and questions

NOTE You can view a Advice for sections and question preview on the right side of the page.

Click Add section. You can create up to five sections within each form.
Click the pen icon to edit the section name.
Enter a number in the Section target % field. This is the target score that the company wants the agent to achieve at a section level. If you have enabled manual weighting, you must also enter a number in the Section weight % field .To delete a section, click the three dots icon and then click Delete section.
Click Add Question. You can view the drop-down list of all AI scored questions and select up to five questions for each section.
Select a category from the Question category drop-down list.
Select a question from the Question drop-down list.

NOTE For more information about evaluation criteria, go to About the Auto QM evaluation criteria. For information about how to create custom questions, go to Create a custom question.
(Optional) If you enabled manual weighting, enter a number in the Question weight % field.
(Optional) Use intelligent phrase matching to provide examples of specific phrases that you want to associate with one of the preset questions that you already selected through the Question category and Question drop-down lists. If the agent and customer interaction contains these phrases, it has an effect on the overall score.
1. Select Intelligent phrase matching from the Question category drop-down list.
2. Enter the previously selected question in the Question field.
3. Enter a description in the Question description field. For further information, see Advice for question descriptions.
4. Select AI match or exact match from the Match type drop-down list. The AI match uses AI to identify similar phrases used during the interaction, even if they do not exactly match the given phrase. The exact match identifies an exact match for the given phrase.
5. Select Agent, Customer, or Any from the Sent by drop-down list.
6. In the Phrases field, enter phrases related to one of your selected questions. Auto QM will automatically identify these phrases in conversations. There is a limit of 50 characters for each phrase.
(Optional) To delete a question, click the three dots icon and then click Delete questions.
Enter a description in the Question description field. The character limit is 1,000.
(Optional) Select the KPI failure check box and select Form or Section. This allows you to configure KPI-based failures at the form or section level. If the agent does not achieve the appropriate percentage, they automatically fail the section or form and will be assigned a score of zero.
Where necessary, add time thresholds, hold events, and so on for specific questions.
Continue to add questions and sections until the form is finished.
(Optional) If you enabled manual weighting, click Balance question weight to give all questions the same weight. or click Balance section weight to give all sections the same weight.
(Optional) In the Additional AI evaluation instructions field, enter additional descriptions specific to your contact center that could be helpful in the evaluation process. This information will allow Auto QM to gain more knowledge and better understand your agent conversations. For example, you can add business jargon, abbreviations, or acronyms. The character limit is 3,000.
(Optional) To save a draft, click Save draft.
Click Publish to publish the draft or click Delete and click Yes to delete the draft.
To archive a form, click Archive and then click Yes. To close a form, click the X icon.

NOTE If a form is included in a running task, you cannot edit, delete, or archive the form.

Manage existing evaluation forms

On the Auto QM Evaluation Form Manager page, you can complete the following actions:
- To view a form, click a magnifying glass icon.
- To edit a draft form, click the pencil icon, edit the information, and click Save draft or Publish.
- To delete a form, click the trash can icon and then click Yes. Alternatively, click the pencil icon, click Delete, and then click Yes.
- To clone a form, click the clone icon, enter a name for the form, and click Save. If you change your mind about cloning a form, click the trash can icon.
- To publish a form, click the arrow icon.
- To archive a form, click the cabinet icon and then click Yes.

Create a custom question

You can create custom questions tailored to your business, including options, descriptions, and scoring criteria, which can be shared across evaluations.

On the Auto QM Evaluation Form Manager page, click Manage custom questions.
Click New question.
Enter the Advice for questions name in the Question name field. The character limit is 100 for the question name.
Enter a Advice for question descriptions in the Question description field. The character limit is 1,000.
In the Options section, enter at least two Advice for options for the question and then enter an Advice for option descriptions for each answer to help the AI evaluate responses more accurately. To change the order of the options, click the arrow icons. The character limit for the option name is 30, and the limit for the option description is 500. You can also include N/A as an option for each question. Click Add options to add more options.
Click Publish. Alternatively, click Save draft, if you want to save a draft and publish it later.

IMPORTANT After you publish a question, you cannot edit it.

Manage existing custom questions

On the Manage custom questions page, you can complete the following actions:
- To view a custom question, click the magnifying glass icon.
- To edit a draft custom question, click the pencil icon, edit the information, and click Save draft or Publish.
- To delete a custom question, click the trash can icon and then click Yes. Alternatively, click the pencil icon, click Delete, and then click Yes.
- To clone a custom question, click the clone icon, enter a name for the question, and click Save. If you change your mind about cloning the question, click the trash can icon.
- To publish a custom question, click the arrow icon.
- To archive a custom question, click the cabinet icon and then click Yes.