Scorecards

A scorecard is an assessment template that defines which criteria are checked in the quality assessment of a work item. It forms the basis for every quality assessment in the system.

Scorecards are created and maintained by quality management. As soon as a scorecard is active, new work items can be linked to it automatically if the configured assignment rules apply.

Scope and Permissions

Scorecards are managed under Settings → User Management → Quality Scorecards. The page itself is headed "Quality Scorecards".

Access to scorecards requires the permission "Manage quality scorecards". It allows viewing, creating and editing scorecards. Deleting additionally requires the permission "Delete quality scorecards".

Anyone allowed to view an assessment sees the categories and criteria of the underlying scorecard in it, but cannot open or edit the scorecard itself.

Structure

A scorecard consists of categories, which in turn contain criteria.

Categories group related criteria. Each category has a label and an order.

Criteria define the actual assessment points within a category:

• Label and description – What is being assessed?
• Maximum points – weighting of the criterion
• Scoring type – Met / Not met or Numeric scale
• Auto-generate by AI – whether the AI assesses this criterion automatically
• Assessment prompt for AI – instruction to the AI on which standards to apply
• Make-or-break for category (knockout criterion) – if this criterion scores 0, the category is set to 0
• Make-or-break for assessment (knockout criterion) – if this criterion scores 0, the overall assessment is set to 0

Assignment and Response Context

Scorecards can be restricted to specific work items. Assignment works via:

• Application by team – only work items of members of specific teams
• Application by ticket tag – only tickets with specific tags
• Application by channel – e.g. email, phone or letter
• Application by customer contact – whether the scorecard is meant for replies to customer messages or for proactive replies

For customer contact, two cases are distinguished:

• After customer message – classic reactive quality assurance: an agent answers a previous customer request. New scorecards use this setting by default.
• Without prior customer message (proactive) – for proactive communication, where the agent writes a message without a direct previous customer request.

If several assignment rules are configured, all of them must apply. A rule left empty does not restrict anything – for example, if no team is selected, the scorecard applies to all teams.

Work items can also be excluded explicitly:

• Exclusion by team
• Exclusion by ticket tag
• Exclusion by channel

If an exclusion rule applies, the work item is not assessed with this scorecard – even if all other rules match.

There is no sampling: every work item the rules apply to is assessed.

Automatic Creation of Assessments

Every minute, Enneo automatically checks whether there are new work items that have not been assessed yet. For each matching work item, it determines which active scorecards apply according to the assignment rules.

For each matching scorecard, an assessment is created, which the AI then pre-assesses.

A work item can be assessed by several scorecards if several scorecards apply.

Calls: Calls handled by an agent are also assessed automatically as soon as an active scorecard applies. The agent who was on the call when it ended is assessed. If the call was transferred between agents, only the last agent is assessed; previous agents on the same call do not receive an assessment. Agents whose phone only rang without them picking up are not assessed. The assessment does not appear immediately when the call ends, but shortly after the agent has left the ticket and their work item is complete.

Live Quality Coach

The Live Quality Coach option determines whether a scorecard is also used during ticket handling.

If the Live Quality Coach is enabled, an agent can have the current reply draft checked before sending. To do so, they choose Run Quality Check in the menu next to the send button. The AI assesses the draft against the AI-assessable criteria of the matching scorecards.

For each matching scorecard, the Live Quality Coach returns:

• the percentage achieved
• the required minimum value
• whether the draft passed
• an AI summary
• details per category and criterion

The Live Quality Coach is available for the reply channels email and letter.

With the permission "Promote Live Quality Coach as primary action", Run Quality Check becomes the main action of the send button instead of sitting in the menu. This suits less experienced agents who should check their replies before sending. By default, no role has this permission.

Minimum Value for the Live Quality Coach

In the field Threshold (%), a minimum value can be set for the Live Quality Coach. This value lies between 0 and 100 percent and defines the minimum score a reply draft should reach.

The default value is 85 %.

If a draft does not reach the minimum value, this is shown in the reply area. The reply can still be sent: the send button then reads Send anyway, making it clear that the agent is deliberately skipping the quality check.

Scorecards as a Guideline for AI Replies

Scorecards do not only assess replies afterwards – they can also guide the AI while it writes. For this, every AI agent has the switch Follow Quality Guidelines from QA Scorecards under Instructions. It is switched on for newly created AI agents.

If the switch is on, the AI receives the AI-assessable criteria of the matching scorecards when writing a reply and aligns the reply with them. Which scorecards match is determined by the same assignment and exclusion rules as for assessments. At most three matching scorecards are taken into account.

AI Pre-Assessment

Criteria with Auto-generate by AI enabled are assessed automatically by the AI. The AI follows the Assessment prompt for AI.

The result can be reviewed and adjusted by the supervisor.

Criteria without AI assessment initially remain unscored and must be assessed manually. The Live Quality Coach only uses AI-assessable criteria for its automatic check.

Knockout Criteria

Knockout criteria make it possible to mark individual requirements as absolute.

• Category level: If a knockout criterion within a category scores 0, the whole category is scored 0 points.
• Assessment level: If a knockout criterion for the overall assessment scores 0, the overall assessment drops to 0.

Versioning

When an active scorecard is changed, Enneo automatically creates a new version. The previous version is retired, but stays linked to the assessments created with it.

Existing assessments remain linked to the scorecard version that was active when they were created. Changes only affect newly created assessments.

Scorecard States

• Draft – being edited, not yet active
• Active – active and used for new assessments
• Retired – replaced by a newer version
• Deleted – deleted and no longer visible

In the form, only Draft and Active can be selected; Enneo sets Retired automatically as soon as a new version is created.

Assessment Workflow

The states an assessment goes through are described under Quality Assessment.

Quality Scorecard Preview

The Quality Scorecard Preview lets you test a scorecard before it goes live or before changes take effect.

Typically you need:

• Ticket ID – provides context such as the customer request, history and relevant ticket data
• Answer Text – the draft or sample text to be checked

The preview does not create a regular assessment in the quality workflow. It is used to validate criteria, assessment prompts, weightings, knockout rules and Live Coach behaviour.

Export and Import

When editing a scorecard, the current scorecard configuration can be exported and copied as JSON via the Export button. This is useful for reviews, technical analysis or comparing environments.

Conversely, a scorecard can be taken over via the Import button in the scorecard list, for example from another environment: paste the exported JSON configuration and click Create.

Best Practices

• Phrase criteria clearly: AI-assessable criteria should be concrete and checkable.
• Think of criteria as writing instructions: Phrase criteria so that they also work as instructions for the AI when it writes replies.
• Keep assessment prompts precise: Describe how the AI can tell whether a criterion is met.
• Use knockout criteria sparingly: They have a strong effect on category and overall results.
• Choose the threshold deliberately: If the minimum value is too high, many drafts fail the check; if it is too low, the Live Quality Coach loses its effect.
• Separate proactive and reactive cases: Different scorecards can make sense for replies to customer messages and for proactive communication.
• Configure assignment rules carefully: Broad assignments increase the number of assessments.
• Keep versioning in mind: For analyses, it matters which scorecard version an assessment is based on.