Scorecards

A scorecard is an evaluation template specifying the criteria to be examined in the quality evaluation of a work process. It forms the basis for every quality evaluation (assessment) in the system. Scorecards are created and maintained by the quality management team. As soon as a scorecard is active, new work processes can automatically be linked to it if the configured assignment rules apply.

​

Scope and Permissions

Scorecards are managed under Settings → Quality Scorecards. The following privileges apply for access:

• View: All registered users can view scorecards.
• Create and edit: Requires the qualityManageScorecards permission.
• Delete: Additionally requires the qualityDeleteScorecard permission.

​

Structure

A scorecard consists of categories, which in turn contain criteria. Categories group together related criteria. Each category has a name and an order. Criteria define the actual evaluation points within a category:

• Name and description - What is being evaluated?
• Maximum points - Weighting of the criterion
• Evaluation type - metNotMet or numericScale
• AI evaluation enabled - Whether AI evaluates this criterion automatically
• Evaluation prompt - Instructions to the AI on which standards to evaluate
• Knockout criterion for category - If this criterion is scored 0, the category is set to 0
• Knockout criterion for evaluation - If this criterion is scored 0, the overall score is set to 0

​

Allocation and Response Context

Scorecards can be restricted to certain work processes. Assignment is based on:

• Teams - only work processes by members of specific teams
• Ticket tags - only tickets with specific tags
• Channels - e.g., Email, phone, or letter
• Response context - whether the scorecard is intended for responses to customer messages or for proactive responses

Two cases are distinguished in the response context:

• After customer message (afterCustomerMessage) - classic reactive quality assurance: an employee responds to a previous customer inquiry.
• Without prior customer message / proactive (withoutCustomerMessage) - for proactive communication, where the employee creates a message without direct previous customer inquiry.

If multiple allocation rules are configured, they need to apply collectively.

​

Automatic Creation of Assessments

The system regularly checks whether there are new work routines that have not yet been evaluated. For each matching work process, it determines which active scorecards apply according to assignment rules. For each matching scorecard, an assessment is created and submitted for AI processing. A work process can be evaluated by multiple scorecards if multiple scorecards apply.

​

Live Quality Coach

The Live Quality Coach option determines whether a scorecard is also used while handling the ticket. If the Live Quality Coach is enabled, an employee can have the current draft response checked before sending. The AI evaluates the draft based on the AI-evaluable criteria of the matching scorecards. For each matching scorecard, the Live Quality Coach provides:

• the achieved percentage score
• the required minimum score
• whether the draft has passed
• an AI summary
• details per category and criterion

The Live Quality Coach is currently planned for supported response channels such as Email and letter.

​

Minimum Value for the Live Quality Coach

A Threshold can be configured for the Live Quality Coach. This value is between 0 and 100 percent and defines the minimum score a draft response should achieve. The default value is 85%. If a draft does not reach the minimum value, this is made visible in the response area. The response can still be sent, but the interface makes it clear that the employee is knowingly skipping the quality check.

​

AI-assisted Preliminary Scoring

Criteria with AI evaluation enabled are automatically evaluated by the AI. The AI follows the configured evaluation prompt. The result can be reviewed and adjusted by the supervisor. Criteria without AI evaluation remain initially unscored and must be manually scored. In the Live Quality Coach, only AI-evaluable criteria are used for the automatic check.

​

Knockout Criteria

Knockout criteria allow marking individual requirements as absolute.

• Category level: If a knockout criterion within a category is scored 0, the entire category is scored with 0 points.
• Evaluation level: If a knockout criterion for the overall evaluation is scored 0, the overall evaluation falls to 0.

​

Versioning

Scorecards are versioned. Each scorecard has a baseId and a revision.

• baseId remains the same across all versions of a scorecard.
• revision increases with each new version.
• If an active scorecard is edited, the previous version is set to retired, and a new revision is created.

Existing assessments remain linked with the scorecard version that was active at the time of creation. Changes only affect newly created assessments.

​

States of a scorecard

• Draft – in progress, not yet active
• Active – active and usable for new evaluations
• Archived – superseded by a new revision
• Deleted – soft-deleted and no longer visible

​

Evaluation Workflow

Assessments can go through the following states:

1. Unprocessed – Assessment was created, AI processing pending
2. AI processing in progress – AI is currently evaluating the criteria
3. AI evaluation ready – AI pre-evaluation completed, ready for review
4. Review in progress – Supervisor has started the review
5. Reviewed by Supervisor – Review completed
6. Discussed with employee – Assessment has been discussed with the evaluated employee
7. Error – An error occurred during processing
8. Deleted – Assessment was deleted

If a supervisor makes changes to a completed assessment, the status automatically reverts to reviewOngoing.

​

Quality Evaluation Preview

The Quality Evaluation Preview can be used to test a scorecard before it is used in production or after changes are actively applied. Typically required for this are:

• Ticket ID – provides context such as customer inquiry, history, and relevant ticket data
• Response text – the draft or sample text to be checked

The preview does not create a regular evaluation in the quality workflow. It is used to validate criteria, prompts, weights, knockout rules, and live coach behavior.

​

Export

When editing a Scorecard, the current Scorecard configuration can be exported and copied as JSON. This is useful for reviews, technical analysis, or sync between environments.

​

Best Practices

• Clearly formulate criteria: AI-evaluable criteria should be concrete and verifiable.- Maintain precise prompts: Describe how the AI should identify whether a criterion is met.
• Use knock-out criteria sparingly: They have strong effects on category and overall results.
• Deliberately choose thresholds: Too high a minimum value can block many drafts, while a lower value weakens Live Quality Coach.
• Separate proactive and reactive cases: Different scorecards may be useful for responses after customer messages and proactive communication.
• Configure assignment rules intentionally: Broad assignments increase the assessment volume.
• Take versioning into account: For evaluations, it is relevant which revision of an assessment is the basis.