Scorecards

Documentation Index

Fetch the complete documentation index at: https://docs.enneo.ai/llms.txt

Use this file to discover all available pages before exploring further.

A scorecard is an evaluation template that specifies the criteria used in the quality assessment of a work process. It forms the basis for each quality assessment (Assessment) in the system. Scorecards are created and maintained by the quality management. As soon as a scorecard is active, new work processes (Worklogs) are automatically linked with it, provided the configured assignment rules apply.

​

Area and Permissions

Scorecards are managed under Settings → Quality Scorecards. The following permissions apply:

• View: All logged-in users can view scorecards – no special permission required.
• Create and Edit: Requires the qualityManageScorecards permission.
• Delete: Requires the additional qualityDeleteScorecard permission.

Permissions are granted via user roles. Usually, the management of scorecards lies with the quality management or administrators.

​

Structure

A scorecard consists of categories which in turn contain criteria. This hierarchy allows for thematic grouping of evaluation points. Categories group together criteria that are related in content (e.g., “Communication”, “Process Compliance”). Each category has a freely selectable designation and sequence. Criteria define the actual evaluation points within a category:

• Designation and Description – What is being assessed?
• Maximum points – Weighting of the criterion
• Assessment type – metNotMet (met / not met, binary) or numericScale (numerical scale from 0 to maximum value)
• AI review activated – Whether or not the AI automatically pre-assesses this criterion
• Review prompt – Instruction to the AI according to which standards it evaluates the criterion
• Knockout criterion for category – If this criterion is evaluated with 0, the entire category is set to 0
• Knockout criterion for assessment – If this criterion is rated with 0, the overall assessment is set to 0

​

Knockout Criteria

Knockout criteria allow individual requirements to be marked as absolute. The system distinguishes two levels:

• Category level: If a knockout criterion within a category is not met (Score = 0), the entire category is rated with 0 points – regardless of the other criteria.
• Assessment level: If such a criterion is not met, the total assessment drops to 0 – regardless of all other results.

This makes sense for criteria that represent legal, regulatory, or safety-related minimum requirements, e.g., data protection notices or security queries.

​

Live Quality Coach

The Live Quality Coach option determines whether a scorecard is also used for quality support during ticket processing.

• On: The scorecard can be used as a basis for live hints. The AI checks the current draft response based on the activated criteria and can provide hints on which quality requirements have already been met or should still be improved.
• Off: The scorecard is not used for live coaching. However, it can still be used for regular quality assessments of work processes.

The Live Quality Coach is particularly helpful when employees should receive feedback on the quality of their response text before sending a response. For purely downstream quality control, the option can remain disabled.

​

AI-based Preliminary Assessment

Criteria with AI review activated are automatically pre-assessed by the AI as soon as an assessment is processed. The AI takes into account the configured review prompt. The result has the status aiGenerated and can be checked and adjusted by the supervisor. After a manual adjustment, the status changes to humanVerified. Criteria without AI review initially remain unscored and must be fully manually evaluated.

​

Assignment

Scorecards can be restricted to certain work procedures. The assignment is made via a combination of:

• Ticket tags – The scorecard is only applied to tickets with certain tags
• Teams – Only work processes from members of certain teams are assessed
• Channels - Limitation to specific incoming channels (e.g., e-mail, phone)

The conditions act as an AND link: All configured criteria must apply at the same time. If no assignment is configured, the scorecard applies to all work processes.

​

Automatic creation of Evaluations

The system checks every minute whether there are new work procedures that have not yet been evaluated. For each such process, it is checked which active scorecards match according to the assignment rules. For each matching scorecard, an assessment is automatically created and submitted for AI processing. A work process can be assessed simultaneously by several scorecards if several assignment rules apply.

​

Versioning

Scorecards are versioned. Each scorecard has a baseId (constant over all versions) and a revision (continuous). If an active scorecard is edited, the previous version is set to retired and a new revision with the status active is created. Existing assessments remain linked to the scorecard version that was active at the time of their creation. A change in the scorecard only affects newly created assessments.

​

Scorecard States

​

Assessment Workflow

Assessments go through the following states:

1. unprocessed – Assessment was created, AI processing is pending
2. aiInProgress – AI is currently evaluating the criteria
3. aiReady – AI preassessment completed, ready for review
4. reviewOngoing – Supervisor has started the review
5. reviewedBySupervisor – Review completed
6. discussedWithAssessee – Assessment discussed with the evaluated employee

If a supervisor makes changes to an already completed assessment (reviewedBySupervisor or discussedWithAssessee), the status automatically switches back to reviewOngoing.

​

Quality Assessment Preview

With the Quality Assessment Preview, a scorecard can be tested before it is used in production or activated after changes. Typically, two details are required for the preview:

• Ticket ID: The ticket provides the technical context, e.g., customer request, history, and relevant ticket data.
• Response Text: The text to be assessed – for example, a draft response from an employee or a sample text for testing purposes.

The system evaluates the specified response text based on the scorecard criteria and shows a preview of the expected assessment. This allows testing whether the criteria, review prompts, weighting, and knockout rules work as desired. The preview does not create a regular assessment in the quality workflow. It serves exclusively for testing and refining the scorecard configuration.

​

Best Practices

• Granularity of Criteria: Criteria should be formulated in such a way that they can be clearly evaluated by the AI. Vague wording leads to unreliable AI assessments.
• Use Knockout Criteria Sparingly: They have a strong impact on the overall result. Only truly non-negotiable requirements should be marked as such.
• Configure Assignment Rules Carefully: Overly broad assignments significantly increase the assessment volume. If different teams or channels require different standards, it is advisable to have separate scorecards for each one.
• Consider Versioning: Changes to an active scorecard always create a new revision. For ongoing comparative analyses, it’s relevant which revision forms the basis of an assessment.