The Quality Assurance of AI capabilities is based on Test Cases and Test Runs. A thorough Verification of Functionality is essential, in particular after systemic changes or adaptations to AI Agents. Test Runs provide a structured validation, detect potential errors at an early stage, and ensure a consistent Performance Evaluation of AI Agents.

​
Test Cases

A Test Case describes a single scenario for verifying the performance of an AI Agent. By adding specific tickets, different use cases can be tested to ensure that the AI Agent responds as expected. Creation:
  • In the area Quality Testing - Test Cases, open the sidebar for the desired AI Agent.
  • Add the tickets intended for the test via ticket ID.
  • If a specific ticket ID is not yet available:
    • In the filter of the ticket overview, select the point**** “Issue”.
    • Display appropriate tickets and add them to the test run by ID.
  • Test cases can be removed via the sidebar if necessary.

​
Test Runs

A Test Run carries out a defined number of test cases to evaluate the AI functionality.
Test Runs do not cause any write operations.
Test Runs Overview The overview displays all conducted test runs with the following details:
  • ID of the Test Run: Unique identification for the test run
  • AI-Agent: Name of the tested AI Agent
  • Start date: Timestamp of the test run
  • Status: Indicates the progress (e.g., “Processing”, “Completed”, “Failed”)
  • Tickets: Number of tickets used in the test run
  • Results: Percentage evaluation of the test results
Test runs are initiated via the Play button in the “Test Cases” tab to the right of the AI Agent. Any number of test runs can be initiated in parallel.