The Quality Assurance of AI capabilities is based on Test Cases and Test Runs. A thorough Verification of Functionality is essential, in particular after systemic changes or adaptations to AI Agents. Test Runs provide a structured validation, detect potential errors at an early stage, and ensure a consistent Performance Evaluation of AI Agents.
Test Cases
A Test Case describes a single scenario for verifying the performance of an AI Agent. By adding specific tickets, different use cases can be tested to ensure that the AI Agent responds as expected. Creation:- In the area Quality Testing - Test Cases, open the sidebar for the desired AI Agent.
- Add the tickets intended for the test via ticket ID.
-
If a specific ticket ID is not yet available:
- In the filter of the ticket overview, select the point**** “Issue”.
- Display appropriate tickets and add them to the test run by ID.
- Test cases can be removed via the sidebar if necessary.
Test Runs
A Test Run carries out a defined number of test cases to evaluate the AI functionality.Test Runs do not cause any write operations.
- ID of the Test Run: Unique identification for the test run
- AI-Agent: Name of the tested AI Agent
- Start date: Timestamp of the test run
- Status: Indicates the progress (e.g., “Processing”, “Completed”, “Failed”)
- Tickets: Number of tickets used in the test run
- Results: Percentage evaluation of the test results